diff --git a/cmuratori/hero/code/code457.hmml b/cmuratori/hero/code/code457.hmml new file mode 100644 index 0000000..c783a1a --- /dev/null +++ b/cmuratori/hero/code/code457.hmml @@ -0,0 +1,70 @@ +[video member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Implementing PNG Reconstruction Filters" vod_platform=youtube id=M27KjGYbWvs annotator=Miblo] +[0:00][Recap and set the stage for the day continuing with our PNG reader][:compression :parsing :speech] +[2:44][Our understanding of the DEFLATE encoding][:compression :parsing :speech] +[6:00][Pull up our piece of structured :art][:drawing] +[7:43][Create a simpler piece of structured :art][:drawing] +[11:33][A few thoughts on structured :art][:speech] +[12:33][Add a BYTE == 1 case in ParsePNG() by advice of @mmozeiko][:compression :parsing] +[13:51][:Run our PNG reader on gimp_8x8_compression0.png to see which :compression case we hit in ParsePNG()][:parsing] +[16:19][Prevent ParsePNG() from performing integral promotion on the LEN != NLEN comparison][:parsing] +[16:43][Continue to step through ParsePNG()][:compression :parsing :run] +[18:32][Prevent ParsePNG() from calling EndianSwap() on the LEN][:memory :parsing] +[18:53][Continue to step through ParsePNG() to completion, and inspect the DecompressedPixels][:compression :parsing :run] +[24:57][Deduce that the DecompressedPixels contain row transform filter indicators that we have not yet learnt][:compression :parsing :run] +[26:56][Rename DecompressedPixels to FinalPixels, initialising it before the :parsing loop in ParsePNG()] +[27:59][Crash ~4coder and step into it to see what's going on][:admin] +[30:10][Continue setting up ParsePNG() to apply the filters to the FinalPixels][:parsing] +[33:27][4.3 Reference image to PNG image transformation[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research] +[40:27][9 Filtering (PNG)[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research] +[49:05][Finish setting up ParsePNG() to apply the filters to our parsed pixels, noting that it seems tailor-made for SIMD[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :performance] +[1:08:10][Introduce our PNGFilter*() functions[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing] +[1:22:16][Introduce PNGFilterReconstruct() to perform the filtering part of ParsePNG()][:parsing] +[1:26:03][Step through PNGFilterReconstruct() to see what we end up with][:parsing :run] +[1:29:32][Add assertions in the untested Average and Paeth filter cases in PNGFilterReconstruct()][:parsing] +[1:30:55][:Run the program on the compressed gimp_8x8.png to see that it looks right][:compression :parsing] +[1:39:16][Draw a more complicated image][:art :drawing] +[1:41:00][:Run it on our more complicated, uncompressed image and hit the Paeth filter] +[1:42:40][Remove that assertion from the Paeth filter case in PNGFilterReconstruct()][:parsing] +[1:42:53][Step through PNGFilterReconstruct() into PNGFilter4() (Paeth) to see that it looks reasonable][:parsing :run] +[1:46:21][:Run it on our compressed complex image, to see that it look good too][:parsing] +[1:46:57][:Run it on our fully complex gimp_test.png and crash in ParsePNG()][:parsing] +[1:48:41][Step through ConsumeSize() to realise that the crash occurs even in the first IDAT chunk, so the bug must be in the Huffman decode][:parsing] +[1:53:53][Perform coverage checking on every EncodedLen case in ParsePNG() to find that the 16-long (0x10) case is never used in the successfully parsed file][:parsing :run] +[1:58:56][3.2.7 Compression with dynamic Huffman codes BTYE=10[ref + site=IETF + page="DEFLATE Compressed Data Format Specification version 1.3" + url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :research] +[2:01:44][Step through the BTYPE=10 case in ParsePNG() to see that the LitLenDistTable seems to be built correctly][:compression :parsing :run] +[2:02:40][Consider our current situation and ways to proceed][:parsing :speech] +[2:03:18][Q&A][:speech] +[2:03:38][@ratchetfreak][Q: The LEN, NLEN is so the decompressor can resync with a compressor if a block got corrupted (and the compressor emitted an empty block) by looking for the bytes 0000FFFF, mostly handy for byte streams][:compression] +[2:04:09][@rooctag][Q: C is up and left[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing] +[2:05:26][Fix PNGFilterReconstruct() to correctly filter Paeth][:parsing] +[2:06:24][@geekengale][Q: Did I see for(;;) in your code? What's that for / how does it work?][:language] +[2:07:56][@john_diresta][Q: In the :memory window when you were writing the "decoded" pixel (non-compressed 8x8) you were writing to some memory that you have not allocated, accordingly to the ASCII representation, there were some strings about a path in windows] +[2:09:16][@Brian][Q: In Day 055 when you create the hash table, you added your TODO on getting a better hash. Other than for educational purposes investigating better hashes, what kinds of problems would you start to see if the hash was not good enough? Like would it pop-up when you are profiling or something? Would you get any errors?][:"data structure"] +[2:11:09][@sharlock93][Q: There is a VS plugin that lets you see images in memory. It's called Image Watch. Was pretty useful when I did the PNG decoding][:parsing] +[2:11:38][@john_diresta][Q: Oh okay, it's that it looked strange. Usually it's filled with 0xDD of 0xCD if I remember correctly, or maybe it's just for stack :memory?] +[2:13:22][@uplinkcoder][Q: Try a big empty image which compresses very well] +[2:14:49][@quickshift_][Q: So on Linux, would you have to use calloc() instead of VirtualAlloc() or is there an alternative?[ref + site="Linux Programmer's Manual" + page="mmap(2)" + url=http://man7.org/linux/man-pages/man2/mmap.2.html]][:memory] +[2:15:32][@sgtrumbi][Q: Will you integrate the decoder directly into the game or into the simple_preprocessor?] +[2:16:04][Close it up with a glimpse into the future][:speech] +[/video]