[video output=day457 member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Implementing PNG Reconstruction Filters" vod_platform=youtube id=M27KjGYbWvs annotator=Miblo]
[0:00][Recap and set the stage for the day continuing with our PNG reader][:compression :parsing :speech]
[2:44][Our understanding of the DEFLATE encoding][:compression :parsing :speech]
[6:00][Pull up our piece of structured :art][:drawing]
[7:43][Create a simpler piece of structured :art][:drawing]
[11:33][A few thoughts on structured :art][:speech]
[12:33][Add a BYTE == 1 case in ParsePNG() by advice of @mmozeiko][:compression :parsing]
[13:51][:Run our PNG reader on gimp_8x8_compression0.png to see which :compression case we hit in ParsePNG()][:parsing]
[16:19][Prevent ParsePNG() from performing integral promotion on the LEN != NLEN comparison][:parsing]
[16:43][Continue to step through ParsePNG()][:compression :parsing :run]
[18:32][Prevent ParsePNG() from calling EndianSwap() on the LEN][:memory :parsing]
[18:53][Continue to step through ParsePNG() to completion, and inspect the DecompressedPixels][:compression :parsing :run]
[24:57][Deduce that the DecompressedPixels contain row transform filter indicators that we have not yet learnt][:compression :parsing :run]
[26:56][Rename DecompressedPixels to FinalPixels, initialising it before the :parsing loop in ParsePNG()]
[27:59][Crash ~4coder and step into it to see what's going on][:admin]
[30:10][Continue setting up ParsePNG() to apply the filters to the FinalPixels][:parsing]
[33:27][4.3 Reference image to PNG image transformation[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research]
[40:27][9 Filtering (PNG)[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research]
[49:05][Finish setting up ParsePNG() to apply the filters to our parsed pixels, noting that it seems tailor-made for SIMD[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :performance]
[1:08:10][Introduce our PNGFilter*() functions[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing]
[1:22:16][Introduce PNGFilterReconstruct() to perform the filtering part of ParsePNG()][:parsing]
[1:26:03][Step through PNGFilterReconstruct() to see what we end up with][:parsing :run]
[1:29:32][Add assertions in the untested Average and Paeth filter cases in PNGFilterReconstruct()][:parsing]
[1:30:55][:Run the program on the compressed gimp_8x8.png to see that it looks right][:compression :parsing]
[1:39:16][Draw a more complicated image][:art :drawing]
[1:41:00][:Run it on our more complicated, uncompressed image and hit the Paeth filter]
[1:42:40][Remove that assertion from the Paeth filter case in PNGFilterReconstruct()][:parsing]
[1:42:53][Step through PNGFilterReconstruct() into PNGFilter4() (Paeth) to see that it looks reasonable][:parsing :run]
[1:46:21][:Run it on our compressed complex image, to see that it look good too][:parsing]
[1:46:57][:Run it on our fully complex gimp_test.png and crash in ParsePNG()][:parsing]
[1:48:41][Step through ConsumeSize() to realise that the crash occurs even in the first IDAT chunk, so the bug must be in the Huffman decode][:parsing]
[1:53:53][Perform coverage checking on every EncodedLen case in ParsePNG() to find that the 16-long (0x10) case is never used in the successfully parsed file][:parsing :run]
[1:58:56][3.2.7 Compression with dynamic Huffman codes BTYE=10[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :research]
[2:01:44][Step through the BTYPE=10 case in ParsePNG() to see that the LitLenDistTable seems to be built correctly][:compression :parsing :run]
[2:02:40][Consider our current situation and ways to proceed][:parsing :speech]
[2:03:18][Q&A][:speech]
[2:03:38][@ratchetfreak][Q: The LEN, NLEN is so the decompressor can resync with a compressor if a block got corrupted (and the compressor emitted an empty block) by looking for the bytes 0000FFFF, mostly handy for byte streams][:compression]
[2:04:09][@rooctag][Q: C is up and left[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing]
[2:05:26][Fix PNGFilterReconstruct() to correctly filter Paeth][:parsing]
[2:06:24][@geekengale][Q: Did I see for(;;) in your code? What's that for / how does it work?][:language]
[2:07:56][@john_diresta][Q: In the :memory window when you were writing the "decoded" pixel (non-compressed 8x8) you were writing to some memory that you have not allocated, accordingly to the ASCII representation, there were some strings about a path in windows]
[2:09:16][@Brian][Q: In Day 055 when you create the hash table, you added your TODO on getting a better hash. Other than for educational purposes investigating better hashes, what kinds of problems would you start to see if the hash was not good enough? Like would it pop-up when you are profiling or something? Would you get any errors?][:"data structure"]
[2:11:09][@sharlock93][Q: There is a VS plugin that lets you see images in memory. It's called Image Watch. Was pretty useful when I did the PNG decoding][:parsing]
[2:11:38][@john_diresta][Q: Oh okay, it's that it looked strange. Usually it's filled with 0xDD of 0xCD if I remember correctly, or maybe it's just for stack :memory?]
[2:13:22][@uplinkcoder][Q: Try a big empty image which compresses very well]
[2:14:49][@quickshift_][Q: So on Linux, would you have to use calloc() instead of VirtualAlloc() or is there an alternative?[ref
    site="Linux Programmer's Manual"
    page="mmap(2)"
    url=http://man7.org/linux/man-pages/man2/mmap.2.html]][:memory]
[2:15:32][@sgtrumbi][Q: Will you integrate the decoder directly into the game or into the simple_preprocessor?]
[2:16:04][Close it up with a glimpse into the future][:speech]
[/video]