cinera_handmade.network/cmuratori/hero/code/code458.hmml

[video output=day458 member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Debugging the PNG Reader" vod_platform=youtube id=iqpGuyc308w annotator=Miblo]
[0:01][Recap and set the stage for the day debugging our PNG reader][:speech]
[1:01][On starting by making our decompressor produce the correct number of bytes from our input stream][:compression :speech]
[4:14][Approaching problems either by working from a spec, or comparing our implementation with a working version][:speech]
[6:00][:Run the program on gimp_test.png until a crash caused by the Dest pointer's attempt to write into :memory][:compression]
[13:25][Let ParsePNG() proceed to the next iteration of the loop, to see another length code 258][:compression :parsing :run]
[15:38][Compare our PNGLengthExtra and PNGDistExtra tables with the DEFLATE spec[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt] to ensure they look right][:compression :research]
[21:10][3.2.5 Compressed blocks (length and distance codes)[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :research]
[23:10][Scrutinise the length table building code in ParsePNG()][:compression :research]
[25:14][Assert in ParsePNG() that the LenCount <= number of items in the LitLenDistTable][:compression]
[25:33][Continue to scrutinise the Huffman decoding code in ParsePNG()][:compression :research]
[28:24][Assert in ParsePNG() that the DecompressedPixelsEnd does not get exceeded][:compression]
[29:21][:Run it and hit our Source + Len assertion][:compression]
[30:13][Consider investigating the Huffman table calculation, before determining to nail down the literal length stuff][:compression :speech]
[31:24][Produce a smaller image that exhibits the bug][:art :drawing]
[39:27][Note that our bug exhibiting image contains multiple IDAT chunks][:compression :run]
[42:00][Produce a 64×64 pixel image that probably can't be PNG compressed][:art :drawing]
[44:52][:Run it on gimp_64x64.png to see that the image contains two IDAT chunks and exhibits the crash][:compression]
[45:20][5.3 "Chunk layout" and 11.2.4 "IDAT Image data"[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:compression :parsing :research]
[48:55][10.2 Compression of the sequence of filtered scanlines[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:compression :parsing :research]
[50:17][:Run it and hit an assertion in HuffmanDecode() due to an invalid symbol][:compression :parsing :run]
[53:28][Step through ConsumeSize() to realise that PeekBits() doesn't correctly operate across multiple IDAT chunks][:compression :run]
[54:09][Fix PeekBits() and ConsumeSize() to correctly operate across multiple IDAT chunks][:parsing]
[57:30][:Run it and again crash in HuffmanDecode() on an invalid symbol, although we did process more bits][:compression :parsing]
[59:05][Crash ScriptedSandbox64.exe][:admin]
[59:52][Continue to investigate our bug exhibited by gimp_64x64.png][:compression :parsing :run]
[1:02:51][Assert at the end of ParsePNG() that the Dest == DecompressedPixels][:compression :parsing]
[1:03:16][:Run it and do not hit that assertion][:compression :parsing]
[1:03:34][Determine that gimp_64x64.png crashes 9 bytes from the end, and investigate why][:compression :parsing :run]
[1:05:30][Make PeekBits() print out BitBufferBeforeAdvance if it needs to advance the buffer][:parsing]
[1:07:05][:Run it to see that there are 12 bits in the buffer but, more pressingly, that we do not handle fixed Huffman compressed chunks][:compression :parsing]
[1:08:06][Enable ParsePNG() to handle fixed Huffman compressed chunks[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing]
[1:17:51][Step into ParsePNG() and inspect the LitLenDistTable comparing it with the PNG spec[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing :run]
[1:19:17][Determine to perform some difference tests][:compression :parsing :speech]
[1:21:03][Grab a screenshot of the stream[ref
    site=Twitch
    page="Handmade Hero"
    url=https://twitch.tv/handmade_hero]][:research]
[1:22:43][:Run okay on our captured screenshot][:parsing :compression]
[1:23:06][Q&A][:speech]
[1:23:38][@mtsmox][Q: Yeah, same bug]
[1:23:52][@vaualbus][Q: You could use the ray casting code for save image files so we can see what happens. It should be a quick thing to do!]
[1:24:12][:Run the PNG reader successfully on our ray tracing image][:compression :parsing]
[1:25:49][Credit @rooctag for clarifying our understanding of the Paeth filter][:speech]
[1:26:33][@xxthebigfoxx][Q: Did you purposely screenshot my message saying you are handsome?]
[1:26:42][@frostyninja][Q: I think he meant save out the decoded PNGs out as a raw BMP so we can see the result?]
[1:27:24][Pull WriteImage() in from ray.cpp and make ParsePNG() return an image_u32 for us to write out][:"file io"]
[1:32:27][Inspect our written image to see that it is upside-down and incorrectly coloured][:art :drawing]
[1:33:15][Rename WriteImage() to WriteImageTopDownRGBA() and enable it to swap the rows, introducing SwapRAndB() to swap those colour channels][:rendering]
[1:39:47][Compare our written image with the original, to see that it is off-by-1 pixel vertically][:art :drawing]
[1:43:05][Fix typo in WriteImageTopDownRGBA()][:rendering]
[1:43:20][Compare our written image with the original to see that they match][:art :drawing]
[1:44:09][@bobby1up][Q: Isn't it true that the compiler's optimizations of your code and the way the CPU works can sometimes cause instructions to happen in a different order than you wrote them? If so, won't that screw up :profiling measurements?]
[1:44:43][@tavqua][Q: Can you quantify how fast you type?][:trivia]
[1:44:59][@jacksonbanan][Q: Would you say that ~4coder is the ideal text editor?]
[1:45:32][@jacksonbanan][Q: Also, how often do you work out? You're buff][:trivia]
[1:45:40][@lkalinovcic][Q: In the BMP write routine, I think you have a bug. You advance Row0 too soon]
[1:45:44][@enemymouse][Q: Double-check the shifting is not a GIMP paster error]
[1:45:57][@jnog92][Q: Just joined a week ago and haven't had time to go through all the series. Can you say in overall what have you done on the project and what is left to do?]
[1:47:10][@bbbyan][Q: Do you have any project recommendations for beginner-intermediate C programmers?]
[1:47:46][@ivereadthesequel][Q: Seems a bit casual today, what's your favorite album?][:trivia]
[1:49:54][@jessef][Q: What is the craziest file format to parse, in your opinion?][:parsing]
[1:50:54][@vaualbus][Q: What is the most difficult file format to read? Have you ever try to read ttf files? And try doing a vector renderer for those in OpenGL?]
[1:51:08][@frostyninja][Q: What's your favourite pasta dish?][:trivia]
[1:51:41][@jacksonbanan][Q: How come no one has made a solid debugger? Is it difficult?]
[1:52:34][@runamar][Q: Will the game be only room based?]
[1:52:43][@quickshift_][Q: Bit off topic, but are you a musician? Didn't you write some music for Handmade Hero yourself?][:trivia]
[1:52:54][@tavqua][Q: This may be too in-depth of a question, but: how can I send data over the internet for a game that can be picked up in a code that I write][:networking]
[1:53:18][@thejimjames40][Q: Will we be :parsing compiler debug output (DWARF?) for cool in-game debug stuff?]
[1:53:27][@enemymouse][Q: 3ds is worse than psd?]
[1:53:47][@bbbyan][Q: We've had a Handmade Ray bonus series, what about Handmade Asteroids?]
[1:53:52][Call it a day][:speech]
[/video]