From 047649266d1079a44c7aec8f9aac38520b7d54a0 Mon Sep 17 00:00:00 2001 From: Matt Mascarenhas Date: Tue, 26 Jun 2018 20:19:32 +0100 Subject: [PATCH] Annotate hero/code458 --- cmuratori/hero/code/code458.hmml | 91 ++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 cmuratori/hero/code/code458.hmml diff --git a/cmuratori/hero/code/code458.hmml b/cmuratori/hero/code/code458.hmml new file mode 100644 index 0000000..0b73589 --- /dev/null +++ b/cmuratori/hero/code/code458.hmml @@ -0,0 +1,91 @@ +[video member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Debugging the PNG Reader" vod_platform=youtube id=iqpGuyc308w annotator=Miblo] +[0:01][Recap and set the stage for the day debugging our PNG reader][:speech] +[1:01][On starting by making our decompressor produce the correct number of bytes from our input stream][:compression :speech] +[4:14][Approaching problems either by working from a spec, or comparing our implementation with a working version][:speech] +[6:00][:Run the program on gimp_test.png until a crash caused by the Dest pointer's attempt to write into :memory][:compression] +[13:25][Let ParsePNG() proceed to the next iteration of the loop, to see another length code 258][:compression :parsing :run] +[15:38][Compare our PNGLengthExtra and PNGDistExtra tables with the DEFLATE spec[ref + site=IETF + page="DEFLATE Compressed Data Format Specification version 1.3" + url=https://www.ietf.org/rfc/rfc1951.txt] to ensure they look right][:compression :research] +[21:10][3.2.5 Compressed blocks (length and distance codes)[ref + site=IETF + page="DEFLATE Compressed Data Format Specification version 1.3" + url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :research] +[23:10][Scrutinise the length table building code in ParsePNG()][:compression :research] +[25:14][Assert in ParsePNG() that the LenCount <= number of items in the LitLenDistTable][:compression] +[25:33][Continue to scrutinise the Huffman decoding code in ParsePNG()][:compression :research] +[28:24][Assert in ParsePNG() that the DecompressedPixelsEnd does not get exceeded][:compression] +[29:21][:Run it and hit our Source + Len assertion][:compression] +[30:13][Consider investigating the Huffman table calculation, before determining to nail down the literal length stuff][:compression :speech] +[31:24][Produce a smaller image that exhibits the bug][:art :drawing] +[39:27][Note that our bug exhibiting image contains multiple IDAT chunks][:compression :run] +[42:00][Produce a 64×64 pixel image that probably can't be PNG compressed][:art :drawing] +[44:52][:Run it on gimp_64x64.png to see that the image contains two IDAT chunks and exhibits the crash][:compression] +[45:20][5.3 "Chunk layout" and 11.2.4 "IDAT Image data"[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:compression :parsing :research] +[48:55][10.2 Compression of the sequence of filtered scanlines[ref + site=W3C + page="Portable Network Graphics (PNG) Specification (Second Edition)" + url=https://w3.org/TR/2003/REC-PNG-20031110/]][:compression :parsing :research] +[50:17][:Run it and hit an assertion in HuffmanDecode() due to an invalid symbol][:compression :parsing :run] +[53:28][Step through ConsumeSize() to realise that PeekBits() doesn't correctly operate across multiple IDAT chunks][:compression :run] +[54:09][Fix PeekBits() and ConsumeSize() to correctly operate across multiple IDAT chunks][:parsing] +[57:30][:Run it and again crash in HuffmanDecode() on an invalid symbol, although we did process more bits][:compression :parsing] +[59:05][Crash ScriptedSandbox64.exe][:admin] +[59:52][Continue to investigate our bug exhibited by gimp_64x64.png][:compression :parsing :run] +[1:02:51][Assert at the end of ParsePNG() that the Dest == DecompressedPixels][:compression :parsing] +[1:03:16][:Run it and do not hit that assertion][:compression :parsing] +[1:03:34][Determine that gimp_64x64.png crashes 9 bytes from the end, and investigate why][:compression :parsing :run] +[1:05:30][Make PeekBits() print out BitBufferBeforeAdvance if it needs to advance the buffer][:parsing] +[1:07:05][:Run it to see that there are 12 bits in the buffer but, more pressingly, that we do not handle fixed Huffman compressed chunks][:compression :parsing] +[1:08:06][Enable ParsePNG() to handle fixed Huffman compressed chunks[ref + site=IETF + page="DEFLATE Compressed Data Format Specification version 1.3" + url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing] +[1:17:51][Step into ParsePNG() and inspect the LitLenDistTable comparing it with the PNG spec[ref + site=IETF + page="DEFLATE Compressed Data Format Specification version 1.3" + url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing :run] +[1:19:17][Determine to perform some difference tests][:compression :parsing :speech] +[1:21:03][Grab a screenshot of the stream[ref + site=Twitch + page="Handmade Hero" + url=https://twitch.tv/handmade_hero]][:research] +[1:22:43][:Run okay on our captured screenshot][:parsing :compression] +[1:23:06][Q&A][:speech] +[1:23:38][@mtsmox][Q: Yeah, same bug] +[1:23:52][@vaualbus][Q: You could use the ray casting code for save image files so we can see what happens. It should be a quick thing to do!] +[1:24:12][:Run the PNG reader successfully on our ray tracing image][:compression :parsing] +[1:25:49][Credit @rooctag for clarifying our understanding of the Paeth filter][:speech] +[1:26:33][@xxthebigfoxx][Q: Did you purposely screenshot my message saying you are handsome?] +[1:26:42][@frostyninja][Q: I think he meant save out the decoded PNGs out as a raw BMP so we can see the result?] +[1:27:24][Pull WriteImage() in from ray.cpp and make ParsePNG() return an image_u32 for us to write out][:"file io"] +[1:32:27][Inspect our written image to see that it is upside-down and incorrectly coloured][:art :drawing] +[1:33:15][Rename WriteImage() to WriteImageTopDownRGBA() and enable it to swap the rows, introducing SwapRAndB() to swap those colour channels][:rendering] +[1:39:47][Compare our written image with the original, to see that it is off-by-1 pixel vertically][:art :drawing] +[1:43:05][Fix typo in WriteImageTopDownRGBA()][:rendering] +[1:43:20][Compare our written image with the original to see that they match][:art :drawing] +[1:44:09][@bobby1up][Q: Isn't it true that the compiler's optimizations of your code and the way the CPU works can sometimes cause instructions to happen in a different order than you wrote them? If so, won't that screw up :profiling measurements?] +[1:44:43][@tavqua][Q: Can you quantify how fast you type?][:trivia] +[1:44:59][@jacksonbanan][Q: Would you say that ~4coder is the ideal text editor?] +[1:45:32][@jacksonbanan][Q: Also, how often do you work out? You're buff][:trivia] +[1:45:40][@lkalinovcic][Q: In the BMP write routine, I think you have a bug. You advance Row0 too soon] +[1:45:44][@enemymouse][Q: Double-check the shifting is not a GIMP paster error] +[1:45:57][@jnog92][Q: Just joined a week ago and haven't had time to go through all the series. Can you say in overall what have you done on the project and what is left to do?] +[1:47:10][@bbbyan][Q: Do you have any project recommendations for beginner-intermediate C programmers?] +[1:47:46][@ivereadthesequel][Q: Seems a bit casual today, what's your favorite album?][:trivia] +[1:49:54][@jessef][Q: What is the craziest file format to parse, in your opinion?][:parsing] +[1:50:54][@vaualbus][Q: What is the most difficult file format to read? Have you ever try to read ttf files? And try doing a vector renderer for those in OpenGL?] +[1:51:08][@frostyninja][Q: What's your favourite pasta dish?][:trivia] +[1:51:41][@jacksonbanan][Q: How come no one has made a solid debugger? Is it difficult?] +[1:52:34][@runamar][Q: Will the game be only room based?] +[1:52:43][@quickshift_][Q: Bit off topic, but are you a musician? Didn't you write some music for Handmade Hero yourself?][:trivia] +[1:52:54][@tavqua][Q: This may be too in-depth of a question, but: how can I send data over the internet for a game that can be picked up in a code that I write][:networking] +[1:53:18][@thejimjames40][Q: Will we be :parsing compiler debug output (DWARF?) for cool in-game debug stuff?] +[1:53:27][@enemymouse][Q: 3ds is worse than psd?] +[1:53:47][@bbbyan][Q: We've had a Handmade Ray bonus series, what about Handmade Asteroids?] +[1:53:52][Call it a day][:speech] +[/video]