cinera_handmade.network/cmuratori/hero/code/code454.hmml

[video output=day454 member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Parsing ZLIB Headers" vod_platform=youtube id=nQ0ctmQPs-E annotator=Miblo]
[0:00][Welcome to the stream][:speech]
[0:21][Recap and set the stage for :parsing PNG IDAT data][:speech]
[1:18][:Run the program to look at our IDAT chunks][:parsing]
[3:58][Set up to parse IDAT chunks[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research]
[6:44][Prepare ReadEntireFile() and ConsumeSize() to read the .png file into a linked list of streaming_chunks][:"file io" :parsing]
[13:38][:Run it to see that it works][:"file io" :parsing]
[13:39][Enable ParsePNG() to build up a linked list of IDAT chunks, introducing AllocateChunk()][:memory :parsing]
[19:09][Describe this obtuse, single-expression singly linked list append][:memory :research]
[20:11][Enable ParsePNG() to parse the IDATHeader][:parsing]
[21:19][:Run it to see our decompressed IDAT chunks][:parsing]
[21:30][Make ParsePNG() print the IDAT chunk sizes][:parsing]
[21:45][:Run it see the sizes of the IDAT chunks and determine that GIMP is outputting in 8192-byte chunks][:parsing]
[23:30][Consult the PNG[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/] and ZLIB specs[ref
        site=IETF
        page="ZLIB Compressed Data Format Specification version 3.3"
        url=http://www.ietf.org/rfc/rfc1950.txt] to understand what we need to support][:compression :parsing :research]
[25:19][Further condense ParsePNG() to support :compression method 8 (DEFLATE) and no preset dictionary][:parsing]
[26:21][:Run it to see our "Supported" message][:parsing]
[26:27][Begin to make ParsePNG() decompress the IDAT data using DEFLATE[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing]
[35:19][LZ-Style Compressors][:blackboard :compression]
[40:55][Run-Length Encoding as LZ Mode 2][:blackboard :compression]
[44:45][Enable ParsePNG() to LZ-decompress the IDAT data[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing]
[49:37][Read about non-compressed and compressed blocks, including Huffman codes, in the DEFLATE spec[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :research]
[57:56][Implement ConsumeBits() and FlushByte() and enable ParsePNG() to parse BTYPE 00 chunks[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :memory :parsing]
[1:09:13][Step through ParsePNG() to see our consumed bits][:compression :memory :parsing :run]
[1:10:36][Read about dynamic Huffman codes (BTYPE 10)[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing :research]
[1:13:13][Enable ParsePNG() to handle dynamic Huffman coded (BTYPE 10) chunks[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing]
[1:32:44][Step through ParsePNG() to inspect our HLIT, HDIST and HCLEN, and realise that the code lengths are swizzled from most to least likely][:compression :parsing :run]
[1:36:25][Set up ParsePNG() to build up our Huffman table, introducing stubs for ComputeHuffman() and HuffmanDecode()[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing]
[1:52:40][Q&A][:speech]
[1:53:41][@lorymaster][Q: I'm happy you're doing this. I've had problems making my inflate decoder (for fun) working, and you being a spec decoder works out perfectly][:compression]
[1:53:57][@ratchetfreak][Q: I believe the length for the LZ should be interpreted like u32 copy_length = length_table_base\[LitLen-257\] + ConsumeBits(length_table_extra_bits\[LitLen - 257\]); And the distance is similar with the other table(s)][:compression]
[1:54:47][@Brian][Q: How have you found the png and zlib specification, compared to other specifications you have read and implemented?]
[1:55:51][@gg_nate][Q: Any way to fix the pink flashing? It was fairly persistent today]
[1:56:09][@bigmofo1][Q: Will Yangtian iterate over the :art live?]
[1:56:46][@Brian][Q: What do you do when you have an incomplete specification? Is it then more just trial and error, or are there techniques on how to continue?]
[1:59:12][@jim0_o][Q: What ~4coder version are you using on stream?]
[1:59:56][@rationalcoder][Q: Just caught up with the stream, but I have had a question for a while: When you come up with a reusable data structure and want to share it between projects / other people, how do you deal with custom allocation? I have been writing some hybrid data structures, and I want to add custom allocation, but adding a template parameter seems like a bad solution to the problem][:memory]
[2:00:19][@jim0_o][Q: I see it in the *Message* window that you don't get at start up apparently]
[2:00:30][Custom :memory allocation in Granny][:speech]
[2:04:03][@snovind92][Q: Why does no one spec the simplest solution possible, so people can easily implement the parsers and exporters they need (in cases when space / speed does not matter)?]
[2:07:01][@flyinginthedark][Q: Are you going to do :audio formats next?]
[2:07:47][We're all done][:speech]
[/video]