cinera_handmade.network/cmuratori/hero/code/code453.hmml

[video member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Parsing PNG Headers" vod_platform=youtube id=lkEWbIUEuN0 annotator=Miblo]
[0:00][Recap and set the stage for the day][:speech]
[0:16][:Run the game to show the current :camera and :lighting situation, and consider our next steps][:rendering]
[2:22][Determine to support hot-loadable :art assets in a readily usable format by [@aerettberg Anna], i.e. PNG][:run]
[4:53][Consider our current BMP-loading code, with a view to implementing minimal PNG-loading][:art :research]
[9:24][Install GIMP[ref
    site=GIMP
    page=Download
    url=https://www.gimp.org/downloads/]][:admin]
[11:43][Bring up the PNG specification[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/][ref
        site=libpng.org
        page="PNG (Portable Network Graphics) Specification, Version 1.2"
        url=http://www.libpng.org/pub/png/spec/1.2/PNG-Contents.html]][:research]
[12:56][Create a piece of structured :art as a 1024x1024 8bpc RGBA PNG called gimp_test.png][:drawing]
[19:33][Embark on a PNG parser, creating handmade_png.cpp]
[21:59][:Run it to see our printf]
[22:09][Grab ReadEntireFile() from test_asset_builder.cpp][:"file io"]
[23:18][:Run it to see it in action]
[23:34][Introduce ParsePNG(), noting that it is not meant to be fault tolerant][:parsing]
[25:44][Setup the handmade_png.cpp project in MSVC][:admin]
[27:02][Step through ReadEntireFile() and inspect its Result][:run]
[27:45][Create handmade_png.h]
[28:33][Consult the PNG spec[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/] with an explanation of the RIFF generic file container format][:research]
[31:42][Introduce png_header, png_chunk_header and png_chunk_footer structs, using #pragma pack[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing]
[35:54][Begin to implement ParsePNG(), introducing ConsumeSize() to extract desired data from our file contents][:parsing]
[43:49][:Run it and hit a file underflow error][:parsing]
[44:25][Step through ParsePNG() to realise that the ContentsSize doesn't decrease][:parsing :run]
[44:43][Fix ConsumeSize() to decrement the ContentsSize][:parsing]
[45:03][Step through ParsePNG()][:parsing :run]
[46:23][Consult the PNG spec for information on the Length field[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research]
[47:07][Change Type in png_chunk_header to be an array of four chars][:parsing]
[47:30][Step in to ParsePNG() to see our Type more easily][:parsing :run]
[48:09][Consult the PNG spec about for information on the Length field[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research]
[50:20][Inspect the FileContents as raw :memory to determine that PNG may be big-endian][:run]
[51:32][Consult the PNG spec[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/] to see that it is indeed big-endian][quote 616]
[52:41][Introduce EndianSwap()][:memory]
[1:01:35][Add a test value and assertion in EndianSwap()][:memory]
[1:02:05][Step in to EndianSwap() to see what it produces][:memory :run]
[1:02:37][Step in to ParsePNG() to inspect our endian-swapped PNG data][:parsing :run]
[1:03:41][Enable ParsePNG() to print out the chunk types][:parsing]
[1:04:34][:Run it to see our chunks][:parsing]
[1:04:49][Start to enable ParsePNG() to not bother :parsing certain chunk types, introducing FOURCC() to turn a string into a 32-bit identifier]
[1:08:26][:Run it to see that our FOURCC() works][:parsing]
[1:09:00][Check the web[ref
    site=stackoverflow
    page="Set a FourCC value in C++"
    url=https://stackoverflow.com/questions/811361/set-a-fourcc-value-in-c] to show other FOURCC() approaches][:parsing :research]
[1:09:46][Try to make FOURCC() straight up cast the string to a u32 pointer][:parsing]
[1:10:21][:Run it to see that this works][:parsing]
[1:10:33][Enable ParsePNG() to switch on our chunk types for conditional :parsing]
[1:11:13][Hit a compile error and revert FOURCC() to our bit-shifting approach][:parsing]
[1:11:50][Trim down ParsePNG() to only handle IHDR and IDAT, introducing png_ihdr struct[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing]
[1:21:30][:Run it to see our IHDR data][:parsing]
[1:22:32][Condense ParsePNG() down to only support 8bpp RBGA, deflate / inflate, and the International Standard-defined adaptive filtering and not interlaced PNGs[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]]
[1:25:59][Jump into the IDAT chunk[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing :research]
[1:30:03][Consult the DEFLATE spec[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :parsing :research]
[1:31:56][Huffman Tree][:blackboard :compression]
[1:37:04][Consult DEFLATE's use of Huffman coding[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt] with a note that Huffman excels with integral numbers of bits][:compression :research]
[1:40:43][][:research][quote 617]
[1:40:50][Continue to consult the DEFLATE spec[ref
    site=IETF
    page="DEFLATE Compressed Data Format Specification version 1.3"
    url=https://www.ietf.org/rfc/rfc1951.txt]][:compression :research]
[1:50:41][Setup ParsePNG() to decompress our images, introducing AllocatePixels()[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:compression :memory :parsing]
[1:57:50][Introduce png_idat_header and png_idat_footer for ParsePNG() to parse out of our file[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing]
[2:05:03][:Run it to see our IDAT chunk data[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]][:parsing]
[2:06:44][Q&A][:speech]
[2:07:28][@x13pixels][Q: Instead of using the FOURCC() macro why not just use a single-quoted literal? For example, if ((ChunkHeader->TypeU32) == 'IHDR') {}]
[2:07:55][@bigmofo1][Q: What is wrong about using ImageMagick from the build system to convert all the images?]
[2:08:22][@x13pixels][Q: Don't think so. I used it 20 years ago at least]
[2:09:14][@mtc743][@handmade_hero Q: What math subject do I need to know to follow along? Should I learn linear algebra?][:mathematics]
[2:09:56][@ozkayace][Q: Just little bit brainstorming, can somehow Zipf's law be used for benefit on Huffman?][:compression]
[2:10:11][@saidwho12][@handmade_hero It would work if you bswap the type]
[2:10:21][Change ParsePNG() to EndianSwap() the chunk type, and replace FOURCC() with direct equality checks on 'IHDR' and 'IDAT']
[2:11:07][@ozkayace][Q: Zipf's law[ref
    site=Wikipedia
    page="Zipf's law"
    url=https://en.wikipedia.org/wiki/Zipf%27s_law]][:compression]
[2:13:28][@dudeinbasement1][Q: How could someone abuse the PNG loader? You brought up malformed code]
[2:14:28][@tybereon][@handmade_hero The png_idat_footer should be 4 bytes. I think you just have it as a single u8 right now]
[2:14:35][Fix png_idat_footer]
[2:14:40][@mihaicris][Q: Why are there several IDAT chunks and not one single one?]
[2:15:33][@vtlmks][@handmade_hero Not using _bswap() intrinsic instead?[ref
    site=Intel
    page="Intel Intrinsics Guide"
    url=https://software.intel.com/sites/landingpage/IntrinsicsGuide/]][:memory]
[2:17:00][Try to change EndianSwap() to use the _bswap intrinsic[ref
    site=Intel
    page="Intel Intrinsics Guide"
    url=https://software.intel.com/sites/landingpage/IntrinsicsGuide/]][:memory]
[2:18:32][Try to find the definition of _bswap][:memory :run]
[2:20:23][Make EndianSwap() use the _byteswap_ulong intrinsic][:memory]
[2:21:00][Step in to EndianSwap() and check out the :asm][:run]
[2:21:34][Make EndianSwap() use our original approach, and compile in -O2][:memory]
[2:21:53][Step in to EndianSwap() to see that the compiler output a bswap anyway][:memory :run]
[2:22:46][@bigmofo1][Q: Is the PNG loader to support allow live :art iteration on stream? Art stream soon?]
[2:23:01][@pmttavara][Q: Sorry if I missed it, but was there any reason the chunk types have the weird mixed case, like tEXt and pHYs etc?[ref
    site=W3C
    page="Portable Network Graphics (PNG) Specification (Second Edition)"
    url=https://w3.org/TR/2003/REC-PNG-20031110/]]
[2:24:12][@saidwho12][Q: What's the 32-bit crc for?]
[2:25:02][We're all done][:speech]
[/video]