cinera_handmade.network/cmuratori/hero/code/code353.hmml

[video output=day353 member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Simple RLE Compression" vod_platform=youtube id=kikLEdc3C1c annotator=Miblo]
[0:18][Set the stage for the day]
[1:10][Blackboard: RLE vs LZ]
[1:53][Blackboard: On exploiting the properties of the data we want to compress]
[5:56][Blackboard: Run-length Encoding]
[8:24][Blackboard: Distinguishing between "literals" and "runs"]
[9:59][Blackboard: Various formats developed by Electronic Arts: IFF, ILBM, PCX]
[11:53][Blackboard: LZ (Lempel-Ziv) as a more heavy-duty form of compression]
[16:03][Blackboard: Encoding an additional offset in LZ]
[20:07][Blackboard: How LZ gives you RLE almost for free]
[25:50][Blackboard: Replicating blocks]
[32:28][Blackboard: Why LZ schemes use relative positioning, and choosing how many bits to allocate for the look-back distance]
[39:06][Blackboard: LZ encoding]
[44:50][Blackboard: Keeping a hash table and information of overlapping runs for the back catalogue]
[46:48][Blackboard: Entropy encoding and backends, e.g. Huffman, Arithmetic, rANS / tANS]
[49:52][Blackboard: Charting the frequency of values]
[53:10][Blackboard: Huffman]
[1:01:21][Blackboard: On the inefficiency, size-wise, of Huffman compared to Arithmetic]
[1:02:32]["Please take it with a grain of salt, or even a cube of salt"][quote 539]
[1:03:27][Blackboard: rANS / tANS[ref
    site="The ryg blog"
    page="rANS notes"
    url="https://fgiesen.wordpress.com/2014/02/02/rans-notes/"]]
[1:06:43][Determine to compress our art assets]
[1:08:16][Save test_scene_layer_00.bmp as a png and compress it with 7zip for comparison]
[1:11:09][simple_compressor.cpp: Setup the code to make an actual command-line compressor]
[1:14:56][Run the compressor and view the documentation]
[1:15:16][simple_compressor.cpp: Parse the Args]
[1:17:46][simple_compressor.cpp: Introduce file_contents struct for ReadEntireFileIntoMemory() to return]
[1:20:39][simple_compressor.cpp: Set up the call sites for Compress() and Decompress()]
[1:28:53][simple_compressor.cpp: Introduce Compress(), Decompress() and Copy()]
[1:32:38][Run the compressor on test1.hha]
[1:35:00][simple_compressor.cpp: Introduce RLECompress() and RLEDecompress()]
[1:43:01][simple_compressor.cpp: Implement RLEDecompress()]
[1:45:32][simple_compressor.cpp: Enable RLECompress() to encode a run]
[1:51:33][Run the compressor in the debugger]
[1:53:18][simple_compressor.cpp: Make RLECompress() and RLEDecompress() more symmetric]
[1:54:59][Step in to RLECompress() and inspect the values]
[1:58:20][simple_compressor.cpp: Make RLECompress() correctly loop to the MAX_RUN_COUNT and compute the OutSize]
[1:59:41][Run the compressor on test1.hha to find that it has output exactly the same size]
[2:01:08][simple_compressor.cpp: Make Compress() use the MaxOutSize value, run the compressor on test1.hha and find that the output is much smaller]
[2:02:15][Run the decompressor on test1.scp to find that it doesn't work at all]
[2:02:56][simple_compressor.cpp: Correctly store the InFile.FileSize]
[2:05:02][Step in to the compressor and inspect what it does]
[2:07:04][Step in to the decompressor to find that it works fine, and diff the files it produced]
[2:08:39][Run the compressor on intro_art.hha to see how it fares with that]
[2:10:34][Q&A][:speech]
[2:11:21][Compress test1.hha and intro_art.hha with 7zip for comparison]
[2:12:38][@Miblo][Awesome stuff! I hadn't been making the distinction between "literals" and "runs" in my thing. My test stuff has been just 0s and 1s, mind, so I was just storing the first Token and then the Counter values]
[2:14:14][@magik603][Tomorrow you will show us what again?]
[2:15:36][@ttbjm][Are hardware image decompressors still a thing?]
[2:22:54][Conclude][:speech]
[/video]