[video member=pervognsen stream_platform=twitch project=bitwise title="End-to-End Workflow & Clean-Up" vod_platform=youtube id=Y4ykmb9zdJM annotator=Miblo]
[0:09][Recap and set the stage for the day][:speech]
[1:10][Review the C :"code generation" work done on Day 010 Extra[ref
    site=Bitwise
    page="C Code Generation (Extra)"
    url=https://bitwise.handmade.network/episode/bitwise/bitwise010_2/]][:language :research]
[2:37][:Run the program to see our generated output][:"code generation"]
[3:16][Toggle on example_test()][:"code generation"]
[3:43][:Run it to see the result of example_test()][:"code generation"]
[3:54][Continue to review the C :"code generation" work done on Day 010 Extra[ref
    site=Bitwise
    page="C Code Generation (Extra)"
    url=https://bitwise.handmade.network/episode/bitwise/bitwise010_2/]][:language :research]
[8:45][Determine to work on the end-to-end workflow][:"code generation" :speech]
[10:46][Create ion.c and introduce ion_compile_file() and ion_compile_str(), and the general utility read_file()][:"code generation" :"file io"]
[27:28][@badflydog][@pervognsen You should probably use "rb" in fread, I believe the CRT messes with newlines otherwise][:"file io"]
[27:31][Fix read_file() to pass "rb" as the second argument to fopen()][:"file io"]
[27:42][:Run it to see that it works][:"file io"]
[27:50][Explain how fopen() works on Windows][:"file io" :speech]
[28:44][@nothings2][Unless it's actually a single-line file, I suspect it's actually the ^Z at the end of the file, not the CR-LF stuff][:"file io"]
[29:37][Introduce ion_test()][:"code generation"]
[30:18][@nothings2][@pervognsen The loop should be totally unnecessary for reading an actual file, although reading from a tty or network stream redirected or something, sure][:"file io"]
[31:11][Create test1.ion]
[31:38][@nothings2][For what it's worth, Posix says "Upon successful completion, fread() shall return the number of elements successfully read which is less than items only if a read error or end-of-file is encountered."][:"file io"]
[31:51][Make read_file() read in the entire file in one call][:"file io"]
[32:29][@james4k][Zeal looks nice, thanks whoever pointed that out]
[32:51][:Run it to see that it still works][:"file io"]
[33:10][Enable ion_compile_file() to handle empty files][:"code generation" :"file io"]
[34:37][:Run it to see that it actually worked][:"code generation" :"file io"]
[35:08][Enable read_file() to handle 0-length files][:"file io"]
[36:14][:Run it to see that it works][:"code generation" :"file io"]
[36:36][Add a load of test code to test1.ion][:"code generation" :"file io"]
[38:30][@badflydog][Probably Intel shortcuts, I believe you can disable them in the Intel control panel]
[39:06][Finish up adding test code to test1.ion][:"code generation" :"file io"]
[39:38][:Run it to see our generated output][:"code generation" :"file io"]
[39:43][@twitchplaysp0ng][Will there be a syntax definition for Ion?]
[40:03][Introduce write_file()][:"file io"]
[43:34][:Run it to find that it works][:"file io"]
[44:02][Enjoy MSCV's regex-matching search][:rant]
[45:05][Introduce get_ext() and replace_ext()][:"string manipulation"]
[52:17][:Run it to see how these file naming functions work][:"string manipulation"]
[53:04][Enable ion_compile_file() to write out to file][:"file io"]
[54:12][:Run it to find that we successfully write out a .c file][:"file io"]
[55:00][@badflydog][In before POSIX strdup warning]
[55:09][@badflydog][@pervognsen You should use "wb" in write file as well][:"file io"]
[55:54][Temporarily change write_file() to pass "wb" to fopen(), and inspect the (undesired) generated output][:"file io"]
[56:59][Create generate_test.py to facilitate the production of many lines of Ion code][:"code generation"]
[1:02:01][@heasummn][@pervognsen Do we want to focus heavily on :optimisation in the C version since we will rewrite it eventually?][:"code generation" :language]
[1:03:00][Change generate_test.py to use regex][:"code generation"]
[1:05:20][:Run it to see that it works][:"code generation"]
[1:05:25][Enable generate_test.py to generate a fair amount of code][:"code generation"]
[1:07:42][Determine to compare the compilation times of gcc and ion][:run]
[1:08:10][Enable syntax_error() and fatal_syntax_error() to report the line number containing the error][:"error handling" :parsing]
[1:10:02][:Run it and hit an error on line 1][:parsing]
[1:10:11][Enable main() to call init_keywords(), the latter to set arena_end after calling KEYWORD(typedef), and increase the ARENA_BLOCK_SIZE from 1024 to 1024 * 1024][:memory :parsing]
[1:12:55][:Run it to find that it does the right thing][:"code generation"]
[1:13:25][Introduce ion_main(), and print out compilation success / failure][:"code generation" :"error handling"]
[1:16:00][:Run it (successfully) on test1.ion and (unsuccessfully) on test2.ion][:"code generation"]
[1:16:52][Enable init_stream() and the error handling functions to handle anonymous sources (i.e. ones lacking a file path)][:"error handling" :"file io"]
[1:20:14][:Run it and no longer hit the "FATAL: Undeclared name" error][:"error handling"]
[1:20:58][Make ion_compile_file() pass the path to init_stream()][:"file io"]
[1:21:04][:Run it to find that it all works as desired][:"file io"]
[1:21:18][Introduce SrcLoc struct, and pull fatal(), syntax_error() and fatal_syntax_error() out into a new error.c file, and introduce loc_syntax_error()][:"error handling"]
[1:28:00][:Run it to find that it all works][:"error handling"]
[1:28:08][Rename loc_syntax_error() to error(), make resolve_typespec() call it, and provoke a "must denote a type" error][:"error handling"]
[1:31:09][:Run it, correctly hit the error, but segfault][:"error handling"]
[1:31:16][Introduce fatal_error() for resolve_typespec() to call][:"error handling"]
[1:31:57][:Run it to find that it works][:"error handling"]
[1:32:06][Q&A][:speech]
[1:32:20][@heasummn][@pervognsen You do sizeof(p), that should have a (?)]
[1:32:29][Fix generate_test.py as per @heasummn's suggestion][:"code generation"]
[1:33:11][:Run it find that it worked][:"code generation"]
[1:33:13][Make generate_test.py produce much more code][:"code generation"]
[1:33:18][:Run it to find that it compiles slowly][:"code generation" :performance]
[1:34:45][@praet_a51][Also path is declared but not used]
[1:35:04][Fix ion_main() to pass path to ion_compile_file()][:"file io"]
[1:35:13][@cubercaleb][@pervognsen Can you run a test with -O2 or -O3 to see how fast ion is with optimizations?][:performance]
[1:35:22][Compile Ion with -O3 to see that it's still slower than it should be][:performance]
[1:35:51][@xanatos387][As far as speed goes, we still don't have a hashtable interning right?][:performance]
[1:36:09][Make generate_test.py produce 16× more code]
[1:36:41][:Run it to find that it takes forever]
[1:36:51][@praet_a51][Could trading from disk itself be an issue as well][:"file io" :performance]
[1:37:40][@twitchplaysp0ng][Where in the code is n^2][:performance]
[1:38:58][@heasummn][Could you use Visual Studio's profiler to see what's causing the most slowdown?][:performance]
[1:39:01][See how long it took][:performance :run]
[1:39:35][@praet_a51][Are we optimising a bit today or will that start of tomorrow][:optimisation]
[1:39:41][@godling72][@pervognsen Isn't a bit of premature optimization? You aren't close to having 650k of real ion code?][:optimisation]
[1:40:06][Stop recording][:speech]
[/video]
