education-2022/articles/compilers.md

Compilers: How To Make a Programming Language

Other compiler-related topics out of scope:

• What are programs anyway (Forth, LISP)
• Computation: Turing machines, lambda calculus
• Compiler optimization
• GC (motivated by LISP)

aesthetics:

• stay motivated; stick close to a c-like, imperative, procedural model because that's what people are used to

motivation / goals / what questions are we trying to answer:

• I want to be able to make a toy procedural language.
  • like C, Algol, JS, Lua, etc.
  • possible user motivations:
    • I want to make a game scripting language
    • I want to make a DSL for my job (and I want syntax highlighting!)
• I want to do simple static analysis of my projects

ok what do we want to cover

• classical compiler structure (lexer -> parser -> codegen)
• semantic analysis / type checking
• modern compiler structures (IR / SSA, optimization)
• interpreters vs. JITs vs. AOTs vs. "transpilers"
• executables and linkers
• regular expressions?
• regular languages / grammars / language structure / automata

the c compilation process

• translation units
• preprocessing -> (the whole compilation process) -> object files -> linking
• c compilation model is not in favor any more, don't like compiling all these files separately
• ABIs and FFI
• should maybe be in separate article

experts / consultants:

• Bill
• NeGate

The actual progression

• Simple expression interpreter (parse and evaluate)
• Classical compiler construction (lex -> parse -> output), semantic analysis / type checking
  • motivation: complex structures! recursion! etc.
  • many of these resources exist and cover different aspects of the process in different ways
• Grammars and language structure
• Types of output (interpreter vs. AOT vs. JIT, etc.)
  • We can probably find resources on specific ones of these
• Modern phases (IR / SSA)
  • Mention WASM?
• The terrors of the real world
  • Executables, linkers, and debug info
  • The C ABI and FFI
  • Register allocation
• C is not the only language
  • LISP
  • Forth
  • This could be a whole topic maybe

Link dump

• lua grammar: http://lua-users.org/wiki/LuaGrammar
• pascal railroad diagrams: https://www.cs.utexas.edu/users/novak/grammar.html
• tons of links: https://github.com/aalhour/awesome-compilers
• dragon book: http://ce.sharif.edu/courses/94-95/1/ce414-2/resources/root/Text%20Books/Compiler%20Design/Alfred%20V.%20Aho,%20Monica%20S.%20Lam,%20Ravi%20Sethi,%20Jeffrey%20D.%20Ullman-Compilers%20-%20Principles,%20Techniques,%20and%20Tools-Pearson_Addison%20Wesley%20(2006).pdf
• expression parsing examples:
  • pratt parsing and recursive descent: https://journal.stuffwithstuff.com/2011/03/19/pratt-parsers-expression-parsing-made-easy/
  • dunno, not recursive descent: https://www.cs.rochester.edu/u/nelson/courses/csc_173/grammars/parsing.html
• gary bernhardt's compiler from scratch: https://www.destroyallsoftware.com/screencasts/catalog/a-compiler-from-scratch
• lambda calculus interpreter: https://justine.lol/lambda/