rhysd / awesome-compilers

:sunglasses: A curated list of awesome resources on Compilers, Interpreters and Runtimes.

A curated list of awesome resources, learning materials, tools, frameworks, platforms, technologies and source code projects in the field of Compilers, Interpreters and Runtimes. This list has a bias towards education.

---

Contents

• Learning
  • Books
    • General Overview
    • Introductory
    • Advanced
  • Papers
    • Researchers and Institutes
  • Specifications
  • Courses
  • Talks and Conferences
    • Channels
    • Videos
  • Articles
  • Tutorials
  • Community Discussions
• Tools and Frameworks
  • Language Agnostic
  • C (as a Backend)
  • CLR
  • D
  • Graal
  • Haskell
  • JavaScript
  • JVM
  • Kotlin
  • Python
    • Lists of Python Parsing Tools
  • Rust
• Compilers and Interpreters
  • Serious Projects
  • Educational and Toy Projects
• Runtimes and VMs
• Blogs
• Communities
• Verticals

---

Learning

Books

General Overview

• Computer Systems: A Programmer’s Perspective, R. Bryant, D. O'Hallaron - Comprehensive treatement of Computer Systems including Compilers, Interpreters and Runtimes.
• Elements of Computing Systems, N. Nisan, S. Schocken - How to build a computer starting with Nand Logic Gates and then move to Machine Code, Assemblers, Compilers and Operating Systems.
• SICP, H. Abelson, G. Sussman - Structure and Interpretation of Computer Programs.

Introductory

• Basics of Compiler Design, T. Mogensen.
• Beautiful Racket - How to Make Your Own PL with Racket, M. But­t­er­ick.
• Compilers: Principles, Techniques, and Tools - The Dragons Book (2nd Edition).
• Crafting Interpreters: A Handbook for Making Programming Languages, B. Nystrom.
  • GitHub Repo.
  • Discussions: HN.
• Create Your Own Programming Language.
  • Discussions: HN.
• Engineering a Compiler, K. Cooper, L. Torczon.
• Essentials of Programming Languages, D. Friedman & M. Wand.
• Language Implementation Patterns, T. Parr.
• Modern Compiler Implementation in ML, A. Appel.
  • Other editions: MCI in C, MCI in Java.
• Programming Language Pragmatics, M. Scott.
• Programming Languages: Application and Interpretation, S. Krishnamurthi.
  • PDF.
• Programming Languages: Theory and Practice.
• Project Oberon, N. Wirth & J. Gutknecht.
  • Other editions: 2013 Edition.
• The BEAM Book - Description of the ERTS (Erlang Runtime System) and the BEAM Virtual Machine.
• Virtual Machines, Smith and Nait.
• Virtual Machines, Iain Craig.

Advanced

• Advanced Compiler Design and Implementation, S. Muchnick.
• Advanced Topics in Types and Programming Languages, B. Pierce.
• A Retargetable C Compiler: Design and Implementation, D. Hanson.
• Building an Optimizing Compiler, B. Morgan.
• Compiling with Continuations, A. Appel.
• Design Concepts in Programming Languages, F. Turbak, D. Gifford, M. Sheldon - also contains some introductory level material.
• Instruction Level Parallelism, A. Aiken, U. Banerjee, A. Kejariwal, A. Nicolau.
• Linkers and Loaders.
• Optimizing Compilers for Modern Architectures, R. Allen & K. Kennedy.
• Parsing Techniques: A Practical Guide.
  • 1st Edition, PDF.
• The Garbage Collection Handbook: The Art of Automatic Memory Management, R. Jones, A. Hosking, E. Moss.
• The Implemetation of Functional Programming Langauges, S. P. Jones.
• The SSA Book, Springer, Zadeck.
• Types and Programming Languages, B. Pierce.
• Warren's Abstract Machine - Prolog in Haskell, H. Aït-Kaci.

Papers

• A Brief History of JIT Compilation, J. Aycock.
• A Flexible Prolog Interpreter in Python, C. Bolz & M. Leuschel.
• A Graph Higher-Order IR, R. Leißa, M. Koster & S. Hack.
• A Prolog Interpreter in Python, C. Bolz.
• A Simple Multi-Processor Computer Based on Subleq, O. Mazonka, A. Kolodin.
• Compiler Construction Using Scheme, E. Hilsdale, J. Ashley, R. Dybvig & D. Friedman.
• Compiling with Continuations: Continued, A. Kennedy.
• Definitional Interpreters for Higher-Order Programming Languages, J. Reynolds.
• Draining the Swamp: Micro Virtual Machines as Solid Foundation for Languauage Development, K. Wang, Y. Lin, S. Blackburn, M. Norrish & A. Hosking.
• Engineering Definitional Interpreters, J. Midtgaard, N. Ramsey, B. Larsen.
• Garbage Collection in an Uncooperative Environment, H. Boehm, M. Weiser.
• Machine Code Obfuscation via Instruction Set Reduction and CFG Linearization, C. Jonischkeit.
• MOV is Turing-Complete, S. Dolan.
  • Discussions: HN, Reddit.
• Nanopass Framework for Commercial Compiler Development, A. Keep & R. Dybvig.
• Nanopass Framework for Compiler Education, D. Sarkar, O. Waddell & R. Dybvig.
• Notes on Graph Algorithms Used in Optimizing Compilers, C. Offner.
• Packrat Parsing Thesis on PEG, B. Ford.
• PEG-based transformer provides front-end, middle-end and back-end stages in a simple Compiler, I. Piumarta.
• Pycket: A Tracing JIT for a Functional Language.
• PyPy’s Approach to VM Construction, A. Rigo & S. Pedroni.
• RABBIT: A Compiler for SCHEME, G. Steele.
• Simple and Efficient Construction of SSA Form.
• SSA-based Register Allocation, S. Hack & G. Goos.
• The Essence of Compiling with Continuations, C. Flanagan, A. Sabry, B. Duba & M. Felleisen.
• The Page-Faults Weird Machine: Lessons in Instruction-less Computation, J. Bangert, S. Bratus, R. Shapiro, S. Smith.
• Trace-based JIT Compilation for Lazy Functional Languages, T. Schilling.
• Using Datalog with Binary Decision Diagrams for Program Analysis, J. Whaley, D. Avots, M. Carbin & M. Lam.

Researchers and Institutes

• C. Bolz’s Research Publications.
• Compilers Lab at Saarland University.
• Packrat Parsing (PEG) Papers and Resources.

Specifications

• CLI (ECMA-335) Specification.
• Mu Specification.
• JVM SE8 Specification.

Courses

• Compilers Construction, Cambridge - Introduction to compiler construction course from the University of Cambridge.
• Compiler Construction for Undergrads, RICE University - Introduction to compiler construction and language translators course from the RICE University.
• Compilers Theory, Stanford - YouTube, Stanford.edu, Class Notes -Introduction to Compilers theory and construction course from Stanford.
• Design and Construction of Compilers, University of Texas - Design and construction of compilers including lexical analysis, parsing, code generation techniques, error analysis and simple code optimizations.
  • Lecture Notes: PDF, HTML.
• DSL Design and Implementation Summer School - Summer School program on the topics of DSL Design and Implementation hosted by the EPFL University.
• Foundations of Programming Languages - Concepts that underlie the design, definition, implementation and use of modern programming languages from a formal standpoint.
• Nand2Tetris: How to Build a Computer from First Principles, Part 2 - This 2nd part of the Nand2Tetris course covers basic language design and elementary compiler construction concepts in addition to many other topics on a basic level.
• NPTEL's Principles of Compiler Design Course - Introductory course from NPTEL on Compiler Design.
• NPTEL's Compiler Design Course - Slightly more advanced course than their Principles of Compiler Design course, covers SSA Form to a good degree.
  • YouTube Video Playlist.
• Programming Languages: Part A, by Grossman - Part 1 of a 3-part course series to the basic concepts of programming languages, with a strong emphasis on functional programming.
• Programming Languages: Part B, by Grossman - Part 2 of a 3-part course series to the basic concepts of programming languages, with a strong emphasis on functional programming.
• Programming Languages: Part C, by Grossman - Part 3 of a 3-part course series to the basic concepts of programming languages, with a strong emphasis on functional programming.
• Types, Logic, Semantics, and Verification from Oregon University's Summer School - Summer School program that consists of 80 minute lectures presented by internationally recognized leaders in programming languages and formal reasoning research.
• Virtual Machines and Managed Runtimes, UCB CS294 - Introductory course on Virtual Machines and Managed Runtimes from the University of Berkeley.
• Virtual Machines Summer School 2016 (VMSS 2016) - VMSS is a Summer School program that aims to give an overview of the field, targeted at early career researchers.
  • YouTube Videos Playlist.

Talks and Conferences

Channels

• Curry On! Conference - Programming Languages Conference.
  • YouTube Channel.
• Lang.NEXT Conference 2012.
• Lang.NEXT Conference 2014.
• LLVM Developers Meeting YouTube Channel.

Videos

• Anders Hejlsberg on Modern Compiler Construction.
• An Introduction to Combinator Compilers and Graph Reduction Machines.
• Building an Interpreter in RPython.
• Exploring Python’s Bytecode.
• Java AOT (Ahead of Time) Compilation.
• MetaScala: A Tiny DIY JVM - Metascala is a tiny metacircular Java Virtual Machine (JVM) written in the Scala programming language.
• Meta-Tracing, RPython and PyPy.
• One VM to Rule Them All, One VM to Bind Them - Tutorial on the Truffel technology.
• Python, Linkers and Virtual Memory - PYCON US.
• Reverse Engineering the MOS 6502 CPU.
• Single Static Assignment Form Seminar - Introductory seminar on SSA Form, Compiler Optimizations under it and its applications in other areas such as Program Analysis and Verification.
• The JVM (Java Virtual Machine) Architecture.
• The MoVfuscator: turning mov into a soul crushing RE nightmare.
  • Discussions: HN, Reddit.
• The SSA Form - Video Lectures from NPTEL's Compilers Course (see: Courses for more info):
  • Part 1.
  • Part 2.
  • Part 3.
• Type-Driven Development with Idris.
• Understanding Compiler Optimization.

Articles

• A Tourist’s Guide to the LLVM Source Code.
• Accidentally Turing Complete.
• Adventures in JIT Compilation by Eli Bendersky:
  • Part 1 - An Interpreter.
  • Part 2 - An x64 JIT.
  • Part 3 - LLVM.
  • Part 4 - In Python.
• ALIVe: Automatic LLVM InstCombine Verifier.
• Graal and Truffel - Obscure research project could radically accelerate innovation in programming language design.
  • Discussions: Reddit.
• How to Compile with Continuations.
• Interpreter, Compiler and JIT.
• Introducing the B3 JIT Compiler.
• LLVM Architecture - Chapter from the book Architecture of Open Source Applications.
• LLVM for Graduate Students - Introduction to doing research with the LLVM compiler infrastructure.
• Pratt Parsers: Expression Parsing Made Easy.
• Rust Compiler Walk-Through.
• Rust's Incremental Compilation.
• Rust’s Upcoming MIR.
• The Memory Models that Underlie Programming Languages.

Tutorials

• A Tutorial Implementation of a Dependently Typed Lambda Calculus.
• Algorithm W Step By Step.
• Compiler Optmization Tutorial.
• How I Wrote a Programming Language, and How You Can Too.
  • Discussions: Reddit.
• Implementing a JIT Compiled Language with Haskell and LLVM.
• Kaleidoscope: Implementing a Language with LLVM in Objective Caml.
• Let’s Build A Simple Interpreter.
• Lisperator - How to implement a programming language in JavaScript.
• LLVM Tutorial: Implementing Kaleidoscope.
  • Python version with LLVMPY.
• Metacompiler Tutorial, Part 1.
• Project: A Programming Language - Chapter 11 from the book Eloquent JavaScript, 2nd Edition.
• Write You a Haskell.
• Writing a Language in Truffel - Interpreter development tutorial using Truffel, by Cristian Esquivias.

Community Discussions

• Can we stop recommending the Dragon Book, please? - StackExchange thread criticising the Dragons Book in favor of alternatives.
• What is difference between an LL and Recursive Descent parser?.
• Does a compiler use all x86 instructions? - @HN, @Reddit - Article discussions on HN and Reddit.
• How to Write a Compiler - Article discussion on Reddit.
• How to Write a Very Basic Compiler.
• How to Write a Prolog Interpreter in a Purely Functional Language.
  • Discussions: Reddit.
• Implementing Type Inference.
• I want to build a VM, any good references?.
• Resources for Amatuer Compilers Writers.
• What are the good open source implementations of Java Virtual Machine?.
• What are the latest research trends in Compilers and PLs.

Tools and Frameworks

Language Agnostic

• B3: The Bare Bones Backend - WebKit's optimizing JIT Compiler for procedures containing C-like code.
• LLILCL - LLVM-based Compiler Backend for .NET Core.
• LLVM - The LLVM Compiler-Backend Framework.
• MicroVM - The "Mu" Framewrok for Programming Languages development based on the MuVM Specification.
• Movfuscator Compiler - The M/o/Vfuscator compiles programs into "mov" instructions, and only "mov" instructions.
• QBE: The Quick Backend - Pure C embeddable SSA-based compiler backend.
• Rubinius - Programming Languages Development Platform.
• Summus - Basic, reusable, compiler-frontend implementation using LLVM as a backend.

C (as a Backend)

• GCC - The GNU Compiler Collection.
• LCC - The lcc retargetable ANSI C compiler.
• libFirm - C-library that provides a graph-based intermediate representation, optimizations, and assembly code generation suitable for use in compilers.
• libJIT - Library for generic Just-In-Time compiler functionality independent of any particular bytecode, language, or runtime.
• myJIT - Library for machine-code generation and execution at run-time.
• PCC - The Portable C Compiler.
• TCC - The Tiny C Compiler.

CLR

• Cecil - Library to generate and inspect programs and libraries in the ECMA CIL format.
• ILSpy - .NET Decompiler.
• Reflector - .NET Decompiler.
• Reflexil - Assembly code editor which can be used as a plugin with other .NET/CLR tools.

D

• dunnart - LALR(1) Parser Generator.
• FancyPars-lite - Fast parser generator.
• libdparse - Library allowing to build lexers and parsers. Contains a lexer and a parser for the D language itself.
• llvm-d - D bindings for LLVM.
• Pegged - design, test and generate parsers for PEG grammars.

Graal

• Graal - High-Performance Polyglot Runtime.
• Graal Core - Compiler and Truffel Partial Evaluator.
• Graal VM - Graal's multi-language VM distribution.

Haskell

• Bound / unbound / unbound-generics - Libraries for manipulating bound variables.
• Hoopl - Higher-order optimization library.
• llvm-general - Haskell bindings for LLVM.
• Parsec / attoparsec / Megaparsec / Trifecta / Alex + Happy - Parsers for every use case.
• wl-pprint-text / ansi-wl-pprint - Walder-style pretty-printing libraries.

JavaScript

• IRHudra - Tool for displaying intermediate representations used by V8 and Dart VM optimizing compilers.
  • GitHub Repo.
• JISON - Context-free grammar parser generator for JavaScript.
• PEG.js - Simple parser generator for JavaScript.

JVM

• ANTLR - Parser generator for reading, processing, executing, or translating structured text or binary files.
• BYAAC/J - BYACC/Java is an extension of the Berkeley v 1.8 YACC-compatible parser generator for Java.
• CGLIB - High level API library for generating and transforming Java Byte Code.
• FCP JVM - JVM Backend for generating Java Byte Code that conforms to the JDK v1.5+ Specification and the Dalvik VM.
• JavaCC - Java Compiler Construction and Parser Generator Toolkit.
• JFlex - JFlex is a lexical analyzer generator for Java with full Unicode support.
• JLex - JLex is a lexical analyzer generator, that can be used in combination with CUP.

Kotlin

• The Whimsy Compiler Framework - Compiler framework research project in Kotlin.

Python

• AST - Python's builtin Abstract Syntax Tree package.
• Dis - Python's builtin Disassembler package.
• Parsing - Pure-Python module that implements an LR(1) parser generator, as well as CFSM and GLR parser drivers.
• PLY - Implementation of lex and yacc parsing tools for Python.
• PyParsing - Alternative approach to creating and executing simple grammars, vs. the traditional lex/yacc approach, or the use of regular expressions.
• RPLY - Port of the PLY project to RPython.
• RPython - RPython is a framework for the implementatation of dynamic languages.

Lists of Python Parsing Tools

• List of Language Parsing Tools at the Python Wiki.
• Survey of Python Parsers, by Ned Batchelder.

Rust

• Combine - Parser Combinator Library for Rust.
• IronLLVM - Safe LLVM bindings for Rust.
• LALRPOP - LR(1) parser generator for Rust.
• Nom - Parser Combinator Framework.
• PEG - PEG Parser Generator.
• Pest - PEG Parser Generator.
• RLS - The Rust Language Server implementation (aka RLS).

Compilers and Interpreters

This section aims at listing code projects of Compilers, Interpreters, Translators, Runtimes, Virtual Machines and the like.

Serious Projects

• Babel.js - Next-generation JavaScript Compiler.
• Chez Scheme's Compiler - ChezScheme Language Compiler.
• Gluon's Compiler - Embedded Language Compiler written in Rust.
• Lily's Interpreter.
• Lua's Interpreter - Official Lua Language Interperter.
  • Lua's Annotated Source Code - Annotated source code of the Lua Programming Language Interpreter v5.1.4.
• Mirah's Compiler - JVM-based Compiler for Mirah Language.
• Nim's Compiler.
• Red's Compiler.
• Roslyn - The .NET "Roslyn" Compiler Platform.
• TypeScript's Compiler.
• Wren's Compiler.

Educational and Toy Projects

• C4 - C Lang in 4 Functions.
  • Discussions: HN.
• CarpVM - Experimental VM implementation in C.
• Dale - Lisp-flavoured C: a system programming language.
• Eschelle - Open source cross platform multi-paradigm language with VM & JIT
• Gecho - Simple-stack language implementation in C.
• Hython - Haskell-powered Python 3 interpreter.
  • Discussions: Reddit.
• MetaScala - Metacircular JVM implementation in Scala.
• mini-js - Experimental self-hosted JavaScript compiler in 1K LoC.
• MunVM - Lua VM & Compiler in C.
• Poprc - Compiler for the Popr Language.
• PyCOOLC - Compiler for the COOL Programming Language written in Python 3.
• RabbitVM - RISC-based VM implementation in C.
• stack_cpu - Stack-machine simulator.
• The Super Tiny Compiler - Tiny educational compiler project in JavaScript.
  • Discussions: HN.
• tinyc.c - Tiny-C language compiler in C.

Runtimes and VMs

• CakeML.
• CoreCLR - The .NET's Common Language Runtime.
• Erlang BEAM.
• HHVM - Facebook's Open Source VM for running Hack and PHP programs.
• HLVM.
• JVM Implementations:
  • OpenJDK.
  • Kaffe.
  • JamVM - GitHub project mirror.
  • Apache Harmony.
  • Other JVM Runtimes.

Blogs

• Eli Bendersky.
• John Regehr.
• Krister Walfridsson.
• Lambda The Ultimate.
• LLVM Developers' Meetings.
• LLVM Weekly - Weekly newsletter about LLVM.

Communities

• /r/Compilers - Subreddit community about the theory and development of compilers.
• /r/ProgrammingLanguages - Subreddit community that is dedicated to discussion of programming languages, programming language theory, design, their syntax and compilers.

Verticals

• Awesome Static Analysis.
• Compiler Conferences, Workshops and Journals.
• Languages and Compilers Network Graph.
• PLT Enlightenment.
• Resources for Amateur Compiler Writers.
• SSA Form Bibliography.
• Summer Schools.
• The Witchcraft Compiler Collection.
• TypeFunc.

License

To the extent possible under law, Ahmad Alhour has waived all copyright and related or neighboring rights to this work.

The logo was designed using TextCraft.