Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update compiler architecture documentation. #1636

Merged
merged 3 commits into from
Jan 3, 2025
Merged

Update compiler architecture documentation. #1636

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3f4238c
Update compiler architecture documentation.
lucteo Dec 20, 2024
042aedc
Merge remote-tracking branch 'origin/main' into doc_compiler
lucteo Jan 3, 2025
62eb3f5
Fix case for `FrontEnd` folder.
lucteo Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 75 additions & 25 deletions Docs/Compiler/CompilerArchitecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,20 @@ backend. It conforms to the the standard swift project layout:
Then there are some extra directories specific to the Hylo project:
* [Tools] Scripts to aid in development
* [Examples] Some real world Hylo programs
* [Library] The Hylo standard (and core) library
* [StandardLibrary] The Hylo standard (and core) library

## Stages of compilation

The Hylo compiler goes through the standard stages of compilation:

1. Tokenisation: Transforms Hylo source code (Strings) to a stream of distinct
tokens
1. Parsing: Creates an [abstract syntax tree] from the token stream
1. Type-checking: Inspects the abstract syntax tree for type errors
1. IR-lowering: Generates the [intermediate representation] from the abstract
syntax tree
1. LLVM IR generation: Convert Hylo IR into [LLVM] IR
1. Machine Code Generation: This is completely handled by [LLVM]
| Stage | Description | Important files |
| ----- | ----------- | --------------- |
| Tokenisation | Transforms Hylo source code (Strings) to a stream of distinct tokens | [Lexer.swift], [Token.swift] |
| Parsing | Creates an [abstract syntax tree] from the token stream | [Parser.swift] |
| Type-checking | Inspects the abstract syntax tree for type errors | [TypeChecking], [TypeChecker.swift] |
| IR-lowering | Generates the [intermediate representation] from the abstract syntax tree | [IR], [Emitter.swift] |
| LLVM IR generation | Convert Hylo IR into [LLVM] IR | [Transpilation.swift], [Swifty-LLVM] |
| Machine Code Generation | This is completely handled by [LLVM] | |

These top-level stages of the compiler are laid out in [Driver] where you
can see the outline of the compilation phases with their entry points.
Expand Down Expand Up @@ -76,25 +76,75 @@ defined in `Module+*` files of [IR/Analysis]. After these passes the IR should
be valid and executable by a *theoretical* Hylo VM. Some [more passes] may be
necessary depending on the target.

## Important compiler modules

![PlantUML model](https://www.plantuml.com/plantuml/png/VL1TQiCm37xNANI1v01xA1qT6u4MHbRsxYLQOkBAHNOAPUpkmtYDxItqoVv-97tIrdabVZuPEAjkKWFXs2tV9z4N4bZmirZsn3r-0UADV7mxPkXLYQCC7wzdsetKJPg7FxG3mEZ7gjh4FeN_7XqRO90nmVZYRzbtIjwqPMPMvkgqowY3ui64a5dLplRqGboKxlVlX-0Pkp3cTg7uObrluaOFgFbY9tAvaY-nQcZQDNf9kog6F4c00B6gGY-0UDnQz33wV_DbPeSVz6c585UIlsbBrvQudq_c1m00)

[comment]: # (To generate this URL, copy the content of Modules.puml in http://www.plantuml.com/plantuml/uml/, and get the URL from there)

**Legend**:
| <b>Item</b> | <b>Meaning</b> |
| -- | -- |
| package | A folder in the source repository |
| solid arrow | Dependency |
| dotted arrow | phase invocation / "runs before" relations |

**Packages**:
| Package | Description |
| -- | -- |
| [hc] | The actual compiler executable. Just calls [Driver].
| [Driver/] | Defines and executes the stages of the compiler. Takes care of compiler arguments. |
| [Frontend] | The frontend of the compiler. Handles lexing, parsing and type checking. Also stores the AST representation of the program. |
| [AST/] | The structures used to describe the abstract syntax tree of the input program. |
| [Parse] | The lexer and the parser for the Hylo source files. |
| [TypeChecking] | Implements the type checking of the Hylo programs. |
| [Types] | Defines the possible types that a Hylo entity might have. |
| [IR] | Defines the intermediate representation of Hylo programs, and the operarations associated with it. |
| [Analysis] | The analysis phases that can be run on the IR. |
| [Mangling] | The name-mangling algorithm used by Hylo. |
| [CodeGen] | Implements the code-generation phase for Hylo compiler. Currently only implements LLVM code generation. |
| [LLVM/] | The code generation into LLVM. |

[Swift]: https://en.wikipedia.org/wiki/Swift_(programming_language)
[LLVM]: https://en.wikipedia.org/wiki/LLVM
[SPM]: https://www.swift.org/package-manager/
[intermediate representation]: https://en.wikipedia.org/wiki/Intermediate_representation
[abstract syntax tree]: https://en.wikipedia.org/wiki/Abstract_syntax_tree

[Driver]: ../Sources/Driver/Driver.swift
[Package.swift]: ../Package.swift
[Sources]: ../Sources
[Tests]: ../Tests
[Tools]: ../Tools
[Examples]: ../Examples
[Library]: ../StandardLibrary
[AST]: ../Sources/FrontEnd/AST/AST.swift
[ScopedProgram]: ../Sources/FrontEnd/ScopedProgram.swift
[TypedProgram]: ../Sources/FrontEnd/TypedProgram.swift
[Instruction]: ../Sources/IR/Operands/Instruction/
[Emitter]: ../Sources/IR/Emitter.swift
[IR/Module]: ../Sources/IR/Module.swift
[IR/Program]: ../Sources/IR/Program.swift
[IR/Analysis]: ../Sources/IR/Analysis/
[more passes]: ../Sources/IR/Analysis/Module+Depolymorphize.swift
[Driver]: ../../Sources/Driver/Driver.swift
[Package.swift]: ../../Package.swift
[Sources]: ../../Sources
[Tests]: ../../Tests
[Tools]: ../../Tools
[Examples]: ../../Examples
[StandardLibrary]: ../../StandardLibrary
[AST]: ../../Sources/FrontEnd/AST/AST.swift
[ScopedProgram]: ../../Sources/FrontEnd/ScopedProgram.swift
[TypedProgram]: ../../Sources/FrontEnd/TypedProgram.swift
[Instruction]: ../../Sources/IR/Operands/Instruction/
[Emitter]: ../../Sources/IR/Emitter.swift
[IR/Module]: ../../Sources/IR/Module.swift
[IR/Program]: ../../Sources/IR/Program.swift
[IR/Analysis]: ../../Sources/IR/Analysis/
[more passes]: ../../Sources/IR/Analysis/Module+Depolymorphize.swift

[Lexer.swift]: ../../Sources/FrontEnd/Parse/Lexer.swift
[Token.swift]: ../../Sources/FrontEnd/Parse/Token.swift
[Parser.swift]: ../../Sources/FrontEnd/Parse/Parser.swift
[TypeChecking]: ../../Sources/FrontEnd/TypeChecking
[TypeChecker.swift]: ../../Sources/FrontEnd/TypeChecking/TypeChecker.swift
[IR]: ../../Sources/IR
[Emitter.swift]: ../../Sources/IR/Emitter.swift
[Transpilation.swift]: ../../Sources/CodeGen/LLVM/Transpilation.swift
[Swifty-LLVM]: https://github.com/hylo-lang/Swifty-LLVM
[hc]: ../../Sources/hc
[Driver/]: ../../Sources/Driver
[Frontend]: ../../Sources/Frontend
[AST/]: ../../Sources/Frontend/AST
[Parse]: ../../Sources/Frontend/Parse
[Types]: ../../Sources/Frontend/Types
[Analysis]: ../../Sources/IR/Analysis
[Mangling]: ../../Sources/IR/Mangling
[CodeGen]: ../../Sources/CodeGen
[LLVM/]: ../../Sources/CodeGen/LLVM

39 changes: 39 additions & 0 deletions Docs/Compiler/Modules.puml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
@startuml
package hc
package Driver

package Frontend {
package AST
package Parse
package TypeChecking
package Types

Parse --> AST
TypeChecking --> AST
TypeChecking --> Types
AST -> Types
Types -> AST
}

package IR {
package Analysis
package Mangling
}

package CodeGen {
package LLVM
}

hc -> Driver

Driver ..> Parse: invokes
Driver ..> TypeChecking: invokes
Driver ..> IR: invokes
Driver ..> Analysis: invokes
Driver ..> LLVM: invokes

LLVM --> IR

Parse .right.> TypeChecking: runs before
TypeChecking .right.> IR: runs before
@enduml
Loading