From 3f4238c2408eb498dd194dea8ec36ff6fbac5bc2 Mon Sep 17 00:00:00 2001 From: Lucian Radu Teodorescu Date: Fri, 20 Dec 2024 16:06:45 +0200 Subject: [PATCH 1/2] Update compiler architecture documentation. - make the relative point to the right paths - Add "Important files" column to the stages of compilation - Add "Important compiler modules" section with diagram and explanations --- Docs/Compiler/CompilerArchitecture.md | 100 +++++++++++++++++++------- Docs/Compiler/Modules.puml | 39 ++++++++++ 2 files changed, 114 insertions(+), 25 deletions(-) create mode 100644 Docs/Compiler/Modules.puml diff --git a/Docs/Compiler/CompilerArchitecture.md b/Docs/Compiler/CompilerArchitecture.md index 7547a497e..7f2034e06 100644 --- a/Docs/Compiler/CompilerArchitecture.md +++ b/Docs/Compiler/CompilerArchitecture.md @@ -11,20 +11,20 @@ backend. It conforms to the the standard swift project layout: Then there are some extra directories specific to the Hylo project: * [Tools] Scripts to aid in development * [Examples] Some real world Hylo programs -* [Library] The Hylo standard (and core) library +* [StandardLibrary] The Hylo standard (and core) library ## Stages of compilation The Hylo compiler goes through the standard stages of compilation: -1. Tokenisation: Transforms Hylo source code (Strings) to a stream of distinct - tokens -1. Parsing: Creates an [abstract syntax tree] from the token stream -1. Type-checking: Inspects the abstract syntax tree for type errors -1. IR-lowering: Generates the [intermediate representation] from the abstract - syntax tree -1. LLVM IR generation: Convert Hylo IR into [LLVM] IR -1. Machine Code Generation: This is completely handled by [LLVM] +| Stage | Description | Important files | +| ----- | ----------- | --------------- | +| Tokenisation | Transforms Hylo source code (Strings) to a stream of distinct tokens | [Lexer.swift], [Token.swift] | +| Parsing | Creates an [abstract syntax tree] from the token stream | [Parser.swift] | +| Type-checking | Inspects the abstract syntax tree for type errors | [TypeChecking], [TypeChecker.swift] | +| IR-lowering | Generates the [intermediate representation] from the abstract syntax tree | [IR], [Emitter.swift] | +| LLVM IR generation | Convert Hylo IR into [LLVM] IR | [Transpilation.swift], [Swifty-LLVM] | +| Machine Code Generation | This is completely handled by [LLVM] | | These top-level stages of the compiler are laid out in [Driver] where you can see the outline of the compilation phases with their entry points. @@ -76,25 +76,75 @@ defined in `Module+*` files of [IR/Analysis]. After these passes the IR should be valid and executable by a *theoretical* Hylo VM. Some [more passes] may be necessary depending on the target. +## Important compiler modules + +![PlantUML model](https://www.plantuml.com/plantuml/png/VL1TQiCm37xNANI1v01xA1qT6u4MHbRsxYLQOkBAHNOAPUpkmtYDxItqoVv-97tIrdabVZuPEAjkKWFXs2tV9z4N4bZmirZsn3r-0UADV7mxPkXLYQCC7wzdsetKJPg7FxG3mEZ7gjh4FeN_7XqRO90nmVZYRzbtIjwqPMPMvkgqowY3ui64a5dLplRqGboKxlVlX-0Pkp3cTg7uObrluaOFgFbY9tAvaY-nQcZQDNf9kog6F4c00B6gGY-0UDnQz33wV_DbPeSVz6c585UIlsbBrvQudq_c1m00) + +[comment]: # (To generate this URL, copy the content of Modules.puml in http://www.plantuml.com/plantuml/uml/, and get the URL from there) + +**Legend**: +| Item | Meaning | +| -- | -- | +| package | A folder in the source repository | +| solid arrow | Dependency | +| dotted arrow | phase invocation / "runs before" relations | + +**Packages**: +| Package | Description | +| -- | -- | +| [hc] | The actual compiler executable. Just calls [Driver]. +| [Driver/] | Defines and executes the stages of the compiler. Takes care of compiler arguments. | +| [Frontend] | The frontend of the compiler. Handles lexing, parsing and type checking. Also stores the AST representation of the program. | +| [AST/] | The structures used to describe the abstract syntax tree of the input program. | +| [Parse] | The lexer and the parser for the Hylo source files. | +| [TypeChecking] | Implements the type checking of the Hylo programs. | +| [Types] | Defines the possible types that a Hylo entity might have. | +| [IR] | Defines the intermediate representation of Hylo programs, and the operarations associated with it. | +| [Analysis] | The analysis phases that can be run on the IR. | +| [Mangling] | The name-mangling algorithm used by Hylo. | +| [CodeGen] | Implements the code-generation phase for Hylo compiler. Currently only implements LLVM code generation. | +| [LLVM/] | The code generation into LLVM. | + [Swift]: https://en.wikipedia.org/wiki/Swift_(programming_language) [LLVM]: https://en.wikipedia.org/wiki/LLVM [SPM]: https://www.swift.org/package-manager/ [intermediate representation]: https://en.wikipedia.org/wiki/Intermediate_representation [abstract syntax tree]: https://en.wikipedia.org/wiki/Abstract_syntax_tree -[Driver]: ../Sources/Driver/Driver.swift -[Package.swift]: ../Package.swift -[Sources]: ../Sources -[Tests]: ../Tests -[Tools]: ../Tools -[Examples]: ../Examples -[Library]: ../StandardLibrary -[AST]: ../Sources/FrontEnd/AST/AST.swift -[ScopedProgram]: ../Sources/FrontEnd/ScopedProgram.swift -[TypedProgram]: ../Sources/FrontEnd/TypedProgram.swift -[Instruction]: ../Sources/IR/Operands/Instruction/ -[Emitter]: ../Sources/IR/Emitter.swift -[IR/Module]: ../Sources/IR/Module.swift -[IR/Program]: ../Sources/IR/Program.swift -[IR/Analysis]: ../Sources/IR/Analysis/ -[more passes]: ../Sources/IR/Analysis/Module+Depolymorphize.swift +[Driver]: ../../Sources/Driver/Driver.swift +[Package.swift]: ../../Package.swift +[Sources]: ../../Sources +[Tests]: ../../Tests +[Tools]: ../../Tools +[Examples]: ../../Examples +[StandardLibrary]: ../../StandardLibrary +[AST]: ../../Sources/FrontEnd/AST/AST.swift +[ScopedProgram]: ../../Sources/FrontEnd/ScopedProgram.swift +[TypedProgram]: ../../Sources/FrontEnd/TypedProgram.swift +[Instruction]: ../../Sources/IR/Operands/Instruction/ +[Emitter]: ../../Sources/IR/Emitter.swift +[IR/Module]: ../../Sources/IR/Module.swift +[IR/Program]: ../../Sources/IR/Program.swift +[IR/Analysis]: ../../Sources/IR/Analysis/ +[more passes]: ../../Sources/IR/Analysis/Module+Depolymorphize.swift + +[Lexer.swift]: ../../Sources/FrontEnd/Parse/Lexer.swift +[Token.swift]: ../../Sources/FrontEnd/Parse/Token.swift +[Parser.swift]: ../../Sources/FrontEnd/Parse/Parser.swift +[TypeChecking]: ../../Sources/FrontEnd/TypeChecking +[TypeChecker.swift]: ../../Sources/FrontEnd/TypeChecking/TypeChecker.swift +[IR]: ../../Sources/IR +[Emitter.swift]: ../../Sources/IR/Emitter.swift +[Transpilation.swift]: ../../Sources/CodeGen/LLVM/Transpilation.swift +[Swifty-LLVM]: https://github.com/hylo-lang/Swifty-LLVM +[hc]: ../../Sources/hc +[Driver/]: ../../Sources/Driver +[Frontend]: ../../Sources/Frontend +[AST/]: ../../Sources/Frontend/AST +[Parse]: ../../Sources/Frontend/Parse +[Types]: ../../Sources/Frontend/Types +[Analysis]: ../../Sources/IR/Analysis +[Mangling]: ../../Sources/IR/Mangling +[CodeGen]: ../../Sources/CodeGen +[LLVM/]: ../../Sources/CodeGen/LLVM + diff --git a/Docs/Compiler/Modules.puml b/Docs/Compiler/Modules.puml new file mode 100644 index 000000000..0f2b2e822 --- /dev/null +++ b/Docs/Compiler/Modules.puml @@ -0,0 +1,39 @@ +@startuml +package hc +package Driver + +package Frontend { + package AST + package Parse + package TypeChecking + package Types + + Parse --> AST + TypeChecking --> AST + TypeChecking --> Types + AST -> Types + Types -> AST +} + +package IR { + package Analysis + package Mangling +} + +package CodeGen { + package LLVM +} + +hc -> Driver + +Driver ..> Parse: invokes +Driver ..> TypeChecking: invokes +Driver ..> IR: invokes +Driver ..> Analysis: invokes +Driver ..> LLVM: invokes + +LLVM --> IR + +Parse .right.> TypeChecking: runs before +TypeChecking .right.> IR: runs before +@enduml \ No newline at end of file From 62eb3f51099c37e90947fe8f3104c9d6c9b485cd Mon Sep 17 00:00:00 2001 From: Lucian Radu Teodorescu Date: Fri, 3 Jan 2025 13:54:52 +0200 Subject: [PATCH 2/2] Fix case for `FrontEnd` folder. --- Docs/Compiler/CompilerArchitecture.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Docs/Compiler/CompilerArchitecture.md b/Docs/Compiler/CompilerArchitecture.md index 7f2034e06..0f73fe6fb 100644 --- a/Docs/Compiler/CompilerArchitecture.md +++ b/Docs/Compiler/CompilerArchitecture.md @@ -94,7 +94,7 @@ necessary depending on the target. | -- | -- | | [hc] | The actual compiler executable. Just calls [Driver]. | [Driver/] | Defines and executes the stages of the compiler. Takes care of compiler arguments. | -| [Frontend] | The frontend of the compiler. Handles lexing, parsing and type checking. Also stores the AST representation of the program. | +| [FrontEnd] | The frontend of the compiler. Handles lexing, parsing and type checking. Also stores the AST representation of the program. | | [AST/] | The structures used to describe the abstract syntax tree of the input program. | | [Parse] | The lexer and the parser for the Hylo source files. | | [TypeChecking] | Implements the type checking of the Hylo programs. | @@ -139,10 +139,10 @@ necessary depending on the target. [Swifty-LLVM]: https://github.com/hylo-lang/Swifty-LLVM [hc]: ../../Sources/hc [Driver/]: ../../Sources/Driver -[Frontend]: ../../Sources/Frontend -[AST/]: ../../Sources/Frontend/AST -[Parse]: ../../Sources/Frontend/Parse -[Types]: ../../Sources/Frontend/Types +[FrontEnd]: ../../Sources/FrontEnd +[AST/]: ../../Sources/FrontEnd/AST +[Parse]: ../../Sources/FrontEnd/Parse +[Types]: ../../Sources/FrontEnd/Types [Analysis]: ../../Sources/IR/Analysis [Mangling]: ../../Sources/IR/Mangling [CodeGen]: ../../Sources/CodeGen