Reorganize the repository, update readmes and changelogs

.github/workflows/release.yml

@@ -1,4 +1,4 @@
-name: Release
+name: Build release binaries
 
 on:
   push:
@@ -11,7 +11,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v3
       - name: Install rust stable
         uses: actions-rs/toolchain@v1
         with:
@@ -24,13 +24,13 @@ jobs:
       - name: Upload release artifact
         uses: softprops/action-gh-release@v1
         with:
-          files: target/release/hlbc-cli
+          files: target/release/hlbc
   release-windows:
     name: Build release (windows)
     runs-on: windows-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v3
       - name: Install rust stable
         uses: actions-rs/toolchain@v1
         with:
@@ -43,4 +43,4 @@ jobs:
       - name: Upload release artifact
         uses: softprops/action-gh-release@v1
         with:
-          files: target/release/hlbc-cli.exe
+          files: target/release/hlbc.exe

.gitignore

@@ -1,2 +1,3 @@
 /target
 .idea/
+*.hl

Cargo.toml

@@ -1,34 +1,2 @@
-[package]
-name = "hlbc"
-version = "0.4.0"
-authors = ["Guillaume Anthouard <[email protected]>"]
-edition = "2021"
-rust-version = "1.56"
-description = "Hashlink bytecode disassembler, analyzer and decompiler"
-repository = "https://github.com/Gui-Yom/hlbc"
-license = "MIT"
-keywords = ["hashlink", "bytecode", "disassembler", "analysis", "decompiler"]
-categories = ["parser-implementations", "visualization", "compilers"]
-include = ["src/**/*", "LICENSE", "README.md"]
-
 [workspace]
-members = ["hlbc-derive", "hlbc-decompiler", "hlbc-cli"]
-
-# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
-
-[dependencies]
-# Endianess utilities
-byteorder = "1"
-# Advanced formatting functionalities
-fmtools = "0.1"
-# Compile time code generation for hlbc::Opcode
-hlbc-derive = { version = "0.3", path = "hlbc-derive" }
-# Graph utilities
-petgraph = { version = "0.6", default-features = false, features = ["graphmap"], optional = true }
-# Error types
-thiserror = "1"
-
-[features]
-default = ["graph"]
-# Generate a callgraph
-graph = ["petgraph"]
+members = ["hlbc-derive", "hlbc", "hlbc-decompiler", "hlbc-cli"]

README.md

@@ -1,76 +1,41 @@
-# hlbc ![Crates.io](https://img.shields.io/crates/v/hlbc?label=hlbc) ![Crates.io](https://img.shields.io/crates/v/hlbc-cli?label=hlbc-cli)
+<div align="center">
+    <h1><b>H</b>ash<b>l</b>ink <b>b</b>yte<b>c</b>ode</h1>
+    <a href="https://crates.io/crates/hlbc">
+        <img src="https://img.shields.io/crates/v/hlbc?label=hlbc">
+    </a>
+    <a href="https://crates.io/crates/hlbc-decompiler">
+        <img src="https://img.shields.io/crates/v/hlbc-decompiler?label=hlbc-decompiler">
+    </a>
+    <a href="https://crates.io/crates/hlbc-cli">
+        <img src="https://img.shields.io/crates/v/hlbc-cli?label=hlbc-cli">
+    </a>
+    <br/>
+    This repository contains a collection of Rust crates and cli tools to load, disassemble, decompile and
+    analyze <a href="https://hashlink.haxe.org/">Hashlink</a> bytecode.
+    <br/>
+    <img src="hlbc-cli/screenshot.png">
+</div>
 
-[**H**ash**l**ink](https://hashlink.haxe.org/) **b**yte**c**ode disassembler, analyzer, decompiler and assembler.
-
-*This crate is a library, see [hlbc-cli](hlbc-cli) for an actual program to use.*
-
----
-
-## Features
-
-- Parse the whole bytecode file or any bytecode element
-- Display any bytecode element
-- Restore all possible names
-- Link elements between them (with manual references for flexibility)
-- Link elements with their debug information
-- Serialize bytecode back to bytes
-- Decompiler to haxe source code for classes and functions
-- Commandline interface to use the features of this library ([hlbc-cli](hlbc-cli))
-
-## Planned features
-
-- Integrate with the Haxe/Hashlink standard library to restore more names, exclude them from analysis and such
-- Gui to render the callgraph in real time
-- Gui to render instructions and decompiled code
-- C API to integrate with other tools
-- Text search engine to search for strings and names
-- Assemble and inject bytecode or inject haxe source code directly
-
-## Changelog
-
-See [CHANGELOG.md](CHANGELOG.md).
-
-## Decompiler
+## Repository structure
 
-The decompiler is currently incomplete (and will probably always be).
-[See the wiki](https://github.com/Gui-Yom/hlbc/wiki/Decompilation) for examples of decompilation output.
+- `data/` : Haxe source files to test the tools
+- `hlbc/` : Core library to load and disassemble bytecode
+- `hlbc-cli/` : CLI frontend for `hlbc`
+- `hlbc-decompiler/` : Decompiler library
+- `hlbc-derive/` : helper proc macros for hlbc
 
 ## Wiki
 
-A wiki detailing the specifics of Hashlink bytecode is available [here](https://github.com/Gui-Yom/hlbc/wiki).
-The wiki also details the inner workings of haxe to hashlink compilation and decompilation.
+A wiki detailing the specifics of Hashlink bytecode is available [here](https://github.com/Gui-Yom/hlbc/wiki) or by
+using the command `wiki`.
 
-## Macros
-
-There are 98 different bytecode instructions, there is no way I manually write code for it each time. Most of the code
-for these opcodes is generated through a proc macro (see [hlbc-derive](/hlbc-derive)).
-The only time I needed to write 98 different branches was for the formatting used when displaying the
-instructions ([src/fmt.rs](src/fmt.rs)).
-
-## Repository structure
+## Planned features
 
-- `/` : hlbc (lib)
-- `hlbc-derive/` : hlbc-derive, helper proc macros for hlbc
-- `hlbc-cli/` : CLI using hlbc
-- `data/` : Haxe source files to test the decompiler
+- GUI
+- C API
 
 ## Credits
 
-Development of this crate wouldn't be possible without the [hashlink](https://github.com/HaxeFoundation/hashlink) source
-code. Some algorithms are directly derived from the original C code reading bytecode files.
-
-## Alternatives
-
-This library is made in Rust, a C API is in the works which could permit using this lib in many other projects, but for
-now it is only Rust friendly.
-
-Other alternatives include :
-
-- Tinkering directly with the [hashlink](https://github.com/HaxeFoundation/hashlink) source code in C
-- Using the in-progress [**_*dashlink*_**](https://github.com/Steviegt6/dashlink) made in Haxe but probably compilable
-  to many other languages.
-
-## Notes about using Rust
-
-Rust isn't a very good fit for this project, the whole bytecode is a large graph and self-references do not mix well
-with Rust. Further iterations of this crate could make use of an arena.
+Development of this project would not have been possible without
+the [hashlink](https://github.com/HaxeFoundation/hashlink) source code. Some algorithms are directly derived from the
+original C code reading bytecode files.

data/README.md

@@ -1,3 +1,11 @@
 # Decompiler test data
 
 The files in this directory are used to test the decompiler output.
+
+## Building
+
+To build a particular source file :
+
+```shell
+haxe -hl <file>.hl -main <file>.hx
+```

hlbc-cli/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Changelog
 
-This is the changelog for `hlbc-cli` the cli accompanying the `hlbc` library.
+This is the changelog for `hlbc-cli`, other crates have their own changelog.
 All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### New
 
 - Updated to `hlbc` 0.4 ([Changelog](../CHANGELOG.md))
+- Use `hlbc-decompiler`
 
 ### Fixed
 

hlbc-cli/Cargo.toml

@@ -4,13 +4,18 @@ version = "0.4.0"
 authors = ["Guillaume Anthouard <[email protected]>"]
 edition = "2021"
 rust-version = "1.56"
-description = "Hashlink bytecode disassembler, analyzer and decompiler"
+description = "Hashlink bytecode disassembler, analyzer and decompiler cli"
 repository = "https://github.com/Gui-Yom/hlbc"
 license = "MIT"
 categories = ["command-line-utilities", "parser-implementations", "visualization", "compilers"]
+include = ["src/**/*", "README.md", "CHANGELOG.md"]
 
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
+[[bin]]
+name = "hlbc"
+path = "src/main.rs"
+
 [dependencies]
 # Result type
 anyhow = "1"
@@ -21,7 +26,7 @@ chumsky = { version = "0.8" }
 # CLI args
 clap = { version = "3", features = ["derive"] }
 # Core functionnality
-hlbc = { version = "0.4", path = "..", default-features = false }
+hlbc = { version = "0.4", path = "../hlbc", default-features = false }
 # Decompiler
 hlbc-decompiler = { version = "0.4", path = "../hlbc-decompiler" }
 # File system watching

hlbc-cli/README.md

@@ -1,18 +1,19 @@
-# hlbc-cli ![Crates.io](https://img.shields.io/crates/v/hlbc-cli?label=hlbc-cli)
+# hlbc-cli [![Crates.io](https://img.shields.io/crates/v/hlbc-cli?label=hlbc-cli)](https://crates.io/crates/hlbc-cli)
 
-A CLI to navigate through [Hashlink](https://hashlink.haxe.org/) bytecode files. This is the command line interface
-for [hlbc](https://github.com/Gui-Yom/hlbc), the **H**ash**l**ink **b**yte**c**ode
-disassembler, analyzer, decompiler and assembler.
+[**H**ash**l**ink](https://hashlink.haxe.org/) **b**yte**c**ode disassembler, decompiler and analyzer command line
+interface.
 
-*This crate is a program, see [hlbc](https://github.com/Gui-Yom/hlbc) for a library to use.*
+*This crate is a program, see [hlbc](https://crates.io/crates/hlbc) for the core bytecode library
+or [hlbc-decompiler](https://crates.io/crates/hlbc-decompiler) for the decompiler library*
 
 ![screenshot](screenshot.png)
 
 ---
 
 ## Installation
 
-Download a prebuilt binary from the [releases page](https://github.com/Gui-Yom/hlbc/releases).
+Download a prebuilt binary from the [releases page](https://github.com/Gui-Yom/hlbc/releases) (built from the CI,
+Windows & Linux).
 
 Or build from the latest version :
 
@@ -24,19 +25,19 @@ Or build the crate from the latest sources :
 
 ```shell
 git clone https://github.com/Gui-Yom/hlbc
-cd hlbc/hlbc-cli
+cd hlbc
 cargo build --release
-# The resulting binary can be found in hlbc/target/release
+# The resulting binary can be found in target/release
 ```
 
 ## Usage
 
-`hlbc-cli <file> [-c <command>] [-w <command>]`
+`hlbc <file> [-c <command>] [-w <command>]`
 
 You get access to a prompt where you can enter commands.
 
 You can execute commands on startup with the `-c` switch.
-e.g. Dump all strings from the bytecode then exit : `hlbc-cli main.hl -c "s ..; exit"`.
+e.g. Dump all strings from the bytecode then exit : `hlbc main.hl -c "s ..; exit"`.
 If you omit the `exit` command, the app will simply launch the normal prompt after executing the startup commands.
 
 With `-w`, the given command will execute each time the file changes. The cli won't show a command prompt.
@@ -46,10 +47,12 @@ present in the `PATH`.
 
 ## Commands
 
-- `info` General information about the bytecode
+- `exit` Exit the program
 - `help` Help message
-- `entrypoint` Get the bytecode entrypoint
 - `explain <op>` Get information about an opcode
+- `wiki` Open the bytecode wiki page in a browser
+- `info` General information about the bytecode
+- `entrypoint` Get the bytecode entrypoint
 - `i|int <idx>` Get the int at index
 - `f|float <idx>` Get the float at index
 - `s|string <idx>` Get the string at index
@@ -69,7 +72,7 @@ present in the `PATH`.
 - `saveto <filename>` Serialize the bytecode to a file
 - `callgraph <findex> <depth>` Create a dot call graph from a function and a max depth
 - `decomp <findex>` Decompile a function
-- `decomptype <idx>` Decompile a class
+- `decompt <idx>` Decompile a class
 
 ### Indexes
 
@@ -78,15 +81,16 @@ Where `..10` means '*select the first 10 items*' and `..` means '*display everyt
 
 ## Decompiler
 
-More info about the decompiler [here](https://github.com/Gui-Yom/hlbc/blob/master/README.md#decompiler).
+The decompiler has its own crate ! More info [here](https://github.com/Gui-Yom/hlbc/blob/master/hlbc-decompiler).
 
 ## Changelog
 
 See [CHANGELOG.md](CHANGELOG.md).
 
 ## Wiki
 
-A wiki detailing the specifics of Hashlink bytecode is available [here](https://github.com/Gui-Yom/hlbc/wiki).
+A wiki detailing the specifics of Hashlink bytecode is available [here](https://github.com/Gui-Yom/hlbc/wiki) or by
+using the command `wiki`.
 
 ## Planned features
 

hlbc-decompiler/CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+This is the changelog for `hlbc-decompiler`, other crates have their own changelogs.
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased](https://github.com/Gui-Yom/hlbc/compare/v0.3.0...HEAD)
+
+### Added
+
+- If and else statements are unified for better formatting and easier post-processing
+- New (currently hidden) ast post-processing step (AST-PP) to improve the decompiler output
+- New AST_PP : if-expressions
+- New AST-PP : string concatenations :(`__add__("a", "b")` to `"a" + "b"`)
+- New AST-PP : Hide calls to itos. Int to strings conversions are usually hidden.
+- Ability to generate comments in the AST
+- Display closure if InstanceClosure on an enum (the enum is the closure capture)
+
+### Fixed
+
+- Remove excessive `;` in constructor calls
+
+---
+
+Previous releases of the decompiler can be found in the `hlbc` crate changelog.

hlbc-decompiler/Cargo.toml

@@ -15,7 +15,7 @@ categories = ["visualization", "compilers"]
 [dependencies]
 # Advanced formatting functionalities
 fmtools = "0.1"
-hlbc = { version = "0.4", path = ".." }
+hlbc = { version = "0.4", path = "../hlbc" }
 # Graph utilities
 petgraph = { version = "0.6", default-features = false, features = ["graphmap"], optional = true }
 

hlbc-decompiler/README.md

@@ -0,0 +1,9 @@
+# hlbc-decompiler [![Crates.io](https://img.shields.io/crates/v/hlbc-decompiler?label=hlbc-decompiler)](https://crates.io/crates/hlbc-decompiler)
+
+[**H**ash**l**ink](https://hashlink.haxe.org/) **b**yte**c**ode disassembler and analyzer.
+
+*This crate is a library, see [hlbc-cli](https://crates.io/crates/hlbc-cli) for an actual program to use.*
+
+---
+
+[See the wiki](https://github.com/Gui-Yom/hlbc/wiki/Decompilation) for examples of decompilation output.