HACKING.md

Hacking on Dune

This section is for people who want to work on Dune itself.

Setup

Create an Opam switch with the required dependencies

From the project root, execute

$ make dev-switch
# This takes some time

Building

Common usage

To build and install the development version, run the following while in the project's root directory:

$ make dev

Note that this will build the dune binary twice: once using the bootstrap build procedure and then one normal dune build using the freshly produced dune binary.

For a short synopsis on the usual make commands used in development, run

$ make
# or
$ make help

A note on bootstrapping

In order to build itself, Dune uses a micro dune written as a single boot/duneboot.ml file. This micro build system cannot read dune files and instead has the configuration hard-coded in boot/libs.ml. This latter file is automatically updated during development when we modify the dune files in the repository. boot/duneboot.ml itself is built with a single invocation of ocamlopt or ocamlc via the bootstrap.ml ocaml script.

boot/duneboot.ml builds a dune.exe binary at the root of the source tree and uses this binary to build everything else.

make dev takes care of bootstrapping if needed, but if you want to just run the bootstrapping step itself, build the dune.exe target with

make dune.exe

OCaml compatibility test

Install opam switches for all the entries in the dune-workspace.dev file and run:

$ make all-supported-ocaml-versions

Repository organization

• vendor/ contains dependencies of Dune, that have been vendored
• plugin/ contains the API given to dune files that are OCaml scripts
• src/ contains the core of Dune
• bin/ contains the command line interface
• doc/ contains the manual and rules to generate the manual pages

Design

Dune (nee "JBuilder") was initially designed to sort out the public release of Jane Street packages which became incredibly complicated over time. It is still successfully used for this purpose.

One necessary feature to achieve this is the ability to precisely report the external dependencies necessary to build a given set of targets without running any command, just by looking at the source tree. This is used to automatically generate the <package>.opam files for all Jane Street packages.

To implement this, the build rules are described using a build data type, which is defined in src/dune/build.mli. In the end it makes the development of the internal rules of Dune very composable and quite pleasant.

To deal with process multiplexing, Dune uses a simplified Lwt/Async-like monad, implemented in src/fiber/fiber.mli.

Tests

Dune uses cram style tests for its test suite. The test suite is contained in test/blackbox-tests. A single test consists of a test directory in the test-cases/ sub directory which contains a run.t file defining the test.

An example run.t file:

A description of the test. The command running the tests is preceded by
two spaces and a $. The expected output of the command is also indented by
two spaces and is right below the command (note the absence of a $)

  $ echo "the expected output is below"
  the expected output is below

Running the entire test suite is done with $ make test. A particular test can be executed with $ dune build @<test-name>.

Running the test will execute the command after the $ and its output will be compared against the expected output right below the command. Any differences will result in a test failure.

Adding a Test

Simply add a new directory in test/blackbox-tests/test-cases and then $ make generate the rules for the test , followed by $ make promote to accept the new rules.

Accepting Corrections

A failing expect test will generate a diff between the expected and actual output. If the new output generated by the command is satisfactory, it can be promoted with the $ make promote command.

Documentation

User documentation

User documentation lives in the ./doc directory.

In order to build the user documentation, you must install

• python-sphinx, and
• sphinx_rtd_theme

Build the documentation with

make doc

For automatically updated builds, you can install sphinx-autobuild, and run

make livedoc

Code flow

• src/dune/dune_file.mli contains the internal representation of dune files and the parsing code
• src/dune/dune_load.mli contains the code to scan a source tree and build the internal database by reading the dune files
• src/dune/gen_rules.mli contains all the build rules of Dune
• src/dune/build_system.mli contains a trivial implementation of a Build system. This is what Jenga will provide when implementing the bridge