README.md

OBuilder

OBuilder takes a build script (similar to a Dockerfile) and performs the steps in it in a sandboxed environment.

After each step, OBuilder uses the snapshot feature of the filesystem (ZFS or Btrfs) to store the state of the build. There is also an Rsync backend that copies the build state. On Linux, it uses runc to sandbox the build steps, but any system that can run a command safely in a chroot could be used. Repeating a build will reuse the cached results where possible.

OBuilder can also use Docker as a backend (fully replacing of runc and the snapshotting filesystem) on any system supported by Docker (Linux, Windows, …).

OBuilder stores the log output of each build step. This is useful for CI, where you may still want to see the output even if the result was cached from some other build.

At present, the initial base image is fetched from Docker Hub using docker pull on Linux and then snapshotted into the store. Other systems use a conceptually similar process with the implementation in each platform section.

Usage

OBuilder is designed to be used as a component of a build scheduler such as OCluster. However, there is also a command-line interface for testing.

To check that the system is working correctly, you can run a healthcheck. This checks that Docker is running and then does a simple test build (pulling the busybox image if not already present):

$ obuilder healthcheck --store=zfs:tank
Healthcheck passed

To build example.spec (which builds OBuilder itself) using the ZFS pool tank to cache the build results:

$ obuilder build -f example.spec . --store=zfs:tank

To use Btrfs directory /mnt/btrfs for the build cache, use --store=btrfs:/mnt/btrfs or specify a directory for Rsync to use --store=rsync:/rsync.

Notes

Some operations (such as deleting btrfs snapshots) require root access. OBuilder currently uses sudo as necessary for such operations.

You should only run one instance of the command-line client at a time with a given store. OBuilder does support concurrent builds, but they must be performed using a single builder object:

• If you try to perform an operation that is already being performed by another build, it will just attach to the existing build.

• The new client will get the logs so far, and then stream new log data as it arrives.

• If a client cancels, it just stops following the log. The operation itself is cancelled if all its clients cancel.

OBuilder calculates a digest of the input files to decide whether a copy step needs to be repeated. However, if it decides to copy the file to the build sandbox, it does not check the digest again. Also, it only checks that it is not following symlinks during the initial scan. Therefore, you must not modify the input files while a build is in progress.

Failed build steps are not cached.

Files and directories in the store may have owners and groups that only make sense in the context of some container. The store should therefore be configured so that other processes on the host (which might have the same IDs by coincidence) cannot reach them, e.g. by chmod go-rwx /path/to/store.

Sync operations can be very slow, especially on btrfs. They're also unnecessary, since if the computer crashes then we'll just discard the whole build and start again. If you have runc version v1.0.0-rc92 or later, you can pass the --fast-sync option, which installs a seccomp filter that skips all sync syscalls. However, if you attempt to use this with an earlier version of runc then sync operations will instead fail with EPERM.

The build specification language

The spec files are loosely based on the Dockerfile format. The main difference is that the format uses S-expressions rather than a custom format, which should make it easier to generate and consume it automatically.

When performing a build, the user gives OBuilder a specification file (as described below), and a source directory, containing files which may be copied into the image using copy.

((from BASE) OP…)

Example:

((from busybox@sha256:d366a4665ab44f0648d7a00ae3fae139d55e32f9712c67accd604bb55df9d05a)
 (shell /bin/sh -c)
 (run (shell "echo hello world")))

BASE identifies a Docker image, which will be fetched using docker pull and imported into the OBuilder cache. OBuilder will not check for updates, so BASE should include a digest identifying the exact image, as shown above.

The operations are performed in order. Each operation gets a build context and a filesystem snapshot, and may produce a new context and a new snapshot. The initial filesystem snapshot is BASE. run and copy operations create new snapshots.

The initial context is supplied by the user (see build.mli for details). By default:

• The environment is taken from the Docker configuration of BASE.
• The user is (uid 0) (gid 0) on Linux, (name ContainerAdministrator) on Windows.
• The workdir is /, C:/ on Windows.
• The shell is bash -c, C:\Windows\System32\cmd.exe /S /C on Windows.

Multi-stage builds

You can define nested builds and use the output from them in copy operations. For example:

((build dev
 ((from ocaml/opam:alpine-3.18-ocaml-5.0)
  (user (uid 1000) (gid 1000))
  (workdir /home/opam)
  (run (shell "echo 'print_endline {|Hello, world!|}' > main.ml"))
  (run (shell "opam exec -- ocamlopt -ccopt -static -o hello main.ml"))))
 (from alpine:3.18)
 (shell /bin/sh -c)
 (copy (from (build dev))
  (src /home/opam/hello)
  (dst /usr/local/bin/hello))
 (run (shell "hello")))

At the moment, the (build …) items must appear before the (from …) line.

workdir

(workdir DIR)

Example:

(workdir /usr/local)

This operation sets the current working directory used for the following commands, until the next workdir operation. If the path given is relative, it is combined with the previous setting. WARNING Workdir implementation is idiosyncratic on macOS, use with care.

shell

(shell ARG…)

Example:

(shell /bin/bash -c)

This sets the shell used for future (run (shell COMMAND)) operations. The command run will be this list of arguments followed by the single argument COMMAND.

run

(run
 (cache CACHE…)?
 (network NETWORK…)?
 (secrets SECRET…)?
 (shell COMMAND))

Examples:

(run (shell "echo hello"))

(run
 (cache (opam-archives (target /home/opam/.opam/download-cache)))
 (network host)
 (secrets (password (target /secrets/password)))
 (shell "opam install utop"))

Runs the single argument COMMAND using the values in the current context (set by workdir and shell).

The (cache CACHE…) field can be used to mount one or more persistent caches for the command. Each CACHE takes the form (NAME (target PATH)), where NAME uniquely identifies the cache to use and PATH is the mount point within the container.

If the cache NAME does not yet exist then it is first created as an empty directory, owned by the user in the build context. A mutable copy of the cache is created for the command. When the command finishes (whether successful or not) this copy becomes the new version of the cache, unless some other command updated the same cache first, in which case this one is discarded.

The (network NETWORK…) field specifies which network(s) the container will be connected to. (network host) is a special value which runs the container in the host's network namespace. Otherwise, a fresh network namespace is created for the container, with interfaces for the given networks (if any).

Currently, no other networks can be used, so the only options are host or an isolated private network.

The (secrets SECRET…) field can be used to request values for chosen keys, mounted as read-only files in the image. Each SECRET entry is under the form (ID (target PATH)), where ID selects the secret, and PATH is the location of the mounted secret file within the container. The sandbox context API contains a secrets parameter to provide values to the runtime. If a requested secret isn't provided with a value, the runtime fails. Use the --secret ID:PATH option to provide the path of the file containing the secret for ID. When used with Docker, make sure to use the BuildKit syntax, as only BuildKit supports a --secret option. (See https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information)

copy

(copy
 (from …)?
 (src SRC…)
 (dst DST)
 (exclude EXCL…)?)

Examples:

(copy
 (src .)
 (dst build/)
 (exclude .git _build))

(copy
 (src platform.ml.linux)
 (dst platform.ml))

This copies files, directories and symlinks from the source directory (provided by the user when building) into the image. If DST does not start with / then it is relative to the current workdir.

It has two forms:

• If DST ends with / then it copies each item in SRC to the directory DST.
• Otherwise, it copies the single item SRC as DST.

Files whose basenames are listed in exclude are ignored. If exclude is not given, the empty list is used. At present, glob patterns or full paths cannot be used here.

If (from (build NAME)) is given then the source directory is the root directory of the named nested build. Otherwise, it is the source directory provided by the user.

Notes:

• Unlike Docker's COPY operation, OBuilder copies the files using the current user and group IDs, as set with (user …).

• Both SRC and DST use / as the directory separator on all platforms.

• The copy is currently done by running tar inside the container to receive the files. Therefore, the filesystem must have a working tar binary. On Windows when using the Docker backend, OBuilder provides a tar binary.

• On Windows, copying from a build step image based on Nano Server isn't supported.

user

(user (uid UID) (gid GID))
(user (name NAME)) ; on Windows and FreeBSD

Example:

(user (uid 1000) (gid 1000))

This updates the build context to set the user and group IDs used for the following copy and run commands. Note that only numeric IDs are supported.

env

(env NAME VALUE)

Example:

(env OPTIONS "-O2 -Wall")

Updates the build context so that the environment variable NAME has the value VALUE in future run operations.

Convert to Dockerfile format

You can convert an OBuilder spec to a Dockerfile like this:

$ obuilder dockerfile -f example.spec > Dockerfile

The dockerfile should work the same way as the spec file, except for these limitations:

• In (copy (excludes …) …) the excludes part is ignored. You will need to ensure you have a suitable .dockerignore file instead.

• If you want to include caches or to use secrets, use --buildkit to output in the extended BuildKit syntax.

• All (network …) fields are ignored, as Docker does not allow per-step control of networking.

Other Platforms

OBuilder abstracts over a fetching mechanism for the Docker base image, the sandboxing for the execution of build steps and the store for the cache. This makes OBuilder extremely portable and there exists FreeBSD, macOS and Windows backends. The FreeBSD backend uses jails and ZFS, the macOS backend re-uses ZFS and user isolation, and the Windows backend currently requires Docker for Windows installed.

Licensing

OBuilder is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.