Skip to content

Docker image guidelines

Damian Mee edited this page Oct 16, 2019 · 7 revisions

This consolidate common patterns and best practices used across all images built & provided by @lncm. All rules are numbered for the ease of reference.

NOTE: ⚠️ This is a DRAFT. Any numbers, content and ordering can and will change.

1. Own code

This applies to project written mostly, or completely in-house (ex. invoicer). Especially ones that are meant to be used as composables in eg. docker-compose.yml files within noma.

1.1. Github repos should be named lncm/<project-name>

1.2. Docker Hub repos should inherit the name, lncm/<project-name>

1.2. Dockerfiles should be placed in repo root


2. External code

This applies to all code that's written externally, that we only provide a wrapper for (ex. bitcoind, lnd).

2.1. Github repos should be named lncm/docker-<project-name>

2.2. Docker Hub repos should drop the docker- prefix, and be named lncm/<project-name>

2.3. Dockerfiles should be placed in minor-version directories

That also means that subsequent releases of 0.0.patch versions change the content of the directory, while releases of new 0.minor.0 versions should create a new directory.

2.4. Changes to original source should be applied using .patch files

2.5. Release Candidate (-rc) releases should not be published

2.6. Images MUST NOT introduce opinionated behavior

2.7. Images MUST embrace default behaviour of the software within

2.8. Images MAY be extended to introduce opinionated behavior

Using a variant might be a viable approach, but perhaps for smaller changes an in-place Dockerfile can be created and built by the users.


3. Dockerfile

This defines a set of rules to keep in mind while writing, and within a Dockerfile.

3.1. Each committed Dockerfile MUST be buildable

Meaning: each should have sane defaults, and always allow for a simple docker build . right after git clone. Image built by default should target host architecture, and be locally runnable after build.

3.2. Explicit minor version should always be used in FROM clauses

3.3. Should have a final stage that only includes minimally necessary ingredients

3.4. Should specify LABELS:

3.4.1. maintainer="" for the first/primary maintainer, and maintainer.<n>="" for all subsequent ones

Example:

LABEL maintainer="Dat Guy <[email protected]>"
LABEL maintainer.1="Dat Otha Guy <[email protected]>"

3.4.2. version="" stating the version of the code within

3.4.2. commit="" defining what commit the image was built from

NOTE: For external code commit should be set to the commit of our repository, specifically the one that triggered the automated build.

3.5. Projects targeting Go 1.13 and above, MAY build the same binaries twice using different bases, and only proceed if output binaries are identical

3.6. Should use alpine or scratch as base image

3.7. May use stages and targets

3.8. VOLUME must be used to expose all default app-data directories

3.9. EXPOSE must be used to document all ports software within might use

3.10. ENTRYPOINT should be pointing to a daemon-style binary within

3.11. CMD should only be set for de-facto standard flags, or ones that are required for the image to run

Examples:

# de-facto standard interfaces for ZMQ on Bitcoind
CMD ["-zmqpubrawblock=tcp://0.0.0.0:28332", "-zmqpubrawtx=tcp://0.0.0.0:28333"]

# required config path (if invoicer wouldn't specify default config within):
ENTRYPOINT ["/bin/invoicer"]
CMD ["-config", "~/.lncm/invoicer.conf"]

3.12. Should contain as many comments as necessary for even novices to understand/be able to Google it

3.13. COPY SHOULD be followed by integrity check verification

Example:

ENV BUILDFILE_HASH=a981094129632db17661613fe9d20e0e6b15aa498c65355e2088699b24c97f10
COPY ./build.sh .

# Verify that the build file is what's expected
RUN echo "${BUILDFILE_HASH}  build.sh" | sha256sum -c

4. Git Tags

This defines how the source code of the projects should be tagged for release.

4.1. MUST be GPG-signed

4.2. Should be opentimestamps-ed

4.3. Once pushed to remote, git tags MUST NOT be overridden

4.4. Should be in a format of: v<project-version>[-<variant-name>][+build<build-no>]

Valid examples:

v0.0.1
v0.18.1-nowallet
v0.17.0.1
v0.5.0+build666
v0.19.0-nowallet+build3

4.5. Must not drop version precision from upstream

4.6. Non-semver versions MAY be used if coming from upstream

4.7. +build<N> should be used to ex. indicate another build of the same upstream version

Examples:

  • base Alpine image updated
  • fix of a bug in build-process code
  • optimizations to the final image applied
  • version of qemu is changed

4.8. Git tag push SHOULD trigger an automated build, while pushes of commits to a branch SHOULD NOT


5. Docker Tags

This defines how already built images should be tagged for distribution.

5.1. Docker Tags corresponding directly to git tags MUST NOT be overridden

5.2. Shortened tags should be used

Note: Getting short-tag recommendations on Github Actions can be simplified by using: https://github.com/meeDamian/tag-suggestions

5.3. Docker manifest must be used, for projects targeting multiple architectures

5.4. :latest tag should not be created for stateful projects where it's not safe to skip versions


6. Variants

This defines how alternative flavors/variants should be handled.

6.1. Should be defined as variant-<variant-name>.patch files in minor-version directories

Example (create): To create a monitoring variant for 0.8/ version of lnd:

  1. Make sure all prior changes are committed/stashed/deleted within 0.8/

  2. Apply variant-specific changes to files

  3. Create .patch file using sth like:

    git diff --no-prefix --relative=0.8/ > 0.8/variant-monitoring.patch
    
  4. Add & commit 0.8/variant-monitoring.patch

  5. Un-apply all variant changes with ex: git checkout -- ., or git restore .

Example (apply): Later, to build said variant the patch can be applied with:

patch < variant-monitoring.patch` can be used.
Click to see variant-monitoring.patch
diff --git Dockerfile Dockerfile
index c3f2274..91b48fa 100644
--- Dockerfile
+++ Dockerfile
@@ -147,8 +147,11 @@ COPY  --from=cross-check  /bin/lncli  /bin/
 # Define a root volume for data persistence.
 VOLUME /root/.lnd

-# Expose lnd ports (rest, p2p, rpc respectively).
-EXPOSE 8080 9735 10009
+# Expose lnd ports (rest, monitoring, p2p, rpc respectively).
+EXPOSE 8080 8989 9735 10009

 # Specify the start command and entrypoint as the lnd daemon.
 ENTRYPOINT ["lnd"]
+
+# Okay to hardcode them here, as it's in a variant that specifically wants Prometheus
+CMD ["--prometheus.enable", "--prometheus.listen=0.0.0.0:8989"]
diff --git build.sh build.sh
index 3da6714..0f73bc8 100755
--- build.sh
+++ build.sh
@@ -43,6 +43,9 @@ TAGS="autopilotrpc invoicesrpc walletrpc routerrpc watchtowerrpc"
 # TODO: Check if still needed after Go v1.14 release
 TAGS="${TAGS} osusergo netgo static_build"

+# Added by yours truly (@lncm) to enable monitoring
+TAGS="${TAGS} monitoring"
+
 build() {
   binary_name=$1
   extra_tags=$2

6.2. Should be triggered by git tag containing variant within

More on howto: https://stackoverflow.com/questions/3418277/how-to-apply-git-diff-patch-without-git-installed

Example:

  1. git push origin v0.19.0-nowallet
  2. If 0.19/variant-nowallet.patch exists, apply it
  3. Carry on as usual

6.3. Builds triggered by pushes to master MAY only check if variants apply w/o building them


7. Automated builds

This defines how the Automated-builds system should behave.

7.1. All images pushed to Docker Hub MUST be built using an automated system

7.2. Github Actions should be used

7.3. Travis and other popular ones MAY be used

7.4. Deploy should happen on git-tag

7.5. README file must be synced to Docker Hub on either: git-tag push, or pushes to master

7.6. Checksums of produced artifacts should be printed

7.7. Checksums & authenticity of downloaded software MUST be verified

NOTE: For Github Actions https://github.com/meeDamian/sync-readme can be used to simplify the process

Clone this wiki locally