Support arm64 in cpubuilder dockerfile using multi-arch builds. (#11)

This should let us replace https://github.com/iree-org/iree/blob/main/build_tools/docker/dockerfiles/base-arm64.Dockerfile with this cpubuilder dockerfile, making progress on iree-org/iree#15332. It uses a multi-architecture build rather than a fully forked file, which seems to work reasonably well with some local testing. I can even run the arm64 dockerfile on my x86_64 host. Various related changes are included here: * Drop `x86_64` from file names * `publish_cpubuilder_x86_64.yml` --> `publish_cpubuilder.yml` * `cpubuilder_ubuntu_jammy_x86_64.Dockerfile` --> `cpubuilder_ubuntu_jammy.Dockerfile` * `cpubuilder_ubuntu_jammy_ghr_x86_64.Dockerfile` --> `cpubuilder_ubuntu_jammy_ghr.Dockerfile` * Build with `--platform linux/amd64,linux/arm64` and update docs for this * Build ccache from source as needed in `build_tools/install_ccache.sh` (code lifted from https://github.com/iree-org/iree/blob/main/build_tools/docker/context/install_ccache.sh). Note that if we standardize on sccache we can drop the ccache install entirely * Since this builds from source using CMake, I moved the ccache install step to after the cmake install step * I am _not_ installing qemu here yet, as we did in IREE (https://github.com/iree-org/iree/blob/782f372b070eadd593a727004cf61dc84aabc634/build_tools/docker/dockerfiles/base-arm64.Dockerfile#L75-L80). I want to try installing on demand with https://github.com/docker/setup-qemu-action first.
iree-org · Sep 19, 2024 · 38a7a45 · 38a7a45
1 parent 14d8f4a
commit 38a7a45
Show file tree

Hide file tree

Showing 5 changed files with 85 additions and 35 deletions.
diff --git a/...b/workflows/publish_cpubuilder_x86_64.yml → .github/workflows/publish_cpubuilder.yml b/...b/workflows/publish_cpubuilder_x86_64.yml → .github/workflows/publish_cpubuilder.yml
@@ -1,18 +1,18 @@
-name: Publish cpubuilder x86_64 images
+name: Publish cpubuilder images
 on:
   workflow_dispatch:
   push:
     branches: ['main']
     paths:
-    - dockerfiles/cpubuilder*_x86_64.Dockerfile
-    - .github/workflows/publish_cpubuilder_x86_64.yml
+    - dockerfiles/cpubuilder*.Dockerfile
+    - .github/workflows/publish_cpubuilder.yml
 
 jobs:
   build-cpubuilder-ubuntu-jammy:
     runs-on: ubuntu-latest
     env:
       REGISTRY: ghcr.io
-      IMAGE_NAME: iree-org/cpubuilder_ubuntu_jammy_x86_64
+      IMAGE_NAME: iree-org/cpubuilder_ubuntu_jammy
     # Sets the permissions granted to the `GITHUB_TOKEN` for the actions in this job.
     permissions:
       contents: read
@@ -38,7 +38,8 @@ jobs:
         uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
         with:
           context: .
-          file: dockerfiles/cpubuilder_ubuntu_jammy_x86_64.Dockerfile
+          file: dockerfiles/cpubuilder_ubuntu_jammy.Dockerfile
+          platforms: linux/amd64,linux/arm64
           push: true
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
@@ -47,7 +48,7 @@ jobs:
     runs-on: ubuntu-latest
     env:
       REGISTRY: ghcr.io
-      IMAGE_NAME: iree-org/cpubuilder_ubuntu_jammy_ghr_x86_64
+      IMAGE_NAME: iree-org/cpubuilder_ubuntu_jammy_ghr
     # Sets the permissions granted to the `GITHUB_TOKEN` for the actions in this job.
     permissions:
       contents: read
@@ -73,7 +74,8 @@ jobs:
         uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
         with:
           context: .
-          file: dockerfiles/cpubuilder_ubuntu_jammy_ghr_x86_64.Dockerfile
+          file: dockerfiles/cpubuilder_ubuntu_jammy_ghr.Dockerfile
+          platforms: linux/amd64,linux/arm64
           push: true
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
diff --git a/README.md b/README.md
@@ -10,8 +10,8 @@ Image name | Description | Source Dockerfile
 ---------- | ----------- | -----------------
 `iree-org/amdgpu_ubuntu_jammy_x86_64` | Ubuntu with AMDGPU deps | [Source](./dockerfiles/amdgpu_ubuntu_jammy_x86_64.Dockerfile)
 `iree-org/amdgpu_ubuntu_jammy_ghr_x86_64` | Ubuntu with AMDGPU deps (GitHub runner) | [Source](./dockerfiles/amdgpu_ubuntu_jammy_ghr_x86_64.Dockerfile)
-`iree-org/cpubuilder_ubuntu_jammy_x86_64` | CPU builder with IREE build deps | [Source](./dockerfiles/cpubuilder_ubuntu_jammy_x86_64.Dockerfile)
-`iree-org/cpubuilder_ubuntu_jammy_ghr_x86_64` | CPU builder with IREE build deps (GitHub runner) | [Source](./dockerfiles/cpubuilder_ubuntu_jammy_ghr_x86_64.Dockerfile)
+`iree-org/cpubuilder_ubuntu_jammy` | CPU builder with IREE build deps | [Source](./dockerfiles/cpubuilder_ubuntu_jammy.Dockerfile)
+`iree-org/cpubuilder_ubuntu_jammy_ghr` | CPU builder with IREE build deps (GitHub runner) | [Source](./dockerfiles/cpubuilder_ubuntu_jammy_ghr.Dockerfile)
 `iree-org/manylinux_x86_64` | Portable Linux release builder for Python packaging | [Source](./dockerfiles/manylinux_x86_64.Dockerfile)
 `iree-org/manylinux_ghr_x86_64` | Portable Linux release builder for Python packaging (GitHub runner) | [Source](./dockerfiles/manylinux_ghr_x86_64.Dockerfile)
 
@@ -49,6 +49,39 @@ You can also
 to avoid needing to copy the SHA each time:
 
 ```bash
-sudo docker buildx build --file dockerfiles/cpubuilder_ubuntu_jammy_x86_64.Dockerfile . --tag cpubuilder:latest
+sudo docker buildx build --file dockerfiles/cpubuilder_ubuntu_jammy.Dockerfile . --tag cpubuilder:latest
 sudo docker run --rm --mount type=bind,source="$(realpath ~/iree)",target=/iree -it --entrypoint bash cpubuilder:latest
 ```
+
+## Multi-arch and multi-platform builds
+
+We publish images for multiple architectures, e.g. amd64 and arm64. See
+the documentation for multi-arch and multi-platform builds here:
+
+* https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/
+* https://docs.docker.com/build/building/multi-platform/
+* https://docs.docker.com/build/ci/github-actions/multi-platform/
+
+To build and use a multi-architecture image, pass `--platform`:
+
+```bash
+sudo docker buildx build --file dockerfiles/cpubuilder_ubuntu_jammy.Dockerfile --platform=linux/arm64,linux/amd64 . --tag cpubuilder:latest
+```
+
+* To run a multiplatform image on your host architecture, no changes are needed:
+
+    ```bash
+    sudo docker run --rm -it --entrypoint bash cpubuilder:latest
+
+    root:/$ uname -m
+    x86_64
+    ```
+
+* To run a multiplatform image on a different architecture, pass `--platform`:
+
+    ```bash
+    sudo docker run --rm -it --entrypoint bash --platform=linux/arm64 cpubuilder:latest
+
+    root:/$ uname -m
+    aarch64
+    ```
diff --git a/build_tools/install_ccache.sh b/build_tools/install_ccache.sh
@@ -11,9 +11,18 @@ CCACHE_VERSION="$1"
 
 ARCH="$(uname -m)"
 
-curl --silent --fail --show-error --location \
-    "https://github.com/ccache/ccache/releases/download/v${CCACHE_VERSION}/ccache-${CCACHE_VERSION}-linux-${ARCH}.tar.xz" \
-    --output ccache.tar.xz
+if [[ "${ARCH}" == "x86_64" ]]; then
+  curl --silent --fail --show-error --location \
+      "https://github.com/ccache/ccache/releases/download/v${CCACHE_VERSION}/ccache-${CCACHE_VERSION}-linux-${ARCH}.tar.xz" \
+      --output ccache.tar.xz
 
-tar xf ccache.tar.xz
-cp ccache-${CCACHE_VERSION}-linux-${ARCH}/ccache /usr/local/bin
+  tar xf ccache.tar.xz
+  cp ccache-${CCACHE_VERSION}-linux-${ARCH}/ccache /usr/local/bin
+elif [[ "${ARCH}" == "aarch64" ]]; then
+  # Latest version of ccache is not released for arm64, built it
+  git clone --depth 1 --branch "v${CCACHE_VERSION}" https://github.com/ccache/ccache.git
+  mkdir -p ccache/build && cd "$_"
+  cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release ..
+  ninja
+  cp ccache /usr/bin/
+fi
diff --git a/...cpubuilder_ubuntu_jammy_x86_64.Dockerfile → ...rfiles/cpubuilder_ubuntu_jammy.Dockerfile b/...cpubuilder_ubuntu_jammy_x86_64.Dockerfile → ...rfiles/cpubuilder_ubuntu_jammy.Dockerfile
@@ -1,6 +1,9 @@
 # Stock Ubuntu Jammy (22.04) with IREE build dependencies.
 FROM ubuntu:jammy
 
+ARG TARGETARCH
+ARG TARGETPLATFORM
+
 ######## Apt packages ########
 RUN apt update && \
     apt install -y wget git unzip curl gnupg2 lsb-release
@@ -26,16 +29,6 @@ RUN apt update && \
 # Cleanup.
 RUN apt clean && rm -rf /var/lib/apt/lists/*
 
-######## CCache ########
-WORKDIR /install-ccache
-COPY build_tools/install_ccache.sh ./
-RUN ./install_ccache.sh "4.10.2" && rm -rf /install-ccache
-
-######## sccache ########
-WORKDIR /install-sccache
-COPY build_tools/install_sccache.sh ./
-RUN ./install_sccache.sh "0.8.1" && rm -rf /install-sccache
-
 ######## CMake ########
 WORKDIR /install-cmake
 ENV CMAKE_VERSION="3.23.2"
@@ -52,4 +45,14 @@ RUN ln -s /usr/bin/lld-14 /usr/bin/lld && \
 ENV CC=clang
 ENV CXX=clang++
 
+######## CCache ########
+WORKDIR /install-ccache
+COPY build_tools/install_ccache.sh ./
+RUN ./install_ccache.sh "4.10.2" && rm -rf /install-ccache
+
+######## sccache ########
+WORKDIR /install-sccache
+COPY build_tools/install_sccache.sh ./
+RUN ./install_sccache.sh "0.8.1" && rm -rf /install-sccache
+
 WORKDIR /
diff --git a/...uilder_ubuntu_jammy_ghr_x86_64.Dockerfile → ...es/cpubuilder_ubuntu_jammy_ghr.Dockerfile b/...uilder_ubuntu_jammy_ghr_x86_64.Dockerfile → ...es/cpubuilder_ubuntu_jammy_ghr.Dockerfile
@@ -1,6 +1,9 @@
 # GitHub Actions Runner with IREE build deps.
 FROM docker.io/myoung34/github-runner:ubuntu-jammy
 
+ARG TARGETARCH
+ARG TARGETPLATFORM
+
 ######## Apt packages ########
 RUN apt update && \
     apt install -y wget git unzip curl gnupg2 lsb-release
@@ -26,16 +29,6 @@ RUN apt update && \
 # Cleanup.
 RUN apt clean && rm -rf /var/lib/apt/lists/*
 
-######## CCache ########
-WORKDIR /install-ccache
-COPY build_tools/install_ccache.sh ./
-RUN ./install_ccache.sh "4.10.2" && rm -rf /install-ccache
-
-######## sccache ########
-WORKDIR /install-sccache
-COPY build_tools/install_sccache.sh ./
-RUN ./install_sccache.sh "0.8.1" && rm -rf /install-sccache
-
 ######## CMake ########
 WORKDIR /install-cmake
 ENV CMAKE_VERSION="3.23.2"
@@ -52,5 +45,15 @@ RUN ln -s /usr/bin/lld-14 /usr/bin/lld && \
 ENV CC=clang
 ENV CXX=clang++
 
+######## CCache ########
+WORKDIR /install-ccache
+COPY build_tools/install_ccache.sh ./
+RUN ./install_ccache.sh "4.10.2" && rm -rf /install-ccache
+
+######## sccache ########
+WORKDIR /install-sccache
+COPY build_tools/install_sccache.sh ./
+RUN ./install_sccache.sh "0.8.1" && rm -rf /install-sccache
+
 # Switch back to the working directory upstream expects.
 WORKDIR /actions-runner