Docker automatic build (#841)

* improved dockerfile * added info on building architecture dockerfile * added workflow for docker * modify run conditions for docker workflow * add push event to run workflow at least once * fix * test docker workflow * fix docker build action * explicit docker tag based on spatialdata-xx versions * fix Dockerfile * fix typo dockerfile * checking for existing docker tags * fix workflow * typo * fix problem image tag check * fix * removed on push docker workflow trigger * update docs on docker and conda * small rewrite
scverse · Jan 22, 2025 · f927685 · f927685
1 parent 69040ae
commit f927685
Show file tree

Hide file tree

Showing 3 changed files with 178 additions and 8 deletions.
diff --git a/.github/workflows/build_image.yml b/.github/workflows/build_image.yml
@@ -0,0 +1,100 @@
+name: Build Docker image
+
+on:
+    workflow_dispatch:
+#    schedule:
+#        -   cron: '0 0 * * *'  # run daily at midnight UTC
+
+env:
+    REGISTRY: ghcr.io
+    IMAGE_NAME: ${{ github.repository }}
+
+concurrency:
+    group: ${{ github.workflow }}-${{ github.ref }}
+    cancel-in-progress: true
+
+jobs:
+    build:
+        runs-on: ubuntu-latest
+
+        defaults:
+            run:
+                shell: bash -e {0} # -e to fail on error
+
+        permissions:
+            contents: read
+            packages: write
+            attestations: write
+            id-token: write
+
+        steps:
+            - name: Checkout code
+              uses: actions/checkout@v4
+
+            - name: Set up Python
+              uses: actions/setup-python@v4
+              with:
+                  python-version: "3.x"
+
+            - name: Upgrade pip
+              run: pip install pip
+
+            - name: Get latest versions
+              id: get_versions
+              run: |
+                  SPATIALDATA_VERSION=$(pip index versions spatialdata | grep "Available versions" | sed 's/Available versions: //' | awk -F', ' '{print $1}')
+                  SPATIALDATA_IO_VERSION=$(pip index versions spatialdata-io | grep "Available versions" | sed 's/Available versions: //' | awk -F', ' '{print $1}')
+                  SPATIALDATA_PLOT_VERSION=$(pip index versions spatialdata-plot | grep "Available versions" | sed 's/Available versions: //' | awk -F', ' '{print $1}')
+                  echo "SPATIALDATA_VERSION=${SPATIALDATA_VERSION}" >> $GITHUB_ENV
+                  echo "SPATIALDATA_IO_VERSION=${SPATIALDATA_IO_VERSION}" >> $GITHUB_ENV
+                  echo "SPATIALDATA_PLOT_VERSION=${SPATIALDATA_PLOT_VERSION}" >> $GITHUB_ENV
+
+            - name: Check if image tag exists
+              id: check_tag
+              env:
+                  IMAGE_TAG_SUFFIX: spatialdata${{ env.SPATIALDATA_VERSION }}_spatialdata-io${{ env.SPATIALDATA_IO_VERSION }}_spatialdata-plot${{ env.SPATIALDATA_PLOT_VERSION }}
+                  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+              run: |
+                  # Define the API URL
+                  API_URL="https://api.github.com/orgs/scverse/packages/container/spatialdata/versions"
+
+                  # Fetch all existing versions
+                  existing_tags=$(curl -s -H "Authorization: token $GITHUB_TOKEN" $API_URL | jq -r '.[].metadata.container.tags[]')
+
+                  # Debug: Output all existing tags
+                  echo "Existing tags:"
+                  echo "$existing_tags"
+
+                  # Check if the constructed tag exists
+                  if echo "$existing_tags" | grep -q "$IMAGE_TAG_SUFFIX"; then
+                    echo "Image tag $IMAGE_TAG_SUFFIX already exists. Skipping build."
+                    echo "skip_build=true" >> $GITHUB_ENV
+                  else
+                    echo "Image tag $IMAGE_TAG_SUFFIX does not exist. Proceeding with build."
+                    echo "skip_build=false" >> $GITHUB_ENV
+                    echo "IMAGE_TAG_SUFFIX=${IMAGE_TAG_SUFFIX}" >> $GITHUB_ENV
+                  fi
+
+            - name: Login to GitHub Container Registry
+              if: ${{ env.skip_build == 'false' }}
+              uses: docker/login-action@v3
+              with:
+                  registry: ${{ env.REGISTRY }}
+                  username: ${{ github.actor }}
+                  password: ${{ secrets.GITHUB_TOKEN }}
+
+            - uses: docker/build-push-action@v5
+              if: ${{ env.skip_build == 'false' }}
+              env:
+                  IMAGE_TAG: ${{ env.REGISTRY }}/scverse/spatialdata:${{ env.IMAGE_TAG_SUFFIX }}
+              with:
+                  context: .
+                  file: ./Dockerfile
+                  push: true
+                  cache-from: type=registry,ref=${{ env.REGISTRY }}/scverse/spatialdata:buildcache
+                  cache-to: type=inline,ref=${{ env.REGISTRY }}/scverse/spatialdata:buildcache
+                  build-args: |
+                      SPATIALDATA_VERSION=${{ env.SPATIALDATA_VERSION }}
+                      SPATIALDATA_IO_VERSION=${{ env.SPATIALDATA_IO_VERSION }}
+                      SPATIALDATA_PLOT_VERSION=${{ env.SPATIALDATA_PLOT_VERSION }}
+                  tags: ${{ env.IMAGE_TAG }}
diff --git a/Dockerfile b/Dockerfile
@@ -1,12 +1,51 @@
-FROM condaforge/mambaforge
+ARG TARGETPLATFORM=linux/amd64
 
-RUN mamba install python=3.10
+# Use the specified platform to pull the correct base image.
+# Override TARGETPLATFORM during build for different architectures, such as linux/arm64 for Apple Silicon.
+# For example, to build for ARM64 architecture (e.g., Apple Silicon),
+# use the following command on the command line:
+#
+#     docker build --build-arg TARGETPLATFORM=linux/arm64 -t my-arm-image .
+#
+# Similarly, to build for the default x86_64 architecture, you can use:
+#
+#     docker build --build-arg TARGETPLATFORM=linux/amd64 -t my-amd64-image .
+#
+FROM --platform=$TARGETPLATFORM ubuntu:latest
+LABEL authors="Luca Marconato"
 
-# complex dependencies that needs to be solved with conda
-RUN mamba install -c conda-forge gcc libgdal gxx imagecodecs -y
+ENV PYTHONUNBUFFERED=1
 
-# satellite spatialdata projects
+ARG SPATIALDATA_VERSION
+ARG SPATIALDATA_IO_VERSION
+ARG SPATIALDATA_PLOT_VERSION
+
+# debugging
+RUN echo "Target Platform: ${TARGETPLATFORM}" && \
+    echo "spatialdata version: ${SPATIALDATA_VERSION}" && \
+    echo "spatialdata-io version: ${SPATIALDATA_IO_VERSION}" && \
+    echo "spatialdata-plot version: ${SPATIALDATA_PLOT_VERSION}"
+
+# Update and install system dependencies.
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+    build-essential \
+    python3-venv \
+    python3-dev \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+# setup python virtual environment
+RUN python3 -m venv /opt/venv
+ENV PATH="/opt/venv/bin:$PATH"
+RUN pip install --upgrade pip wheel
+
+# Install the libraries with specific versions
 RUN pip install --no-cache-dir \
-    spatialdata \
-    spatialdata-io \
-    spatialdata-plot
+    spatialdata[torch]==${SPATIALDATA_VERSION} \
+    spatialdata-io==${SPATIALDATA_IO_VERSION} \
+    spatialdata-plot==${SPATIALDATA_PLOT_VERSION}
+
+LABEL spatialdata_version="${SPATIALDATA_VERSION}" \
+      spatialdata_io_version="${SPATIALDATA_IO_VERSION}" \
+      spatialdata_plot_version="${SPATIALDATA_PLOT_VERSION}"
diff --git a/docs/installation.md b/docs/installation.md
@@ -60,6 +60,37 @@ python -c "import spatialdata; print(spatialdata.__path__)"
 
 if you get a path that contains `site-packages`, then your editable installation has been overridden and you need to reinstall the package by rerunning `pip install -e .` in the cloned `spatialdata` repo.
 
+## Conda
+
+You can install the `spatialdata`, `spatialdata-io`, `spatialdata-plot` and `napari-spatialdata` packages from the `conda-forge` channel using
+
+```bash
+mamba install -c conda-forge spatialdata spatialdata-io spatialdata-plot napari-spatialdata
+```
+
+Note: currently (Jan 2025), due to particular versions being unavailable on `conda-forge` for some dependent packages, the latest versions of the packages of the `SpatialData` ecosystem are not available on `conda-forge`. We are working on fixing this issue; please check the latest versions available on the channel.
+
+## Docker
+
+A `Dockerfile` is available in the repository; the image that can be built from it contains `spatialdata` (with `torch`), `spatialdata-io` and `spatialdata-plot` (not `napari-spatialdata`).
+
+To build the image, run:
+
+```bash
+# this is for Apple Silicon machines, if you are not using such machine you can omit the --build-arg
+docker build --build-arg TARGETPLATFORM=linux/arm64 --tag spatialdata .
+docker run -it spatialdata
+```
+
+We also publish images automatically via GitHub Actions; you can see the [list of available images here](https://github.com/scverse/spatialdata/pkgs/container/spatialdata/versions).
+
+Once you have the image name, you can pull and run it with:
+
+```bash
+docker pull ghcr.io/scverse/spatialdata:spatialdata0.3.0_spatialdata-io0.1.7_spatialdata-plot0.2.9
+docker run -it ghcr.io/scverse/spatialdata:spatialdata0.3.0_spatialdata-io0.1.7_spatialdata-plot0.2.9
+```
+
 <!-- Links -->
 
 [napari-spatialdata]: https://github.com/scverse/napari-spatialdata