Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build container based on Debian 12 #180

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
9 changes: 4 additions & 5 deletions .github/workflows/build-publish-containers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ jobs:
contents: read
strategy:
matrix:
tag: ['client:centos7', 'build-node:debian11']
platform: [amd64, arm64, ppc64le]
tag: ['client:centos7', 'build-node:debian11', 'build-node:debian12']
platform: [amd64, arm64]

steps:
- name: Check out the repo
Expand Down Expand Up @@ -91,7 +91,7 @@ jobs:
contents: read
strategy:
matrix:
tag: ['client:centos7', 'build-node:debian11']
tag: ['client:centos7', 'build-node:debian11', 'build-node:debian12']
steps:
- name: Login to GitHub Container Registry
if: github.event_name != 'pull_request'
Expand All @@ -110,8 +110,7 @@ jobs:
run: |
docker buildx imagetools create -t ghcr.io/${{ env.REPOSITORY_OWNER }}/${{ matrix.tag }} \
ghcr.io/${{ env.REPOSITORY_OWNER }}/${{ matrix.tag }}-amd64 \
ghcr.io/${{ env.REPOSITORY_OWNER }}/${{ matrix.tag }}-arm64 \
ghcr.io/${{ env.REPOSITORY_OWNER }}/${{ matrix.tag }}-ppc64le
ghcr.io/${{ env.REPOSITORY_OWNER }}/${{ matrix.tag }}-arm64

- name: Inspect the final image
if: github.event_name != 'pull_request'
Expand Down
4 changes: 3 additions & 1 deletion containers/Dockerfile.EESSI-build-node-debian11
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
ARG cvmfsversion=2.11.2
ARG archspecversion=0.2.2
ARG awscliversion=1.32.22
ARG fuseoverlayfsversion=1.10

Expand All @@ -10,6 +11,7 @@ RUN sh /build-or-download-cvmfs-debs.sh ${cvmfsversion}

FROM debian:11.5
ARG cvmfsversion
ARG archspecversion
ARG awscliversion
ARG fuseoverlayfsversion

Expand Down Expand Up @@ -38,7 +40,7 @@ RUN mkdir -p /cvmfs/software.eessi.io
RUN useradd -ms /bin/bash eessi

# stick to awscli v1.x, 2.x is not available through PyPI (see https://github.com/aws/aws-cli/issues/4947)
RUN pip3 install archspec awscli==${awscliversion}
RUN pip3 install archspec==${archspecversion} awscli==${awscliversion}

RUN curl -OL https://raw.githubusercontent.com/EESSI/eessi-bot-software-layer/develop/scripts/eessi-upload-to-staging \
&& mv eessi-upload-to-staging /usr/bin \
Expand Down
48 changes: 48 additions & 0 deletions containers/Dockerfile.EESSI-build-node-debian12
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
ARG cvmfsversion=2.11.2
ARG archspecversion=0.2.2
ARG awscliversion=1.32.22
ARG fuseoverlayfsversion=1.10

FROM debian:12.5 AS prepare-deb
ARG cvmfsversion
COPY ./containers/build-or-download-cvmfs-debs.sh /build-or-download-cvmfs-debs.sh
RUN sh /build-or-download-cvmfs-debs.sh ${cvmfsversion}


FROM debian:12.5
ARG cvmfsversion
ARG archspecversion
ARG awscliversion
ARG fuseoverlayfsversion

COPY --from=prepare-deb /root/deb /root/deb

RUN apt-get update
RUN apt-get install -y sudo vim openssh-client gawk autofs curl attr uuid fuse3 libfuse2 psmisc gdb uuid-dev lsof strace
# python3 and jq are required for eessi-upload-to-staging script (next to awscli)
RUN apt-get install -y python3-pip python3-venv jq
RUN dpkg -i /root/deb/cvmfs_${cvmfsversion}~1+debian12_$(dpkg --print-architecture).deb \
/root/deb/cvmfs-fuse3_${cvmfsversion}~1+debian12_$(dpkg --print-architecture).deb \
/root/deb/cvmfs-libs_${cvmfsversion}~1+debian12_$(dpkg --print-architecture).deb \
/root/deb/cvmfs-config-default_latest_all.deb \
/root/deb/cvmfs-config-eessi_latest_all.deb

# download binary for specific version of fuse-overlayfs
RUN curl -L -o /usr/local/bin/fuse-overlayfs https://github.com/containers/fuse-overlayfs/releases/download/v${fuseoverlayfsversion}/fuse-overlayfs-$(uname -m) \
&& chmod +x /usr/local/bin/fuse-overlayfs

RUN echo 'CVMFS_QUOTA_LIMIT=10000' > /etc/cvmfs/default.local \
&& echo 'CVMFS_CLIENT_PROFILE="single"' >> /etc/cvmfs/default.local \
&& echo 'CVMFS_HIDE_MAGIC_XATTRS=yes' >> /etc/cvmfs/default.local

RUN mkdir -p /cvmfs/software.eessi.io

RUN useradd -ms /bin/bash eessi

RUN python3 -m venv /opt/archspec && . /opt/archspec/bin/activate && pip3 install archspec==${archspecversion} && deactivate && ln -s /opt/archspec/bin/archspec /usr/local/bin/archspec
# stick to awscli v1.x, 2.x is not available through PyPI (see https://github.com/aws/aws-cli/issues/4947)
RUN python3 -m venv /opt/awscli && . /opt/awscli/bin/activate && pip3 install awscli==${awscliversion} && deactivate && ln -s /opt/awscli/bin/aws /usr/local/bin/aws

RUN curl -OL https://raw.githubusercontent.com/EESSI/eessi-bot-software-layer/develop/scripts/eessi-upload-to-staging \
&& mv eessi-upload-to-staging /usr/bin \
&& chmod a+x /usr/bin/eessi-upload-to-staging
8 changes: 5 additions & 3 deletions containers/build-or-download-cvmfs-debs.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#/bin/bash
cvmfsversion=$1
arch=$(dpkg --print-architecture)
os=debian11

apt-get update
apt-get install -y wget
apt-get install -y wget lsb-release

arch=$(dpkg --print-architecture)
os="$(lsb_release -si | tr [:upper:] [:lower:])$(lsb_release -sr)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
os="$(lsb_release -si | tr [:upper:] [:lower:])$(lsb_release -sr)"
os="$(lsb_release -si | tr [:upper:] [:lower:])$(lsb_release -sr | cut -d '.' -f 1)"

Not sure if this is the problem...


if [ "$arch" = "ppc64el" ] || [ "$arch" = "arm64" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it still need to build the deb packages from source for Arm?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are packages for Ubuntu, see https://cernvm.cern.ch/fs/, but I'm not sure if they work on Debian.

then
apt-get install -y devscripts libfuse3-dev cmake cpio libcap-dev libssl-dev libfuse-dev pkg-config libattr1-dev python-dev python-setuptools python3-dev python3-setuptools uuid-dev valgrind libz-dev lsb-release
Expand Down
Loading