Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for multi-arch i.e. s390x & pp64cle #62

Closed
wants to merge 12 commits into from
55 changes: 27 additions & 28 deletions .github/workflows/build-and-push-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ env:
IMG_REPO: model-registry
DOCKER_USER: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKER_PWD: ${{ secrets.DOCKERHUB_TOKEN }}
PUSH_IMAGE: true
jobs:
build-image:
runs-on: ubuntu-latest
Expand All @@ -33,36 +32,36 @@ jobs:
# checkout branch
- uses: actions/checkout@v4
# set image version
- name: Set main-branch environment
if: env.BUILD_CONTEXT == 'main'
run: |
commit_sha=${{ github.event.after }}
tag=main-${commit_sha:0:7}
echo "VERSION=${tag}" >> $GITHUB_ENV
- name: Set tag environment
if: env.BUILD_CONTEXT == 'tag'
run: |
echo "VERSION=${{ github.ref_name }}" >> $GITHUB_ENV
- name: Build and Push Image
shell: bash
run: ./scripts/build_deploy.sh
- name: Tag Latest
if: env.BUILD_CONTEXT == 'main'
shell: bash
env:
IMG: ${{ env.IMG_ORG }}/${{ env.IMG_REPO }}
BUILD_IMAGE: false # image is already built in "Build and Push Image" step
echo "IMAGE_NAME=${{env.IMG_ORG}}/${{env.IMG_REPO}}" >> $GITHUB_ENV
- name: Set main-branch environment
if: env.BUILD_CONTEXT == 'main'
run: |
docker tag ${{ env.IMG }}:$VERSION ${{ env.IMG }}:latest
# BUILD_IMAGE=false skip the build, just push the tag made above
VERSION=latest ./scripts/build_deploy.sh
- name: Tag Main
echo "TAG=latest" >> $GITHUB_ENV
echo "IMAGE_NAME=${{env.IMG_ORG}}/${{env.IMG_REPO}}" >> $GITHUB_ENV
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and Push Image
if: env.BUILD_CONTEXT == 'tag'
uses: docker/build-push-action@v3
with:
tags: ${{ env.IMAGE_NAME }}:${{ env.VERSION }}
platforms: linux/amd64,linux/s390x,linux/ppc64le
push: true
- name: Build and Push Latest tag
if: env.BUILD_CONTEXT == 'main'
shell: bash
env:
IMG: ${{ env.IMG_ORG }}/${{ env.IMG_REPO }}
BUILD_IMAGE: false # image is already built in "Build and Push Image" step
run: |
docker tag ${{ env.IMG }}:$VERSION ${{ env.IMG }}:main
# BUILD_IMAGE=false skip the build, just push the tag made above
VERSION=main ./scripts/build_deploy.sh
uses: docker/build-push-action@v3
with:
tags: ${{ env.IMAGE_NAME }}:${{ env.TAG }}
platforms: linux/amd64,linux/s390x,linux/ppc64le
push: true
16 changes: 15 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,20 @@ RUN yum install -y nodejs npm java-11
# Copy the go source
COPY ["Makefile", "main.go", ".openapi-generator-ignore", "openapitools.json", "./"]


# Download protoc compiler v24.3
RUN set -ex\
; ARCH=$(uname -m) ; echo $ARCH \
; if [ "$ARCH" == "s390x" ] ; then ARCH="s390_64" ; elif [ "$ARCH" == "ppc64le" ] ; then ARCH="ppcle_64" ; fi \
; wget -q https://github.com/protocolbuffers/protobuf/releases/download/v24.3/protoc-24.3-linux-$ARCH.zip -O protoc.zip \
; unzip -q protoc.zip \
; bin/protoc --version \
; rm protoc.zip \
;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The download of protoc is already performed as part of make deps which includes make bin/protoc, in line below 34.

This was improved with #52

Kindly consider extending scripts/install_protoc.sh for your platforms, so we don't need this extra Docker RUN ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i didn't know about the protoc change . Once it is approved , i will update my changes accordingly for protoc


# Download tools
RUN make deps
modassarrana89 marked this conversation as resolved.
Show resolved Hide resolved

# Copy rest of the source
COPY .git/ .git/
COPY cmd/ cmd/
Expand All @@ -34,7 +48,7 @@ RUN make deps

# Build
USER root
RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 make clean model-registry
RUN CGO_ENABLED=1 GOOS=linux make clean model-registry

# Use distroless as minimal base image to package the model-registry binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
Expand Down
15 changes: 14 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,10 @@ build/odh: vet
gen: deps gen/grpc gen/openapi gen/openapi-server gen/converter
${GO} generate ./...

# golanci lint takes more time while running under qemu and facing timeout issue "level=error msg="Timeout exceeded: try increasing it by passing --timeout option"
.PHONY: lint
lint:
${GOLANGCI_LINT} run main.go
${GOLANGCI_LINT} run main.go --timeout 2m
${GOLANGCI_LINT} run cmd/... internal/... ./pkg/...

.PHONY: test
Expand Down Expand Up @@ -219,4 +220,16 @@ image/build:
image/push:
${DOCKER} push ${IMG}:$(IMG_VERSION)

# build and push multi-arch docker image
PLATFORMS ?= linux/amd64,linux/s390x,linux/ppc64le
.PHONY: image/buildx
image/buildx: ## Build and push docker image for cross-platform support
# copy existing Dockerfile and insert --platform=${BUILDPLATFORM} into Dockerfile.cross, and preserve the original Dockerfile
sed -e '1 s/\(^FROM\)/FROM --platform=\$$\{BUILDPLATFORM\}/; t' -e ' 1,// s//FROM --platform=\$$\{BUILDPLATFORM\}/' ./Dockerfiles/Dockerfile > ./Dockerfiles/Dockerfile.cross
modassarrana89 marked this conversation as resolved.
Show resolved Hide resolved
- $(DOCKER) buildx create --name project-v3-builder
$(DOCKER) buildx use project-v3-builder
- $(DOCKER) buildx build --push --platform=$(PLATFORMS) --tag ${IMG}:$(IMG_VERSION) -f Dockerfile.cross .
- $(DOCKER) buildx rm project-v3-builder
rm Dockerfile.cross

all: model-registry
Loading