Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTI dockerfile support for multi-architecture builds #464

Merged
merged 5 commits into from
Aug 2, 2024

Conversation

elgeeko1
Copy link
Collaborator

@elgeeko1 elgeeko1 commented Jul 4, 2024

Add multiarchitecture support and use build stages to reduce image size.

  1. Add platform-specific base images to allow multiarchitecture support
  2. Use build stage to reduce image size by 20x.
  3. Add maintainer documentation to Dockerfile.

docker build . --file core/federated/RTI/rti.Dockerfile works as it did before.

To build for multiple architectures:

docker build . --file core/federated/RTI/rti.Dockerfile --platform=linux/amd64,linux/aarch64,linux/arm/7,linux/riscv64

Summary by CodeRabbit

  • Chores
    • Updated Dockerfile to support multi-platform builds and optimized the build process.
    • Enhanced GitHub Actions workflow with a new job for multi-architecture Docker builds, maintaining native build functionality.
    • Renamed existing GitHub Actions job for clarity and updated action versions for improved performance.

@elgeeko1 elgeeko1 requested a review from lhstrh July 4, 2024 02:36
@elgeeko1 elgeeko1 self-assigned this Jul 4, 2024
Copy link
Contributor

coderabbitai bot commented Jul 4, 2024

Warning

Rate limit exceeded

@lhstrh has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 32 seconds before requesting another review.

How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

Commits

Files that changed from the base of the PR and between 658c28a and 6816993.

Walkthrough

Recent updates enhance the RTI's Dockerfile and GitHub Actions workflow, introducing multi-platform support and optimizing build efficiency. The modifications refine CI/CD job configurations, adding a new job for Docker builds while maintaining the existing native build functionality. This restructuring improves flexibility and performance across various deployment environments.

Changes

File/Path Change Summary
.github/workflows/build-rti.yml Added docker-build job for Docker builds, renamed run to native-build, and upgraded checkout action version.
core/federated/RTI/rti.Dockerfile Transitioned to multi-stage builds for different architectures, optimized build steps, added dependencies, and refined the entry point.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 38294a3 and ea06ce0.

Files selected for processing (1)
  • core/federated/RTI/rti.Dockerfile (1 hunks)
Additional comments not posted (3)
core/federated/RTI/rti.Dockerfile (3)

10-22: LGTM! Verify the build process.

The builder stage correctly sets up the build environment, installs necessary dependencies, and builds the application.

Ensure that the build process completes successfully for each platform.


1-8: LGTM! Verify base image compatibility.

The initial setup correctly defines a base image and derives from architecture-specific base images to support multiarchitecture builds.

Ensure that the specified base images are compatible with the respective platforms.

Verification successful

Verified: Base image compatibility confirmed

The initial setup correctly defines a base image and derives from architecture-specific base images to support multiarchitecture builds. The specified base images are compatible with the respective platforms.

  • alpine:latest exists and supports amd64.
  • arm64v8/alpine:latest exists and supports arm64.
  • arm32v7/alpine:latest exists and supports arm.
  • riscv64/alpine:latest exists and supports riscv64.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify compatibility of the specified base images with the respective platforms.

# Test: Pull the base images for each platform. Expect: Successful pull for each base image.
docker pull alpine:latest
docker pull arm64v8/alpine:latest
docker pull arm32v7/alpine:latest
docker pull riscv64/alpine:latest

Length of output: 431


Script:

#!/bin/bash
# Description: Verify the existence of the specified base images in the Docker Hub registry.

# Function to check if an image exists on Docker Hub
check_image() {
  local image=$1
  local tag=${2:-latest}
  local url="https://hub.docker.com/v2/repositories/${image}/tags/${tag}"
  
  # Using curl to check the image existence
  response=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X GET "${url}")
  http_status=$(echo $response | tr -d '\n' | sed -e 's/.*HTTPSTATUS://')

  if [ "$http_status" -eq 200 ]; then
    echo "Image ${image}:${tag} exists."
  else
    echo "Image ${image}:${tag} does not exist."
  fi
}

# Check the base images
check_image "library/alpine" "latest"
check_image "arm64v8/alpine" "latest"
check_image "arm32v7/alpine" "latest"
check_image "riscv64/alpine" "latest"

Length of output: 12795


24-32: LGTM! Verify the runtime environment.

The application stage correctly sets up the runtime environment and defines the entry point.

Ensure that the runtime environment is correctly set up and the application starts as expected for each platform.

@lhstrh lhstrh changed the title RTI dockerfile support multiarchitecture builds. RTI dockerfile support for multi-architecture builds Jul 4, 2024
@lhstrh lhstrh added the enhancement Enhancement of existing feature label Jul 4, 2024
Copy link
Member

@lhstrh lhstrh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! The problem is that we're not currently testing Docker builds, only native builds (see https://github.com/lf-lang/reactor-c/blob/main/.github/workflows/build-rti.yml). We should add a simple test to make sure that building using Docker succeeds. Could you take care of this?

As a side note, we need to regularly push updates to DockerHub, which is currently done manually, and way too infrequently. We should automate this, but I'm unsure how to considering that neither the RTI nor reactor-c are versioned.

@elgeeko1
Copy link
Collaborator Author

I'm not sure I'll have time to create a test for Docker, but this is something we could consider getting some help with. Since it's not part of the development workflow and only used by a small group, could we get by without adding a test?

In a sense RTI is versioned, when building xronos-inc/lf-dev-docker, the Dockerfile builds the RTI submodule that is associated with the LF release.

@lhstrh
Copy link
Member

lhstrh commented Jul 11, 2024

I could add a simple test that ensures the docker build works correctly.

@lhstrh
Copy link
Member

lhstrh commented Jul 12, 2024

I'm actually having trouble building manually:

[marten@gen6 reactor-c]$ docker build . --file core/federated/RTI/rti.Dockerfile
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  65.45MB
Step 1/21 : ARG BASEIMAGE=alpine:latest
Step 2/21 : FROM --platform=linux/amd64 ${BASEIMAGE} AS base-amd64
latest: Pulling from library/alpine
Digest: sha256:b89d9c93e9ed3597455c90a0b88a8bbb5cb7188438f70953fede212a0c4394e0
Status: Downloaded newer image for alpine:latest
 ---> a606584aa9aa
Step 3/21 : FROM --platform=linux/arm64 ${BASEIMAGE} AS base-arm64
latest: Pulling from library/alpine
Digest: sha256:b89d9c93e9ed3597455c90a0b88a8bbb5cb7188438f70953fede212a0c4394e0
Status: Downloaded newer image for alpine:latest
 ---> 092561eea88f
Step 4/21 : FROM --platform=linux/arm/v7 ${BASEIMAGE} AS base-arm
latest: Pulling from library/alpine
Digest: sha256:b89d9c93e9ed3597455c90a0b88a8bbb5cb7188438f70953fede212a0c4394e0
Status: Downloaded newer image for alpine:latest
 ---> 7ba4509d8b7b
Step 5/21 : FROM --platform=linux/riscv64 riscv64/${BASEIMAGE} AS base-riscv64
 ---> 04f98b6119b4
Step 6/21 : FROM base-${TARGETARCH} as builder
invalid reference format

and

[marten@gen6 reactor-c]$ docker build . --file core/federated/RTI/rti.Dockerfile --platform=linux/amd64,linux/aarch64,linux/arm/7,linux/riscv64
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  65.45MB
Error response from daemon: "amd64,linux" is an invalid component of "linux/amd64,linux/aarch64,linux/arm/7,linux/riscv64": platform specifier component must match "^[A-Za-z0-9_-]+$": invalid argument

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between ea06ce0 and 7a5aaeb.

Files selected for processing (1)
  • .github/workflows/build-rti.yml (1 hunks)
Additional comments not posted (4)
.github/workflows/build-rti.yml (4)

7-7: LGTM! Approve the job rename and matrix strategy.

The job rename from run to native-build and the addition of the matrix strategy to run on multiple platforms look good.


21-21: LGTM! Approve the addition of the docker-build job.

The addition of the docker-build job to handle multi-architecture Docker builds looks good.


25-25: Verify compatibility of actions/checkout@v4.

Ensure that the actions/checkout@v4 is compatible with the rest of the workflow and does not introduce any breaking changes.


15-15: Verify compatibility of actions/checkout@v4.

Ensure that the actions/checkout@v4 is compatible with the rest of the workflow and does not introduce any breaking changes.

.github/workflows/build-rti.yml Outdated Show resolved Hide resolved
@elgeeko1
Copy link
Collaborator Author

Testing for multiarchitecture builds would be more involved.

Running docker build . should continue to work just fine. If you want to build for multiple platforms, you will need to configure docker accordingly. The same goes for any CI/CD pipeline that tests multiple platforms. See https://docs.docker.com/build/building/multi-platform/

@lhstrh
Copy link
Member

lhstrh commented Jul 16, 2024

Testing for multiarchitecture builds would be more involved.

Running docker build . should continue to work just fine. If you want to build for multiple platforms, you will need to configure docker accordingly. The same goes for any CI/CD pipeline that tests multiple platforms. See https://docs.docker.com/build/building/multi-platform/

But docker build . --file core/federated/RTI/rti.Dockerfile error'ed for me (see above).

@elgeeko1
Copy link
Collaborator Author

@lhstrh it looks like you're working with an older build of Docker? See the output

DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Your docker installation shows you're using a deprecated builder that predates BuildKit (and the notation for build stages and multiarchitecure builds).

@lhstrh
Copy link
Member

lhstrh commented Aug 1, 2024

@lhstrh lhstrh force-pushed the rti-docker-multiarch branch 2 times, most recently from 294164b to bffdaf9 Compare August 2, 2024 04:13
@lhstrh lhstrh self-requested a review August 2, 2024 04:16
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 7a5aaeb and bffdaf9.

Files selected for processing (1)
  • .github/workflows/build-rti.yml (1 hunks)
Additional comments not posted (1)
.github/workflows/build-rti.yml (1)

7-20: LGTM!

The native-build job correctly implements a matrix strategy for different platforms and includes steps to check out the repository and build the RTI with both AUTH options.

.github/workflows/build-rti.yml Outdated Show resolved Hide resolved
Copy link
Member

@lhstrh lhstrh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Also happy that this brought down the size of the image considerably. I added a smoke test to make sure the images are built successfully.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between bffdaf9 and 658c28a.

Files selected for processing (2)
  • .github/workflows/build-rti.yml (1 hunks)
  • core/federated/RTI/rti.Dockerfile (1 hunks)
Additional comments not posted (4)
.github/workflows/build-rti.yml (2)

7-20: LGTM!

The native-build job is well-structured and maintains the original functionality. The upgrade of the checkout action to version v4 is appropriate.


21-35: LGTM!

The docker-build job is well-structured and follows best practices for multi-architecture Docker builds. The use of Docker Buildx is appropriate.

core/federated/RTI/rti.Dockerfile (2)

1-21: LGTM!

The use of ARG BASEIMAGE and multiple base images for different architectures is appropriate. The builder stage is well-structured and follows best practices for multi-stage builds.


22-32: LGTM!

The app stage is well-structured and follows best practices for multi-stage builds. The overall structure of the Dockerfile is clear and efficient.

@lhstrh lhstrh added this pull request to the merge queue Aug 2, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 2, 2024
@lhstrh lhstrh merged commit 4026514 into main Aug 2, 2024
30 checks passed
@lhstrh lhstrh deleted the rti-docker-multiarch branch August 2, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement of existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants