Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating build script to use Google Artifact Registry (SCP-5153) #2187

Merged
merged 1 commit into from
Jan 14, 2025

Conversation

bistline
Copy link
Contributor

@bistline bistline commented Jan 8, 2025

BACKGROUND & CHANGES

This updates the bin/build_image.sh script that runs as a part of the Build and publish single-cell-portal Docker image Github Action to use Google Artifact Registry instead of Container Registry. GCR is being turned off at the end of March. None of our images have been renamed and still use the original GCR names. The only code that changes is how we interact with those images/tags via the gcloud CLI.

MANUAL TESTING

  1. Run gcloud components update to ensure your version of gcloud is up to date
  2. Ensure Docker is running
  3. Go to your project directory and run the following:
VERSION_TAG="`whoami`-gar-update"
bin/build_image.sh -v $VERSION_TAG
  1. You should see an output message like the below, followed by Docker build output:
*** BUILDING IMAGE REF gcr.io/broad-singlecellportal-staging/single-cell-portal:bistline-gar-update ***
  1. Confirm the build & push succeed, and that you see your image listed here: https://console.cloud.google.com/artifacts/docker/broad-singlecellportal-staging/us/gcr.io/single-cell-portal?project=broad-singlecellportal-staging
  2. Click the ellipsis menu next to your image and select "Edit tags", then remove the tag you just created
  3. Run the following commands to remove the image you just built using the same code as the script:
GAR_NAME=us-docker.pkg.dev/broad-singlecellportal-staging/gcr.io/single-cell-portal
UNTAGGED=$(gcloud artifacts docker images list $GAR_NAME --include-tags --filter="-tags:*" --format="get(version)")
echo $UNTAGGED
=> sha256:814fc92bf9725f6b895d223766e6813dd316db7caf72e0019abc21fbe6bba579 # yours will be different

gcloud artifacts docker images delete $GAR_NAME@$UNTAGGED --quiet
=> Digests:
- us-docker.pkg.dev/broad-singlecellportal-staging/gcr.io/single-cell-portal@sha256:814fc92bf9725f6b895d223766e6813dd316db7caf72e0019abc21fbe6bba579
Delete request issued.
Waiting for operation [projects/broad-singlecellportal-staging/locations/us/operations/20604863-671c-4768-985a-9fa6525f352a] to complete...done.    
  1. Reload the page and confirm your digest was deleted

@bistline bistline requested review from eweitz and jlchang January 8, 2025 16:27
Copy link
Member

@eweitz eweitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good! Thanks for the helpful context at standup, and the clear write-up.

Copy link

codecov bot commented Jan 8, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.17%. Comparing base (b6847a4) to head (e5e7ae6).
Report is 2 commits behind head on development.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##           development    #2187   +/-   ##
============================================
  Coverage        70.16%   70.17%           
============================================
  Files              332      332           
  Lines            28197    28197           
  Branches          2422     2422           
============================================
+ Hits             19785    19787    +2     
+ Misses            8265     8263    -2     
  Partials           147      147           

see 2 files with indirect coverage changes

@bistline bistline added the build failure: false positive Build error confirmed as false positive. E.g. upstream service has a problem. label Jan 8, 2025
Copy link
Contributor

@jlchang jlchang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functional review - GAR build and manual deletion of untagged artifact worked as described.

CI failure on "FireCloudClientTest#test_get_methods" has been seen before but previously yielded 500. Now that it emits a more specific error code (406), is it more feasible to not fail CI for this particular test mode when we see 406? I understand it may not be worth the effort given we've only seen this particular error a handful of times.

@bistline
Copy link
Contributor Author

Functional review - GAR build and manual deletion of untagged artifact worked as described.

CI failure on "FireCloudClientTest#test_get_methods" has been seen before but previously yielded 500. Now that it emits a more specific error code (406), is it more feasible to not fail CI for this particular test mode when we see 406? I understand it may not be worth the effort given we've only seen this particular error a handful of times.

The last two smoke tests passed, so this appears to just be a blip. That being said since we don't really do anything with Terra workflows anymore, we could consider removing these methods/tests.

@bistline bistline merged commit 3b983f8 into development Jan 14, 2025
4 of 5 checks passed
@github-actions github-actions bot deleted the jb-artifact-registry-switch branch January 14, 2025 14:31
@eweitz eweitz mentioned this pull request Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build failure: false positive Build error confirmed as false positive. E.g. upstream service has a problem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants