-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(DockerImage): allow mounting existing or volume copy volumes #32832
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.
A comment requesting an exemption should contain the text Exemption Request
. Additionally, if clarification is needed add Clarification Request
to a comment.
Exemption Request (Integration Test): Unsure what sort of integration test would be beneficial here, as it's a synth time change. Clarification Request (Readme): Happy to add a writeup, but the current |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
The pull request linter fails with the following errors:
PRs must pass status checks before we can provide a meaningful review. If you would like to request an exemption from the status checks or clarification on feedback, please leave a comment on this PR containing ✅ A exemption request has been requested. Please wait for a maintainer's review. |
Issue # (if applicable)
Closes #32805
Closes #20601
Reason for this change
This change is to better support image bundling in atypical (e.g. CI, DinD) environments by allowing volumes to be mounted using the more portable "Volume Copy" method, or to mount an existing volume. This supports use cases such as caching dependencies in CI environments to reduce asset bundling times.
Description of changes
The primary change made was the addition of two new
DockerVolume
varients (VolumeCopyDockerVolume
andExistingDockerVolume
), facilitated by the newDockerVolumeHelper
private class. This replaces the old asset-staging private classesAssetBundlingBindMount
andAssetBundlingVolumeCopy
, reproducing all the behaviours in a more flexible manner.The decision to move these utility functions to
private/bundling.ts
was to better encapsulate the docker related behaviours and abstract them away from the asset management.This currently uses a discriminated union type to clearly show which properties are required for each volume type. This passes linting, but if it's an issue it can be implemented just as an extension to the DockerVolume interface. However, I'm unsure how to best provide a clear developer experience with this approach.
How It Works
The interfaces for
DockerRunOptions.volumes
and the relatedBundlingOptions.volumes
have been changed to acceptVolumeCopyDockerVolume
andExistingDockerVolume
as well as the existingDockerVolume
type. TheBundlingDockerImage.run()
function has been modified slightly to instantiate aDockerVolumeHelper
, which accepts the docker run options and uses them to prepare the volumes passed in, creating the volume command strings thatrun
then uses.If any if the passed volumes are VOLUME_COPY volumes, a copyContainer is spun up with each VOLUME_COPY volume atttached anonymously. The input copy is then performed using the same approach from
AssetBundlingVolumeCopy
. As these volumes are mounted to the primary container via--volumes-from
, them being anonymous doesnt matter and makes creation and deletion of the helper more atomic, reducing the possibility of orphaned volumes or containers. Error handling has been added in multiple places to further prevent this happening in the case of a build failure.BundlingDockerImage.run()
makes use of the volume commands fromDockerVolumeHelper
to complete its image run, then runsDockerVolumeHelper.cleanup()
to copy any outputs from the volume copy volumes, and delete the container (using the-v
option to remove attach anonymous volumes).Describe any new or updated permissions being added
N/A - All changes are at bundling time only
Description of how you validated changes
Unit tests were added to cover the new expected volume behaviours, and existing tests updated to match any changes to argument order. Tests were made to cover the new helper class and new error handling cleanup behaviour. No integration tests were made or modified as this is a local/bundling change,
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license