Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Extend Referrer API to image layer blob #517

Open
njucjc opened this issue Feb 22, 2024 · 1 comment
Open

Proposal: Extend Referrer API to image layer blob #517

njucjc opened this issue Feb 22, 2024 · 1 comment

Comments

@njucjc
Copy link

njucjc commented Feb 22, 2024

According to the current definition of the Referrers API, it is limited to specifying one Manifest as the Referrers of another Manifest, but it does not apply to a specific OCI image layer. In some scenarios, it makes sense for one image layer to serve as an referrer pointing to another image layer.

For example, we may be able to use this feature in accelerated container image TurboOCI or SOCI scenarios to avoid repeated conversion of image layer index, thereby speeding up the generation time of accelerated container image layer indexes.

OCI Image                   Accelerated Image Index
  |                              |
  |--- Layer 1                   |--- Layer Index 1
  |                              |
  |--- Layer 2                   |--- Layer Index 2
  |                              |
  |--- Layer 3                   |--- Layer Index 3

When the image layer in the same registry is reused, the accelerated image layer index does not need to be re-produced. Specifically, our enhancement for the registry is as follows:

POST   /v2/<name>/blobs/uploads/?subject=<original image layer digest>
PUT    /v2/<name>/blobs/uploads/<reference>?subject=<original image layer digest>
PATCH  /v2/<name>/blobs/uploads/<reference>?digest=<accerated image index digest>?subject=<original image layer digest>

When uploading the blob for the accelerated container image layer index (e.g., Layer Index 1 digest) of the original image layer, include the digest of the original image layer it indexes (e.g., Layer 1 digest) in the additional subject field. Then, store the reference relationship between them in the image registry for use by the convertor.

When in use, obtain their reference relationship through the newly added blob Referrers API.

HEAD /v2/<name>/blobs/referrer?<original image layer digest>

With the aforementioned capabilities, when working on accelerating image index conversion, we can start by querying the blob referrers API of the image registry to determine if the remote repository contains the corresponding accelerated image layer index. If it does exist, we can skip the conversion for that layer. This is particularly valuable for large image layers, where the acceleration of index conversion can be time-consuming, so I think adding this capability is meaningful.

@sudo-bmitch
Copy link
Contributor

There was discussion on this in opencontainers/image-spec#1012

If the scenarios have layers already converted because of a shared base image, then it seems like reusing and extending the referrers of the base image may help in this scenario without any changes needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants