[TUF] What new registry API's are needed for TUF support #28
Replies: 4 comments 2 replies
-
|
Related to this is the OCI artifact proposal where I'm hoping to see it flexible enough to allow TUF artifacts at a repository level (much of their focus is scoped to individual manifests). @mnm678 have a look at these and think of what sorts of data needs to be pushed and pulled from the registry to support TUF and what the API's may look like. To handle race conditions, we may also want something on the registry to verify nothing changed between two actions. E.g. signing a new manifest could include pulling the current target data, making changes, and pushing that new target, and you'd want to know another part of the CI pipeline running currently didn't push it's own change to the targets in between. I saw some mention of using ETag headers for this in a recent OCI call, but I think that would require referencing the targets by name (tag) rather than by a query, and I'm not sure we'll see the ability to do that without stepping on the image tags namespace. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for bringing this thread to my attention, this is a great list of features.
My initial impression is that TUF metadata can be stored in the OCI artifact proposal, using the included tag functionality to indicate where 'root', 'snapshot', etc can be found. |
Beta Was this translation helpful? Give feedback.
-
|
I've left "creating a tag pointing to the TUF artifacts" as a last option since it shares the same namespace as image manifest tags. Since we don't have any system reserved names in that namespace, it means we can conflict with users that happen to pick the same name. The tag also only points to one sha, and we may have a potential for multiple artifacts when we factor in multiple signers and importing things from upstream registries. E.g. would we have multiple targets metadata, for both Acme Rockets and Wabbit Networks, if they have both signed the images that were copied from Acme's to Wabbit's registry? Some tools copying these artifacts won't be notary clients, so it would be ideal if notary can automatically merge the notary artifacts that get copied in from external sources. With that, here's the quick version of the history of my thought process: Option A: Originally I was looking at getting tags to be namespaced by media type, which would allow us to query the notary tags separate from the image tags. That got quickly turned down since tags can't have that flexibility according to people with a lot more experience fighting these challenges in large registries. Option B: Next I started to look at Steve's proposal and was hoping we would get an artifact specific API to query by Option C: There's already a proposal to get a manifest list API. And what we want from that is a way to filter by We still have option D to use tags, but I'm thinking of the risk of race conditions where an image gets built on multiple platforms, and each platform manifest gets individually signed by separate CI processes on separate hosts. If that signing process first pulls back the latest target tag, adds the new target entry, and pushes it back to the server, it could overwrite the target tag that was concurrently pushed. The downside of option C is we need to handle cleanup of stale registry data in the notary client, including some checks after sending the updated targets data to see if there was a race that needs to be resolved. We don't want that cleanup to be too aggressive either since clients may have recently queried for a list of target/snapshot artifacts and be pulling those for validating a manifest. A cleanup policy that ensures a newer artifact has been available for at least |
Beta Was this translation helpful? Give feedback.
-
This is a good topic for the Notary v2 key management scenarios. Niaz has discussions queued up for our next Notary v2 meeting Updated Key Management Scenarios #38 |
Beta Was this translation helpful? Give feedback.
-
The snapshot only needs to know about all the targets metadata files on the registry, but it would be nice to be able to query that from the registry.
The delegations are handled within the various TUF metadata files, so as long as those are available, delegations should work as is.
This could either be handled by concurency controls on the registry, or TUF's consistent snapshots
It shouldn't need anything else. It could even be generated by the same process as the snapshot.
One additional feature that would be needed is a way to distribute root public keys. Ideally this would happen over a secure channel. |
Beta Was this translation helpful? Give feedback.
-
I think that will be outside of the registry API's (don't think we can consider that a secure channel without returning to TOFU), but certainly a good topic to talk about with the key management team. I'm personally less concerned about this from the smaller environment use cases. We already inject TLS certs into nodes in the cluster to trust on private corporate networks. And with something like an ephemeral builder, that could be a secret file mounted into the builder container that includes the root cert to trust. It would all be out of band and the org would have a process to update those secrets when needed. For the larger organizations, I'll be interested to see what solutions the key management team has to make that more secure and scalable. |
Beta Was this translation helpful? Give feedback.
-
Moving the TUF logic from an external server to one largely based within a registry means that API's will likely be needed to query, push, and pull data from the registry that don't currently exist. I'm starting with the assumption that we need to sign manifests digests rather than only tags based on PR 41 in requirements that I recently submitted.
If we sign manifests by digest, and need to generate a new snapshot based on what is currently signed in the registry, then I believe we'll need a manifest list API to retrieve the list of manifests to check for signatures.
We may also need API's to push/pull various signing keys (hopefully all public keys). Are there API's needed to manage the delegations in the registry?
If multiple artifacts are signed concurrently by different clients, or the same artifact is signed by multiple signers concurrently (particularly when automation is performing the signing) we may need some API's to handle the concurrency to avoid race conditions. Otherwise the snapshot may be updated to show only one artifact is signed if the second signing action is performed between the query of manifests and the push of the snapshot. One API that may help is to have the registry generate the snapshot to be signed, rather than generating that by the client with the manifest list, and that snapshot is signed by the client. The registry then needs an API to query the most recent signed snapshot, ignoring any older snapshots pushed by clients.
For timestamps, I don't think we'll get a registry server option to implement this in the OCI spec itself. Assuming we don't run the timestamping within the registry, that would need to be an external service that could be run, possibly by the registry operators, and possibly by the user signing their images. Does this service need anything other than a way to query the latest snapshot and push a new timestamp to the registry?
Given that full TUF support may require additional setup that some users may want to skip in their deployments of notary, do we want an API on the registry server to indicate what features the signer is providing? For example, clients could query if timestamp signatures are expected, and a client that normally allows timestamps to be optional would then require a valid timestamp from a repository where that feature is enabled.
Are there other areas that I'm missing?
Beta Was this translation helpful? Give feedback.
All reactions