Check that the images we deploy are those we built

[skitt]

To avoid testing old images, instead of those we’re supposed to build, we should:

• store the hashes of the images when we build them (e.g. in the existing package/.image... trackers)
• once our pods are up and running, check that the container hashes match the expected hashes

This can be made generic: we only need to check the package contents.

[stale]

This issue has been automatically marked as stale because it has not had activity for 60 days. It will be closed if no further activity occurs. Please make a comment if this issue/pr is still valid. Thank you for your contributions.

[skitt]

This is still relevant.

[skitt]

buildx --iidfile doesn’t report the right image id in our configuration; see docker/buildx#420 for details. Perhaps we should switch back to plain docker build since we don’t currently build multi-arch images, or move to Buildah.

[skitt]

Depends on docker/buildx#420

[stale]

This issue has been automatically marked as stale because it has not had activity for 60 days. It will be closed if no further activity occurs. Please make a comment if this issue/pr is still valid. Thank you for your contributions.

[dfarrell07]

The issue this depended on upstream is now fixed, so this can be worked on.

[skitt]

The id we get in package/.image.foo files is now correct, but it’s not usable directly. It gives an image sha, but the sha in the repository isn’t the same; so to check against the sha we get in a pod’s properties, we need to extract the repository sha from the image:

docker image inspect sha256:41d0d1dc4ccf6d6d13e079fa17fd48aa633f3fe1e04550506770417a0a7d5d03 | jq -r '.[0].RepoDigests[0]'

This will produce something like

localhost:5000/submariner-gateway@sha256:cc468fb364997c75fb57f8ff6889a5ac191ab088248bb0758e2e11f218a60b0d

and that can be compared with the imageID in the pods’ descriptions:

$ kubectl get pods -A -l app=submariner-gateway -o json | jq -r '.items[].status.containerStatuses[].imageID'
localhost:5000/submariner-gateway@sha256:cc468fb364997c75fb57f8ff6889a5ac191ab088248bb0758e2e11f218a60b0d