CRAYSAT-1767: Set cray-sat version in csm-config

[haasken-hpe]

Summary and Scope

Add an initContainer definition to the values defined for the csm-config Helm chart. This initContainer sets the cray-sat image tag to match the version included in the CSM build, so that they are always consistent. The Ansible content in csm-config sets this cray-sat container image tag in the file /opt/cray/etc/sat/version, so that the sat-podman wrapper script knows which version of the image to run.

Add logic to release.sh that inspects the container image tags specified in docker/index.yaml and injects those tags for the cray-sat container image and the Alpine Linux container image into the csm-config Helm chart values in the Loftsman manifest.

Modify lib/setup-nexus.sh to set the cray-sat container image version and tag the latest one as csm-latest. We are phasing out reliance on this csm-latest tag, but it is still the default used by the sat-podman wrapper script if the file /opt/cray/etc/sat/version does not exist. Modify lib/setup-nexus.sh to also write the tag of the cray-sat image to /opt/cray/etc/sat/version as well. This is only executed on ncn-m001, but that is sufficient for bootstrapping. When the CSM layer of the CFS configuration is applied, this file will be set on all master and worker management nodes.

Issues and Related PRs

• Resolves CRAYSAT-1767
• Merge after CRAYSAT-1767: Add csm.ncn.sat role to configure SAT csm-config#213

Testing

Tested on:

• mug

Test description:

I've done various manual tests of each piece of this.

I have tested that shipping a similar Loftsman manifest on mug does result in the correct content being uploaded to the csm-config-management VCS repository. I have tested the individual yq commands and sed commands added to the release.sh script separately outside of their full context.

Risks and Mitigations

This adds an initContainer to the csm-config Helm chart, and if that fails, the import of the csm-config-management repo to VCS will fail.

It's also possible I don't understand the build process fully, and I may have done something incorrectly that won't work due to the ordering of build steps. This should be reviewed carefully.

Pull Request Checklist

• Version number(s) incremented, if applicable
• Copyrights updated
• License file intact
• Target branch correct
• Testing is appropriate and complete, if applicable
• HPC Product Announcement prepared, if applicable

[haasken-hpe]

I didn't realize I didn't get any automatic reviewers. Added reviewers.

• @rbak-hpe to review the init container definition mostly
• @mtupitsyn to review the changes regarding query of container images from the docker index and other build concerns
• @heemstra to review the install changes to lib/setup-nexus.sh mostly
• @mharding-hpe for awareness

[haasken-hpe]

I realized last week that this is going to require one more change in order to work correctly.

Today, the SAT product stream uploads the cray-sat container image to the image registry in Nexus as cray/cray-sat. CSM uploads the cray-sat container image to the image registry in Nexus as artifactory.algol60.net/sat-docker/stable/cray-sat.

Formerly only the SAT product stream used to deliver the Ansible that caused the file /opt/cray/etc/sat/version to be created. Thus, if this file exists, the sat wrapper script provided by cray-sat-podman assumes that the image in Nexus is cray/cray-sat, which is where it is delivered by the separate SAT product stream. Now that the CSM product delivers the Ansible content that populates the file /opt/cray/etc/sat/version, we need to modify the cray-sat-podman wrapper script logic to always use the CSM-provided path in the image registry in Nexus.

This will create a backwards compatibility issue. If a newer version of cray-sat-podman is installed that has this changed logic, it will no longer be able to find the cray-sat images delivered by older versions of SAT. I think this is okay because people should not need to roll back to older cray-sat versions, and if they really need to, it would be easy enough to do by modifying the sat wrapper shell script or by moving images in Nexus.

Here are the steps I'm going to take to handle this:

• Change the sat builds to publish to csm-docker instead of sat-docker. The way CSM uploads container images to Nexus, it keeps the Artifactory path in the image names. I eventually planned to move SAT container images to publish to csm-docker in CRAYSAT-1771. Since I'm going to be changing the code in sat-podman that looks up the image in Nexus anyway, I might as well make the change to the image publishing location now, so I don't have to make two backwards incompatible changes one after the other. I will probably bump the minor version in cray-sat when I do this because if I need to patch the version of the cray-sat image included in CSM 1.5, I'm going to want to have room to bump just the patch version.
• Change the logic in the sat-podman wrapper script to only look for the cray-sat container image in the new csm-docker path in artifactory. This will probably be a major version bump, since it's incompatible with older versions of SAT.

Until I complete those changes, this should not be merged.

[haasken-hpe]

The actions from my previous comment here have been completed. Here are the PRs, which have been merged:

• CRAYSAT-1771: Change builds to also publish to csm-docker sat#173
• CRAYSAT-1771, CRAYSAT-1767: Use CSM image location in Nexus sat-podman#35

There is still a PR which should be merged in metal-provision, but it should be timed to merge with this change, so both changes appear in the same CSM 1.6.0 build. Here is that PR:

• CRAYSAT-1767: Update cray-sat-podman RPM version for CSM 1.6.0 metal-provision#570