Date: 2023-06-30
Proposed
- Relates to ADR 17. Use our own pipelines
The purpose of this document is to establish container image management practices for Konflux container images that are deployed in the staging and production environments. The goal is to ensure that Konflux is continuously maintaining secure operations that are in accordance with the ESS SEC-PATCH-REQ-2 (OS Patching) requirements.
- The scope of this process is limited to the images found in our quay.io/organization/redhat-appstudio repository.
- Images from dependencies that fall outside of this Konflux process should follow the ESS Security Patching at Application/OS Level (requirements 27 and 28). It is up to the component teams to ensure they are adhering to these requirements.
- Images that are not intended for the staging and/or production environments are out of scope.
Component Team: Develops and maintains components that are built as images and deployed as part of Konflux
Component Teams are responsible for ensuring their container images are continuously built and scanned for vulnerabilities by following the Extending the Service process to onboard their component to the PaC service.
Under the PaC service, images are rebuilt when there are updates to the component’s git repository but additional configuration is needed in the Dockerfile to ensure the underlying base (UBI) images are updated with the latest packages (see the HAS example) or at the very least, the latest security updates. This will minimize the gap between patching and should meet our CVE timelines as long as the repository is active.
Component teams are encouraged to install renovatebot to keep their dependencies up to date.
Since image updates are based on how active our repos are, there is the risk that over time, as code stabilizes and/or enters maintenance mode, the triggers for rebuilds will be less frequent which will cause the images to degrade. To avoid this, component teams should also ensure there are scheduled, weekly builds or builds driven by renovatebot or dependabot in place.
Newly onboarded components are required to use a fully supported and patched major version release for their base images per ESS SEC-PATCH-REQ-2 requirement #3. Installing renovatebot can help achieve this requirement.
It is recommended that component teams set up notifications to receive vulnerability alerts that are at least medium severity. This can be done in a couple of ways:
- Set up an alert in quay.io which supports email and Slack integration
- Use the following github action to report vulnerabilities under the action tab. You can copy this workflow and this script onto your repo and set the variables
While our automation process will ensure that component teams are keeping their images updated, security scanners are not perfect. Vulnerabilities can be reported through other channels, in which case, component teams must assess the severity of these findings and remediate according to the Infosec Remediation Guidelines
Component teams should be aware of the lifecycle policy for their base images by referring to the RedHat Product Lifecycle page. Any base image version that is within 3 months of retiring must be updated to the latest patched major release. This should be supported by the deprecated-base-image check in the PAC pipeline.
The Red Hat Product Security Exception Process must be followed in the event that images cannot be patched or updated within the remediation timelines. Some example scenarios:
- A fix that poses a risk to our service is not being provided by the vendor within the remediation timeline
- A deployed container image containing an EOL base image
- An image that cannot be scanned due to an unsupported manifest
In summary, component teams should have the following in place in order to meet ESS requirements:
- Onboard and integrate with PaC
- Ensure their container images are built regularly relying on both PR triggers and scheduled builds
- Ensure the base image layer is also updated with every image rebuild
- Set up vulnerability alerts
- Understand and follow the remediation timelines
- Understand and follow the exception process
- Images deployed from dependencies must not have vulnerabilities
See also RHTAP-828.