Feat cachi2 remove unsafe symlinks flag #2143
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Conversion function that transforms remote_source into cachi2 params STONEBLD-2586 Signed-off-by: Martin Basti <[email protected]>
Cachi2 doesn't generate ICM as Cachito 1, for backward compatibility generate ICM from SBOM Unfortunatelly cachi2 provides only flat structure, so dependencies are part of the flat structure they are not listed separatelly. STONEBLD-2582 Signed-off-by: Martin Basti <[email protected]>
Init task contains plugins which are responsible for fetching data and validation. It split from prebuild task. Prebuild task compared to init task, does changes to dockerfile. This is prerequisite for having replacebale cachito task with cachi2 task in future, as this task has to be between init and prebuild. Parameter --platforms-result has been moved to init task from prebuild task, as init task now runs check_and_set_platforms plugin. STONEBLD-2587 Signed-off-by: Martin Basti <[email protected]>
Separating resolve_remote_sources into a standalone cachito task in binary build, to allow us replacement with cachi2 in future. STONEBLD-2587 Signed-off-by: Martin Basti <[email protected]>
This commit must be removed before merging to main branch. Signed-off-by: Martin Basti <[email protected]>
Cachi2 doesn't provide remote-source.json but for backward compatibility OSBS must provide it. STONEBLD-2585 Signed-off-by: Martin Basti <[email protected]>
Rubygems pkg_manager is named bundler in cachi, name must be converted. STONEBLD-2919 Signed-off-by: Martin Basti <[email protected]>
Empty package manager (not defined) should deafult to gomod, to keep compatibility in behavior with Cachito STONEBLD-2921 Signed-off-by: Martin Basti <[email protected]>
Add init task/step responsible for creating directory structure, cloning remote sources and preparing options file for cachi2 run. Signed-off-by: Martin Basti <[email protected]>
Shellcheck for tekton github action Signed-off-by: Martin Basti <[email protected]>
Add cachi2 step that: - fetches deps - create env file and env json - make source archives - remove git (option to keep git will be added later) - merge sboms into single one Signed-off-by: Martin Basti <[email protected]>
Fixing format of envvars, Cachi2 have slightly different format Signed-off-by: Martin Basti <[email protected]>
Postprocssing plugin to take cachi2 generated dependencies and generate expected metadata for OSBS and prepare sources into build dirs. Signed-off-by: Martin Basti <[email protected]>
use reflink to copy sources into build dir to save space Signed-off-by: Martin Basti <[email protected]>
Update add_image_content_manifest to process cachi2 results. ICM must be generated from cachi2 SBOM in this case. Signed-off-by: Martin Basti <[email protected]>
Update generate_sbom to process cachi2 results. SBOM must be combined from cachi2 SBOM stored in build_dir in this case. Signed-off-by: Martin Basti <[email protected]>
Update koji_import to process cachi2 results. Signed-off-by: Martin Basti <[email protected]>
User/admin may specify which version of remote sources should be used (or used by default): - 1: Cachito (current default) - 2: Cachi2 Build pipeline will then use the right task based on condition using result from the init task. STONEBLD-2591 Signed-off-by: Martin Basti <[email protected]>
Fixing issues reported by shellcheck via checkton. Signed-off-by: Martin Basti <[email protected]>
Remove before merging into main Signed-off-by: Martin Basti <[email protected]>
When single remote source is used, returned name should be None and not an empty string. This si compatible with Cachito behavior. Signed-off-by: Martin Basti <[email protected]>
When users specify empty list as pkg_managers, only cloning should be done. In this case usage of cachi2 must be skipped and required files like env.json and bom.json must be created by OSBS. Signed-off-by: Martin Basti <[email protected]>
This bump is required to validate allowed pkg_managers also on atomic-reactor side. Signed-off-by: Martin Basti <[email protected]>
Rubygems package manager contains data also out of deps/ and app/ directory. The whole remote dir must be copied. For rubygems, bundler config file hasn't been copied. Signed-off-by: Martin Basti <[email protected]>
We cannot realibly determine if dependency was vendored from SBOM metdata. Keep it safe and export version including git od download URL with subpath. Signed-off-by: Martin Basti <[email protected]>
For security reasons, only relative paths within cloned remote source can be specified by users Don't allow to point to symlink pointing out of cloned remote source Signed-off-by: Martin Basti <[email protected]>
undefined pkg_managers means gomod, record this explictily in metadata, so this information is not lost in workflow and proper metadata are returned. Signed-off-by: Martin Basti <[email protected]>
What/why: implement detection/removal of unsafe symlinks in repos, specifically
covering cachi2 use case: Cachito already does this
How:
- copypasta `_enforce_sandbox()` and related unit tests from Cachito
("cachito/cachito/workers/tasks/general.py" and
"cachito/tests/test_workers/test_tasks/test_general.py", respectively)
- add call to `_enforce_sandbox()`
- add CLI boolean arg `remove-unsafe-symlinks`, which toggles removing all
symlinks which point to location(s) outside of any cloned repository
Signed-off-by: Ben Alkov <[email protected]>
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
What/why: implement parsing `remove-unsafe-symlinks` flag, which toggles removing all symlinks which point to location(s) outside of any cloned repository (default `False`) How: - parse flag `remove-unsafe-symlinks` from "remote-source" section of 'container.yaml' Signed-off-by: Ben Alkov <[email protected]>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| cpu: 395m | ||
| script: | | ||
| set -x | ||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-cachito |
Check warning
Code scanning / shellcheck
SC2016 Warning
| cpu: 395m | ||
| script: | | ||
| set -x | ||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-cachito |
Check warning
Code scanning / shellcheck
SC2046 Warning
| cpu: 395m | ||
| script: | | ||
| set -x | ||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-cachito |
Check warning
Code scanning / shellcheck
SC2046 Warning
| cpu: 395m | ||
| script: | | ||
| set -x | ||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-cachito |
Check warning
Code scanning / shellcheck
SC2046 Warning
| cpu: 395m | ||
| script: | | ||
| set -x | ||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-cachito |
Check warning
Code scanning / shellcheck
SC2046 Warning
| @@ -46,4 +43,4 @@ | |||
| cpu: 395m | |||
| script: | | |||
| set -x | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild --platforms-result=$(results.platforms_result.path) | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild | |||
Check warning
Code scanning / shellcheck
SC2016 Warning
| @@ -46,4 +43,4 @@ | |||
| cpu: 395m | |||
| script: | | |||
| set -x | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild --platforms-result=$(results.platforms_result.path) | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild | |||
Check warning
Code scanning / shellcheck
SC2046 Warning
| @@ -46,4 +43,4 @@ | |||
| cpu: 395m | |||
| script: | | |||
| set -x | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild --platforms-result=$(results.platforms_result.path) | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild | |||
Check warning
Code scanning / shellcheck
SC2046 Warning
| @@ -46,4 +43,4 @@ | |||
| cpu: 395m | |||
| script: | | |||
| set -x | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild --platforms-result=$(results.platforms_result.path) | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild | |||
Check warning
Code scanning / shellcheck
SC2046 Warning
| @@ -46,4 +43,4 @@ | |||
| cpu: 395m | |||
| script: | | |||
| set -x | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild --platforms-result=$(results.platforms_result.path) | |||
| atomic-reactor -v task --user-params='$(params.user-params)' --build-dir=$(workspaces.ws-build-dir.path) --context-dir=$(workspaces.ws-context-dir.path) --config-file=$(workspaces.ws-reactor-config-map.path)/config.yaml --namespace=$(context.taskRun.namespace) --pipeline-run-name="$(params.pipeline-run-name)" binary-container-prebuild | |||
Check warning
Code scanning / shellcheck
SC2046 Warning
|
Not meant for this repo |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Maintainers will complete the following section