From 6f4fc8a43d4a692d48771ff0de3543b22403a974 Mon Sep 17 00:00:00 2001 From: Simon Belmas <22192134+simbelmas@users.noreply.github.com> Date: Mon, 16 Jan 2023 14:06:57 +0100 Subject: [PATCH 1/8] Copy task foo from 0.2 to 0.3 --- task/skopeo-copy/0.3/README.md | 109 ++++++++++++++++++ .../0.3/samples/docker-secret.yaml | 10 ++ task/skopeo-copy/0.3/samples/quay-secret.yaml | 10 ++ task/skopeo-copy/0.3/samples/run.yaml | 12 ++ .../0.3/samples/serviceaccount.yaml | 7 ++ task/skopeo-copy/0.3/skopeo-copy.yaml | 78 +++++++++++++ .../0.3/tests/pre-apply-task-hook.sh | 5 + task/skopeo-copy/0.3/tests/run.yaml | 17 +++ 8 files changed, 248 insertions(+) create mode 100644 task/skopeo-copy/0.3/README.md create mode 100644 task/skopeo-copy/0.3/samples/docker-secret.yaml create mode 100644 task/skopeo-copy/0.3/samples/quay-secret.yaml create mode 100644 task/skopeo-copy/0.3/samples/run.yaml create mode 100644 task/skopeo-copy/0.3/samples/serviceaccount.yaml create mode 100644 task/skopeo-copy/0.3/skopeo-copy.yaml create mode 100644 task/skopeo-copy/0.3/tests/pre-apply-task-hook.sh create mode 100644 task/skopeo-copy/0.3/tests/run.yaml diff --git a/task/skopeo-copy/0.3/README.md b/task/skopeo-copy/0.3/README.md new file mode 100644 index 0000000000..46953a9ab7 --- /dev/null +++ b/task/skopeo-copy/0.3/README.md @@ -0,0 +1,109 @@ +# Skopeo + + +[Skopeo](https://github.com/containers/skopeo) is a command line tool for working with remote image registries. Skopeo doesn’t require a daemon to be running while performing its operations. In particular, the handy skopeo command called `copy` will ease the whole image copy operation. Without further ado, you can copy an image from a registry to another simply by running: +``` +skopeo copy docker://internal.registry/myimage:latest / +docker://production.registry/myimage:v1.0 +``` +The copy command will take care of copying the image from `internal.registry` to `production.registry` + +If your production registry requires credentials to login in order to push the image, skopeo can handle that as well. + +``` +skopeo copy --dest-creds prod_user:prod_pass docker://internal.registry/myimage:latest / +docker://production.registry/myimage:v1.0 +``` + +The same goes for credentials for the source registry (internal.registry) by using the `--src-creds` flag. + +It is also useful for copying images between two remote docker registries, such as the registries of two different OpenShift clusters, as shown +``` +skopeo copy docker://busybox:latest oci:busybox_ocilayout:latest +``` +Skopeo copy isn’t limited to remote containers registries. The image prefix `docker://` from the above command define the transport to be used when handling the image. + +There are others also similar to that: + +- atomic +- containers-storage +- dir +- docker +- docker-daemon +- docker-tar +- oci +- ostree + +This `task` can be used to copy one or more than one images to-and fro various storage mechanisms. + +## Install the Task + +``` +kubectl apply -f https://api.hub.tekton.dev/v1/resource/tekton/task/skopeo-copy/0.2/raw +``` + +## Parameters + +- **srcImageURL**: The URL of the image to be copied to the `destination` registry. +- **destImageURL**: The URL of the image where the image from `source` should be copied to. +- **srcTLSverify**: Verify the TLS on the src registry endpoint +- **destTLSverify**: Verify the TLS on the dest registry endpoint + +## Workspace + +- **images-url**: To mount file containing multiple source and destination images registries URL, which is mounted as configMap. + + +## Secrets and ConfigMap +* `Secret` to provide the credentials of the source and destination registry where the image needs to be copied from and to. +* `ConfigMap` to provide support for copying multiple images, this contains file `url.txt` which stores images registry URL's. + + [This](../0.2/samples/quay-secret.yaml) example can help to use secrets for providing credentials of image registries. + +## Platforms + +The Task can be run on `linux/amd64`, `linux/s390x`, `linux/arm64` and `linux/ppc64le` platforms. + +## Usage + +This task will use the `Service Account` with access to the secrets containing source and destination image registry credentials, this will authorize it to the respective image registries. + +In case of multiple source and destination image registries that needs to be copied to and fro, a file named `url.txt` should be created containing all the source and destination image registries `URL` seperated by a space and each set of images should be written in the new line, as shown below. + +``` +docker://quay.io/temp/kubeconfigwriter:v1 docker://quay.io/skopeotest/kube:v1 +docker://quay.io/temp/kubeconfigwriter:v2 docker://quay.io/skopeotest/kube:v2 +``` + +`ConfigMap` should be created using this file. Following `command` can be used to create configMap from the `file`. +``` +kubectl create configmap image-configmap --from-file=url.txt +``` +In case there is only one source and destination image that needs to be copied then, Source and destination image URL needs to be provided in the input params of the task. + +This will result in the image getting copied from the source registry to the destination registry. + + +[This](../0.2/samples/serviceaccount.yaml) will guide the user to use service account for authorization to image registries. + +See [here](../0.2/samples/run.yaml) for example of `TaskRun`. +### Note + +- `Source credentials` are only required, if the source image registry needs authentication to pull the image, whereas `Destination credentials` are always required. + +- In case of multiple source and destination images, `secret` containing `credentials` of all the image registries must be added to the `service account` and configMap containing `url.txt` should be mounted into the workspace, as shown + ``` + workspaces: + - name: images-url + configmap: + name: image-configmap + ``` + + +- If there is only one source and destination image registry URL, then `emptyDir` needs to be mounted in the `workspace` as shown below: + + ``` + workspaces: + - name: images-url + emptyDir: {} + ``` diff --git a/task/skopeo-copy/0.3/samples/docker-secret.yaml b/task/skopeo-copy/0.3/samples/docker-secret.yaml new file mode 100644 index 0000000000..3fca036138 --- /dev/null +++ b/task/skopeo-copy/0.3/samples/docker-secret.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Secret +metadata: + name: docker-creds + annotations: + tekton.dev/docker-0: https://docker.io +type: kubernetes.io/basic-auth +stringData: + username: test + password: test@1 \ No newline at end of file diff --git a/task/skopeo-copy/0.3/samples/quay-secret.yaml b/task/skopeo-copy/0.3/samples/quay-secret.yaml new file mode 100644 index 0000000000..009ac552be --- /dev/null +++ b/task/skopeo-copy/0.3/samples/quay-secret.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Secret +metadata: + name: quay-creds + annotations: + tekton.dev/docker-0: https://quay.io +type: kubernetes.io/basic-auth +stringData: + username: skopeotest + password: Skopeo@1 \ No newline at end of file diff --git a/task/skopeo-copy/0.3/samples/run.yaml b/task/skopeo-copy/0.3/samples/run.yaml new file mode 100644 index 0000000000..f7f2a93eae --- /dev/null +++ b/task/skopeo-copy/0.3/samples/run.yaml @@ -0,0 +1,12 @@ +apiVersion: tekton.dev/v1beta1 +kind: TaskRun +metadata: + name: skopeo-run +spec: + serviceAccountName: secret-service-account + taskRef: + name: skopeo-copy + workspaces: + - name: images-url + configmap: + name: image-configmap \ No newline at end of file diff --git a/task/skopeo-copy/0.3/samples/serviceaccount.yaml b/task/skopeo-copy/0.3/samples/serviceaccount.yaml new file mode 100644 index 0000000000..f4642d8904 --- /dev/null +++ b/task/skopeo-copy/0.3/samples/serviceaccount.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: secret-service-account +secrets: + - name: docker-creds + - name: quay-creds \ No newline at end of file diff --git a/task/skopeo-copy/0.3/skopeo-copy.yaml b/task/skopeo-copy/0.3/skopeo-copy.yaml new file mode 100644 index 0000000000..978930ad07 --- /dev/null +++ b/task/skopeo-copy/0.3/skopeo-copy.yaml @@ -0,0 +1,78 @@ +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: skopeo-copy + labels: + app.kubernetes.io/version: "0.2" + annotations: + tekton.dev/pipelines.minVersion: "0.12.1" + tekton.dev/categories: CLI + tekton.dev/tags: cli + tekton.dev/displayName: "skopeo copy" + tekton.dev/platforms: "linux/amd64,linux/s390x,linux/ppc64le,linux/arm64" +spec: + description: >- + Skopeo is a command line tool for working with remote image registries. + + Skopeo doesn’t require a daemon to be running while performing its operations. + In particular, the handy skopeo command called copy will ease the whole image + copy operation. The copy command will take care of copying the image from + internal.registry to production.registry. If your production registry requires + credentials to login in order to push the image, skopeo can handle that as well. + + workspaces: + - name: images-url + params: + - name: srcImageURL + description: URL of the image to be copied to the destination registry + type: string + default: "" + - name: destImageURL + description: URL of the image where the image from source should be copied to + type: string + default: "" + - name: srcTLSverify + description: Verify the TLS on the src registry endpoint + type: string + default: "true" + - name: destTLSverify + description: Verify the TLS on the dest registry endpoint + type: string + default: "true" + steps: + - name: skopeo-copy + env: + - name: HOME + value: /tekton/home + image: quay.io/skopeo/stable:v1.9.0 + script: | + # Function to copy multiple images. + # + copyimages() { + filename="$(workspaces.images-url.path)/url.txt" + while IFS= read -r line || [ -n "$line" ] + do + cmd="" + for url in $line + do + # echo $url + cmd="$cmd \ + $url" + done + skopeo copy "$cmd" --src-tls-verify="$(params.srcTLSverify)" --dest-tls-verify="$(params.destTLSverify)" + echo "$cmd" + done < "$filename" + } + # + # If single image is to be copied then, it can be passed through + # params in the taskrun. + if [ "$(params.srcImageURL)" != "" ] && [ "$(params.destImageURL)" != "" ] ; then + skopeo copy "$(params.srcImageURL)" "$(params.destImageURL)" --src-tls-verify="$(params.srcTLSverify)" --dest-tls-verify="$(params.destTLSverify)" + else + # If file is provided as a configmap in the workspace then multiple images can be copied. + # + copyimages + fi + securityContext: + runAsNonRoot: true + runAsUser: 65532 diff --git a/task/skopeo-copy/0.3/tests/pre-apply-task-hook.sh b/task/skopeo-copy/0.3/tests/pre-apply-task-hook.sh new file mode 100644 index 0000000000..f7903cbed0 --- /dev/null +++ b/task/skopeo-copy/0.3/tests/pre-apply-task-hook.sh @@ -0,0 +1,5 @@ +#!/usr/bin/env bash + +# Add an internal registry as sidecar to the task so we can upload it directly +# from our tests without having to go to an external registry. +add_sidecar_registry ${TMPF} \ No newline at end of file diff --git a/task/skopeo-copy/0.3/tests/run.yaml b/task/skopeo-copy/0.3/tests/run.yaml new file mode 100644 index 0000000000..282179e818 --- /dev/null +++ b/task/skopeo-copy/0.3/tests/run.yaml @@ -0,0 +1,17 @@ +apiVersion: tekton.dev/v1beta1 +kind: TaskRun +metadata: + name: skopeo-run +spec: + params: + - name: srcImageURL + value: docker://quay.io/temp/kubeconfigwriter:v1 + - name: destImageURL + value: docker://localhost:5000/kube:latest + - name: destTLSverify + value: "false" + taskRef: + name: skopeo-copy + workspaces: + - name: images-url + emptyDir: {} \ No newline at end of file From 6cce056ed30a8c6f31cb2b4db1022626c21e4987 Mon Sep 17 00:00:00 2001 From: Simon Belmas <22192134+simbelmas@users.noreply.github.com> Date: Mon, 16 Jan 2023 14:10:01 +0100 Subject: [PATCH 2/8] Refactor line analysis and add ability to pass skopeo copy parameters when using ConfigMap --- task/skopeo-copy/0.3/README.md | 6 ++++++ task/skopeo-copy/0.3/skopeo-copy.yaml | 16 ++++++---------- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/task/skopeo-copy/0.3/README.md b/task/skopeo-copy/0.3/README.md index 46953a9ab7..dfeee51c0e 100644 --- a/task/skopeo-copy/0.3/README.md +++ b/task/skopeo-copy/0.3/README.md @@ -75,6 +75,12 @@ docker://quay.io/temp/kubeconfigwriter:v1 docker://quay.io/skopeotest/kube:v1 docker://quay.io/temp/kubeconfigwriter:v2 docker://quay.io/skopeotest/kube:v2 ``` +Each copy can be customized by adding *skopeo copy* parameters before images transport. + +``` +--all docker://quay.io/temp/kubeconfigwriter:v1 docker://quay.io/skopeotest/kube:v1 +``` + `ConfigMap` should be created using this file. Following `command` can be used to create configMap from the `file`. ``` kubectl create configmap image-configmap --from-file=url.txt diff --git a/task/skopeo-copy/0.3/skopeo-copy.yaml b/task/skopeo-copy/0.3/skopeo-copy.yaml index 978930ad07..4e64416bf6 100644 --- a/task/skopeo-copy/0.3/skopeo-copy.yaml +++ b/task/skopeo-copy/0.3/skopeo-copy.yaml @@ -45,22 +45,18 @@ spec: - name: HOME value: /tekton/home image: quay.io/skopeo/stable:v1.9.0 + imagePullPolicy: IfNotPresent script: | # Function to copy multiple images. # copyimages() { filename="$(workspaces.images-url.path)/url.txt" - while IFS= read -r line || [ -n "$line" ] + while read -r line || [ -n "$line" ] do - cmd="" - for url in $line - do - # echo $url - cmd="$cmd \ - $url" - done - skopeo copy "$cmd" --src-tls-verify="$(params.srcTLSverify)" --dest-tls-verify="$(params.destTLSverify)" - echo "$cmd" + ( + set +x + skopeo copy $line --src-tls-verify="$(params.srcTLSverify)" --dest-tls-verify="$(params.destTLSverify)" + ) done < "$filename" } # From ffb1daba1b987addf17366a68550f854ff952a5a Mon Sep 17 00:00:00 2001 From: Simon Belmas <22192134+simbelmas@users.noreply.github.com> Date: Mon, 16 Jan 2023 14:14:36 +0100 Subject: [PATCH 3/8] Update version tag --- task/skopeo-copy/0.3/skopeo-copy.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task/skopeo-copy/0.3/skopeo-copy.yaml b/task/skopeo-copy/0.3/skopeo-copy.yaml index 4e64416bf6..d7d58653dc 100644 --- a/task/skopeo-copy/0.3/skopeo-copy.yaml +++ b/task/skopeo-copy/0.3/skopeo-copy.yaml @@ -3,7 +3,7 @@ kind: Task metadata: name: skopeo-copy labels: - app.kubernetes.io/version: "0.2" + app.kubernetes.io/version: "0.3" annotations: tekton.dev/pipelines.minVersion: "0.12.1" tekton.dev/categories: CLI From 6738a9eaee7c980ffad6d76eb0ac5f8c3284331f Mon Sep 17 00:00:00 2001 From: Simon Belmas <22192134+simbelmas@users.noreply.github.com> Date: Mon, 16 Jan 2023 14:32:37 +0100 Subject: [PATCH 4/8] skopeo-copy: fix wrong shell flag for displaying commands --- task/skopeo-copy/0.3/skopeo-copy.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task/skopeo-copy/0.3/skopeo-copy.yaml b/task/skopeo-copy/0.3/skopeo-copy.yaml index d7d58653dc..681031c77d 100644 --- a/task/skopeo-copy/0.3/skopeo-copy.yaml +++ b/task/skopeo-copy/0.3/skopeo-copy.yaml @@ -54,7 +54,7 @@ spec: while read -r line || [ -n "$line" ] do ( - set +x + set -x skopeo copy $line --src-tls-verify="$(params.srcTLSverify)" --dest-tls-verify="$(params.destTLSverify)" ) done < "$filename" From 88506ba1d949a3374630b5eb8912872df5acfd6b Mon Sep 17 00:00:00 2001 From: Simon <22192134+simbelmas@users.noreply.github.com> Date: Fri, 21 Jul 2023 10:51:08 +0200 Subject: [PATCH 5/8] Update task/skopeo-copy/0.3/README.md Co-authored-by: Vinamra Jain --- task/skopeo-copy/0.3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task/skopeo-copy/0.3/README.md b/task/skopeo-copy/0.3/README.md index dfeee51c0e..b1b5de0632 100644 --- a/task/skopeo-copy/0.3/README.md +++ b/task/skopeo-copy/0.3/README.md @@ -39,7 +39,7 @@ This `task` can be used to copy one or more than one images to-and fro various s ## Install the Task ``` -kubectl apply -f https://api.hub.tekton.dev/v1/resource/tekton/task/skopeo-copy/0.2/raw +kubectl apply -f https://api.hub.tekton.dev/v1/resource/tekton/task/skopeo-copy/0.3/raw ``` ## Parameters From 7e50b4c50cfc6bcf2dc569de46d9621ca8db22e0 Mon Sep 17 00:00:00 2001 From: Simon <22192134+simbelmas@users.noreply.github.com> Date: Fri, 21 Jul 2023 10:51:21 +0200 Subject: [PATCH 6/8] Update task/skopeo-copy/0.3/README.md Co-authored-by: Vinamra Jain --- task/skopeo-copy/0.3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task/skopeo-copy/0.3/README.md b/task/skopeo-copy/0.3/README.md index b1b5de0632..8367766c16 100644 --- a/task/skopeo-copy/0.3/README.md +++ b/task/skopeo-copy/0.3/README.md @@ -92,7 +92,7 @@ This will result in the image getting copied from the source registry to the des [This](../0.2/samples/serviceaccount.yaml) will guide the user to use service account for authorization to image registries. -See [here](../0.2/samples/run.yaml) for example of `TaskRun`. +See [here](../0.3/samples/run.yaml) for example of `TaskRun`. ### Note - `Source credentials` are only required, if the source image registry needs authentication to pull the image, whereas `Destination credentials` are always required. From 5bfdc0fa61671cfe68995e47752a09f34eadd161 Mon Sep 17 00:00:00 2001 From: Simon <22192134+simbelmas@users.noreply.github.com> Date: Fri, 21 Jul 2023 10:51:27 +0200 Subject: [PATCH 7/8] Update task/skopeo-copy/0.3/README.md Co-authored-by: Vinamra Jain --- task/skopeo-copy/0.3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task/skopeo-copy/0.3/README.md b/task/skopeo-copy/0.3/README.md index 8367766c16..45966f4b21 100644 --- a/task/skopeo-copy/0.3/README.md +++ b/task/skopeo-copy/0.3/README.md @@ -90,7 +90,7 @@ In case there is only one source and destination image that needs to be copied t This will result in the image getting copied from the source registry to the destination registry. -[This](../0.2/samples/serviceaccount.yaml) will guide the user to use service account for authorization to image registries. +[This](../0.3/samples/serviceaccount.yaml) will guide the user to use service account for authorization to image registries. See [here](../0.3/samples/run.yaml) for example of `TaskRun`. ### Note From d4fffe5f8f169cafc664692e5d9d8145e8b39316 Mon Sep 17 00:00:00 2001 From: Simon <22192134+simbelmas@users.noreply.github.com> Date: Fri, 21 Jul 2023 10:51:36 +0200 Subject: [PATCH 8/8] Update task/skopeo-copy/0.3/README.md Co-authored-by: Vinamra Jain --- task/skopeo-copy/0.3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/task/skopeo-copy/0.3/README.md b/task/skopeo-copy/0.3/README.md index 45966f4b21..e22286209a 100644 --- a/task/skopeo-copy/0.3/README.md +++ b/task/skopeo-copy/0.3/README.md @@ -58,7 +58,7 @@ kubectl apply -f https://api.hub.tekton.dev/v1/resource/tekton/task/skopeo-copy/ * `Secret` to provide the credentials of the source and destination registry where the image needs to be copied from and to. * `ConfigMap` to provide support for copying multiple images, this contains file `url.txt` which stores images registry URL's. - [This](../0.2/samples/quay-secret.yaml) example can help to use secrets for providing credentials of image registries. + [This](../0.3/samples/quay-secret.yaml) example can help to use secrets for providing credentials of image registries. ## Platforms