build(scorecard): update multiarch build steps #610

tthvo · 2023-08-23T23:54:18Z

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Read the contributing guidelines
Linked a relevant issue which this PR resolves
Linked any other relevant issues, PR's, or documentation, if any
Resolved all conflicts, if any
Rebased your branch PR on top of the latest upstream main branch
Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
Signed all commits: git commit -S -m "YOUR_COMMIT_MESSAGE"

Related to #604

Description of the change:

Specified path to container file when building multi-archs with podman.
Remove multi-arch scorecard build in tests.

Motivation for the change:

Tests failed because scorecard images are build with incorrect container files.

tthvo · 2023-08-24T00:42:13Z

Follow up issue to run tests on arm64, #611.

tthvo · 2023-08-24T04:32:10Z

Sample run:

https://github.com/tthvo/cryostat-operator/actions/runs/5959432448/job/16165164859?pr=6

I tried out the GITHUB_TOKEN and seems to work just fine in pull_request_target event to push images. We can later use it instead of PAT :))

Makefile

ebaron

Thanks for taking care of this @tthvo!

tthvo · 2023-08-24T20:50:43Z

Niceee! Glad I could help!

tthvo · 2023-08-24T20:52:32Z

Actually, I just realized github action is using podman v3. I will see if I can upgrade it to podman v4 to support these new syntaxes.

.github/workflows/ci.yaml

tthvo · 2023-08-24T22:54:23Z

Ready to review again :))

Got a little hiccup with the podman v4 on ubuntu but the workaround seems fine:

PR: https://github.com/tthvo/cryostat-operator/actions/runs/5969689633/job/16195999752?pr=6
Push: https://github.com/tthvo/cryostat-operator/actions/runs/5969687216/job/16195993263

.github/workflows/ci.yaml

andrewazores · 2023-08-25T13:37:54Z

$ MANIFEST_PUSH=false PLATFORMS=linux/amd64,linux/arm64 make scorecard-build 
mkdir -p /home/work/workspace/cryostat-operator/bin
test -s /home/work/workspace/cryostat-operator/bin/go-license || GOBIN=/home/work/workspace/cryostat-operator/bin go install github.com/palantir/[email protected]
Adding license...
/home/work/workspace/cryostat-operator/bin/go-license --config=go-license.yml api/v1beta1/clustercryostat_types.go api/v1beta1/cryostat_types.go api/v1beta1/groupversion_info.go api/v1beta1/zz_generated.deepcopy.go internal/controllers/common/resource_definitions/certificates.go internal/controllers/common/resource_definitions/resource_definitions.go internal/controllers/common/common_utils.go internal/controllers/common/finalizer_utils.go internal/controllers/common/tls.go internal/controllers/constants/constants.go internal/controllers/model/instance.go internal/controllers/const_generated.go internal/controllers/certmanager.go internal/controllers/clustercryostat_controller.go internal/controllers/clustercryostat_controller_test.go internal/controllers/configmaps.go internal/controllers/cryostat_controller.go internal/controllers/cryostat_controller_test.go internal/controllers/ingresses.go internal/controllers/openshift.go internal/controllers/pvc.go internal/controllers/rbac.go internal/controllers/reconciler.go internal/controllers/reconciler_test.go internal/controllers/routes.go internal/controllers/secrets.go internal/controllers/services.go internal/controllers/suite_test.go internal/images/custom-scorecard-tests/main.go internal/test/scorecard/clients.go internal/test/scorecard/openshift.go internal/test/scorecard/tests.go internal/test/clients.go internal/test/reconciler.go internal/test/resources.go internal/tools/const_generator.go internal/main.go
go fmt ./...
go vet ./...
cd internal/images/custom-scorecard-tests/ && \
go build -o bin/cryostat-scorecard-tests main.go
printf '# Code generated by hack/custom.config.yaml.in. DO NOT EDIT.\n' > config/scorecard/patches/custom.config.yaml
envsubst < hack/custom.config.yaml.in >> config/scorecard/patches/custom.config.yaml
sed -e '1 s/\(^FROM\)/FROM --platform=\$\{BUILDPLATFORM\}/; t' -e ' 1,// s//FROM --platform=\$\{BUILDPLATFORM\}/' internal/images/custom-scorecard-tests/Dockerfile > internal/images/custom-scorecard-tests/Dockerfile.cross
BUILDAH_FORMAT=docker podman build -f internal/images/custom-scorecard-tests/Dockerfile.cross --manifest quay.io/cryostat/cryostat-operator-scorecard:2.4.0-20230825133433 --platform linux/amd64,linux/arm64 . ; \
if [ "false" = "true" ] ; then \
	podman manifest push quay.io/cryostat/cryostat-operator-scorecard:2.4.0-20230825133433 quay.io/cryostat/cryostat-operator-scorecard:2.4.0-20230825133433 ; \
fi
[linux/arm64] [1/2] STEP 1/11: FROM docker.io/library/golang:1.20 AS builder
[linux/arm64] [1/2] STEP 2/11: ARG TARGETOS
--> Using cache e113e9f8622597abf8c191c4d43cc158b6dc021a61f9fdea8205a9df3af4b260
--> e113e9f86225
[linux/arm64] [1/2] STEP 3/11: ARG TARGETARCH
--> Using cache c2a23666efb6f1a62a3bb2198e7193a55820c1e68e31f1606d8cf0c28f90e2c1
--> c2a23666efb6
[linux/arm64] [1/2] STEP 4/11: WORKDIR /workspace
--> Using cache 6ab332b98ed9bed37917cf152b70b20129937d6e11f3195212403f81a2197b14
--> 6ab332b98ed9
[linux/arm64] [1/2] STEP 5/11: COPY go.mod go.mod
--> Using cache 2afad439f42a56d76920eb6f8842f3e57c612495b77ace8315b70891ef0f9c3c
--> 2afad439f42a
[linux/arm64] [1/2] STEP 6/11: COPY go.sum go.sum
--> Using cache dda8574cac71832c694c0e8212560f76945fb0720aa1976ab9fc02deba12d8c9
--> dda8574cac71
[linux/arm64] [1/2] STEP 7/11: RUN go mod download
--> Using cache 89c8c9d32e54b94a2e23040a051b5184981f2101b69ac592383859eaeebaf27a
--> 89c8c9d32e54
[linux/arm64] [1/2] STEP 8/11: COPY api/ api/
--> Using cache 474b657f881a7968c6a8858300ddec3d5858670d0bb3cfd026687c5becafdecf
--> 474b657f881a
[linux/arm64] [1/2] STEP 9/11: COPY internal/images/custom-scorecard-tests/main.go internal/images/custom-scorecard-tests/main.go
--> Using cache b6912c5150a8f0a8228a2d2a62242871b957165ce2f3e9044a47e395ba0bad3d
--> b6912c5150a8
[linux/arm64] [1/2] STEP 10/11: COPY internal/test/scorecard/ internal/test/scorecard/
--> Using cache e9e9057a8390a114c4051a584fa1c0bf5bc686c98c9bbfa0d1d4c9e444fc5e35
--> e9e9057a8390
[linux/arm64] [1/2] STEP 11/11: RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} GO111MODULE=on go build -a -o cryostat-scorecard-tests     internal/images/custom-scorecard-tests/main.go
--> 98afc27783e4
[linux/arm64] [2/2] STEP 1/8: FROM registry.access.redhat.com/ubi8/ubi-minimal:latest
Trying to pull registry.access.redhat.com/ubi8/ubi-minimal:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob 0c10cd59e10e skipped: already exists  
Copying config ba3de24863 done  
Writing manifest to image destination
Storing signatures
[linux/arm64] [2/2] STEP 2/8: ENV TEST=/usr/local/bin/cryostat-scorecard-tests     USER_UID=1001     USER_NAME=test
--> ce3cde3c9890
[linux/arm64] [2/2] STEP 3/8: COPY internal/images/custom-scorecard-tests/bin/user_setup /usr/local/bin/
--> b41d2534162f
[linux/arm64] [2/2] STEP 4/8: COPY internal/images/custom-scorecard-tests/bin/entrypoint /usr/local/bin/
--> 49dfb3a01b5c
[linux/arm64] [2/2] STEP 5/8: COPY --from=builder /workspace/cryostat-scorecard-tests /usr/local/bin/
--> 78241fade646
[linux/arm64] [2/2] STEP 6/8: RUN  /usr/local/bin/user_setup
+ echo 'test:x:1001:0:test user:/root:/sbin/nologin'
+ mkdir -p /root
+ chown 1001:0 /root
+ chmod ug+rwx /root
+ rm /usr/local/bin/user_setup
--> 32e1e4d37c19
[linux/arm64] [2/2] STEP 7/8: ENTRYPOINT ["/usr/local/bin/entrypoint"]
--> 870d12e4c714
[linux/arm64] [2/2] STEP 8/8: USER ${USER_UID}
[linux/arm64] [2/2] COMMIT
--> 6919cf9cd43a
[linux/amd64] [1/2] STEP 1/11: FROM docker.io/library/golang:1.20 AS builder
6919cf9cd43af208fbd101b620d93f2ff61f4781c8d3f037ecf9403a0df1921c
[linux/amd64] [1/2] STEP 2/11: ARG TARGETOS
--> Using cache e113e9f8622597abf8c191c4d43cc158b6dc021a61f9fdea8205a9df3af4b260
--> e113e9f86225
[linux/amd64] [1/2] STEP 3/11: ARG TARGETARCH
--> Using cache c2a23666efb6f1a62a3bb2198e7193a55820c1e68e31f1606d8cf0c28f90e2c1
--> c2a23666efb6
[linux/amd64] [1/2] STEP 4/11: WORKDIR /workspace
--> Using cache 6ab332b98ed9bed37917cf152b70b20129937d6e11f3195212403f81a2197b14
--> 6ab332b98ed9
[linux/amd64] [1/2] STEP 5/11: COPY go.mod go.mod
--> Using cache 2afad439f42a56d76920eb6f8842f3e57c612495b77ace8315b70891ef0f9c3c
--> 2afad439f42a
[linux/amd64] [1/2] STEP 6/11: COPY go.sum go.sum
--> Using cache dda8574cac71832c694c0e8212560f76945fb0720aa1976ab9fc02deba12d8c9
--> dda8574cac71
[linux/amd64] [1/2] STEP 7/11: RUN go mod download
--> 739c50737219
[linux/amd64] [1/2] STEP 8/11: COPY api/ api/
--> d782145b16bf
[linux/amd64] [1/2] STEP 9/11: COPY internal/images/custom-scorecard-tests/main.go internal/images/custom-scorecard-tests/main.go
--> 1d4d986d8c38
[linux/amd64] [1/2] STEP 10/11: COPY internal/test/scorecard/ internal/test/scorecard/
--> db07ef9ce2b0
[linux/amd64] [1/2] STEP 11/11: RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} GO111MODULE=on go build -a -o cryostat-scorecard-tests     internal/images/custom-scorecard-tests/main.go
--> 800fc5edfb51
[linux/amd64] [2/2] STEP 1/8: FROM registry.access.redhat.com/ubi8/ubi-minimal:latest
[linux/amd64] [2/2] STEP 2/8: ENV TEST=/usr/local/bin/cryostat-scorecard-tests     USER_UID=1001     USER_NAME=test
--> Using cache ce3cde3c989050149fa28d2ca095a9b8cf4f2b37cc35f9b89be18ce4a8d29301
--> ce3cde3c9890
[linux/amd64] [2/2] STEP 3/8: COPY internal/images/custom-scorecard-tests/bin/user_setup /usr/local/bin/
--> Using cache b41d2534162ff917466cd054b52025e8c929550cc1ac4ab392156524d42f3e22
--> b41d2534162f
[linux/amd64] [2/2] STEP 4/8: COPY internal/images/custom-scorecard-tests/bin/entrypoint /usr/local/bin/
--> Using cache 49dfb3a01b5c094c1a8c9519984a841108519cd6b866840980528576de5e3340
--> 49dfb3a01b5c
[linux/amd64] [2/2] STEP 5/8: COPY --from=builder /workspace/cryostat-scorecard-tests /usr/local/bin/
--> 435318c69569
[linux/amd64] [2/2] STEP 6/8: RUN  /usr/local/bin/user_setup
+ echo 'test:x:1001:0:test user:/root:/sbin/nologin'
+ mkdir -p /root
+ chown 1001:0 /root
+ chmod ug+rwx /root
+ rm /usr/local/bin/user_setup
--> b4ef24f841ff
[linux/amd64] [2/2] STEP 7/8: ENTRYPOINT ["/usr/local/bin/entrypoint"]
--> ab841a97672c
[linux/amd64] [2/2] STEP 8/8: USER ${USER_UID}
[linux/amd64] [2/2] COMMIT
--> 5db7fd5393ca
5db7fd5393cab49cbe4109175fde77d4ccbdf9ed71d66e78428328690b0db3d8
rm internal/images/custom-scorecard-tests/Dockerfile.cross


$ podman images | grep scorecard
quay.io/cryostat/cryostat-operator-scorecard        2.4.0-20230825133433          fa2e5d42b9da  23 seconds ago      1.06 kB

$ podman manifest inspect quay.io/cryostat/cryostat-operator-scorecard:2.4.0-20230825133433 
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 1054,
            "digest": "sha256:cd3fc2c8834dbcdcca45a6fff2644a6dd6f1ffdbc28c8d89b18a2b176ece2946",
            "platform": {
                "architecture": "arm64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 1054,
            "digest": "sha256:51db5122149638ab2ff3bc61833921ceb78d0a78f32f928cee74a7d3aab599dd",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

Build looks good. Basic sanity test that the images really exist and are runnable (QEMU for arm64):

$ podman run --rm --arch amd64 quay.io/cryostat/cryostat-operator-scorecard:2.4.0-20230825133433 
2023/08/25 13:39:48 specify one or more test name arguments
$ podman run --rm --arch arm64 quay.io/cryostat/cryostat-operator-scorecard:2.4.0-20230825133433 
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)
2023/08/25 13:39:41 specify one or more test name arguments

andrewazores

LGTM other than minor suggestion to rename those env vars

Thuan Vo added 2 commits August 23, 2023 16:52

build(scorecard): fix path to container file

ea9c1c2

ci(scorecard): remove multiarch scorecard in tests

cce06fd

tthvo added ci build labels Aug 23, 2023

mergify bot added the safe-to-test label Aug 23, 2023

tthvo marked this pull request as ready for review August 24, 2023 00:38

tthvo requested review from andrewazores and ebaron August 24, 2023 00:38

Thuan Vo added 3 commits August 23, 2023 17:50

fix(make): fix missing Dockerfile.cross

9164222

chore(scorecard): bump scorecard versions

d2d1291

ci(scorecard): limit to amd64 build in tests

4ab7a9a

tthvo force-pushed the multiarch-scorecard branch from 7cfd678 to 4ab7a9a Compare August 24, 2023 03:57

ebaron reviewed Aug 24, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Thuan Vo added 3 commits August 24, 2023 13:24

chore(make): clean up multiarch builds

1843526

chore(api): regenerate source with go 1.20

abddf99

ci(scorecard): remove qemu

2a56af2

tthvo changed the title ~~build(scorecard): specify correct container file for scorecard multiarch builds~~ build(scorecard): update multiarch build steps Aug 24, 2023

ebaron previously approved these changes Aug 24, 2023

View reviewed changes

tthvo dismissed ebaron’s stale review via 6c7e324 August 24, 2023 21:21

tthvo commented Aug 24, 2023

View reviewed changes

.github/workflows/ci.yaml Outdated Show resolved Hide resolved

ci(podman): workaround for podman v4

dfc06aa

tthvo force-pushed the multiarch-scorecard branch 2 times, most recently from a8daae5 to 4d60ed5 Compare August 24, 2023 21:33

ci(scorecard): install podman after check

e8eca86

tthvo force-pushed the multiarch-scorecard branch from 4d60ed5 to e8eca86 Compare August 24, 2023 21:40

ci(podman): clean up podman installation

3a0e836

andrewazores reviewed Aug 25, 2023

View reviewed changes

.github/workflows/ci.yaml Outdated Show resolved Hide resolved

andrewazores reviewed Aug 25, 2023

View reviewed changes

ci(podman): rename env vars

18e05fc

andrewazores approved these changes Aug 25, 2023

View reviewed changes

andrewazores merged commit 46cfe18 into cryostatio:main Aug 25, 2023
10 checks passed

tthvo deleted the multiarch-scorecard branch August 25, 2023 17:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build(scorecard): update multiarch build steps #610

build(scorecard): update multiarch build steps #610

tthvo commented Aug 23, 2023

tthvo commented Aug 24, 2023

tthvo commented Aug 24, 2023

ebaron left a comment

tthvo commented Aug 24, 2023

tthvo commented Aug 24, 2023

tthvo commented Aug 24, 2023 •

edited

Loading

andrewazores commented Aug 25, 2023 •

edited

Loading

andrewazores left a comment

build(scorecard): update multiarch build steps #610

build(scorecard): update multiarch build steps #610

Conversation

tthvo commented Aug 23, 2023

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Description of the change:

Motivation for the change:

tthvo commented Aug 24, 2023

tthvo commented Aug 24, 2023

ebaron left a comment

Choose a reason for hiding this comment

tthvo commented Aug 24, 2023

tthvo commented Aug 24, 2023

tthvo commented Aug 24, 2023 • edited Loading

andrewazores commented Aug 25, 2023 • edited Loading

andrewazores left a comment

Choose a reason for hiding this comment

tthvo commented Aug 24, 2023 •

edited

Loading

andrewazores commented Aug 25, 2023 •

edited

Loading