Sample of building with ko

[evankanderson]

For discussion!!

Many larger projects in the Tekton/Sigstore ecosystem use ko to build container images; it's mostly maintained by Chainguard folks at the moment, but has been donated to the CNCF from Google.

I didn't replicate all the tagging, but the rest of the functionality should be there (and we can remove the Dockerfile as well if we want).

@JAORMX particularly for feedback

[JAORMX]

alright! I love how much this simplifies the pipeline. With the multi-stage build, we would have some reference on what tooling was used to do the build (we should probably switch the moving tag to a sha reference for the golang container). With this, I guess this responsibility falls on the github actions worker; how may we get this info? or can ko generate the SBOM of tooling used?

I'm still quite new to ko, but so far I really like its UX.

[lukehinds]

+1 , I really like ko. I have not played with it for a while, so not sure if I can provide a meaningful review, but I support the adoption.

[evankanderson]

alright! I love how much this simplifies the pipeline. With the multi-stage build, we would have some reference on what tooling was used to do the build (we should probably switch the moving tag to a sha reference for the golang container). With this, I guess this responsibility falls on the github actions worker; how may we get this info? or can ko generate the SBOM of tooling used?

I'm still quite new to ko, but so far I really like its UX.

https://ko.build/features/sboms/

From v0.9+, ko generates and uploads an SBOM for every image it produces by default.

It generates in SPDX by default, but we can also get it to publish in CycloneDX. I don't recall whether or not it already has cosign support, but I'm pretty sure it does and it's probably where a lot of the cosign folks were doing experiments.

I'm not sure how ko records the version of Golang used, I suspect that could be somewhat improved. At my last job, we used a docker container to run ko inside to get bitwise-reproducible builds (tested for each release).

[evankanderson]

I do need to debug:

Error: failed to publish images: error publishing ko://github.com/stacklok/mediator/cmd/server: writing sbom: POST https://ghcr.io/v2/stacklok/mediator/server-548fa59b46198c350e860f2189f53043/blobs/uploads/: DENIED: installation not allowed to Create organization package

[evankanderson]

Oh! This is because I'm developing on a fork...

Which is probably correct. @JAORMX -- how did you see building images from PRs working? I haven't used that flow before.

[yrobla]

i had heard about ko but never experimented with that. PIpeline looks way cleaner, so +1 on this change.

[JAORMX]

Have you tested if the database migration works? I don't see any references that would include the files in the resulting image.

[evankanderson]

I didn't copy in the files yet; to do that, we'll need to add (symlink is fine) files from ./cmd/serve/kodata and fixup the paths to use $KO_DATA_REPO, as described in static assets.

[JAORMX]

I really like the UX for this tool and the direction it takes us. However, with the current setup it breaks the migration container:

Error while creating migration instance: open .: no such file or directory
2023/07/03 13:20:42 Connected to DB

the database/ folder needs to be included in the container for it to work. Is it possible to add that via ko?

[evankanderson]

I really like the UX for this tool and the direction it takes us. However, with the current setup it breaks the migration container:

Error while creating migration instance: open .: no such file or directory
2023/07/03 13:20:42 Connected to DB

the database/ folder needs to be included in the container for it to work. Is it possible to add that via ko?

I think I've fixed all this. Unfortunately, along the way I needed to introduce a make target to get the docker-compose working on MacOS in addition to Linux (irregardless of ko usage).

I also ended up doing some forceful overrides of the filepaths-in-config-files for the server; the other option would be to try to coordinate between the image working directory, the config file location, and the location that the .ssh directory is mounted by the docker-compose.yaml or (speculative, future) Kubernetes configuration. I'm suspecting that config-file-that-contains-paths-to-other-files may be an antipatern. If not, we should be explicit about how the config file and the other referenced files related filesystem wise.

I wanted to be clever and run the docker-compose as docker-compose -f <(ko resolve ...) up, but it turns out that the volume mounts in the docker-compose file are relative to the docker-compose file, so the <() syntax ends up breaking because it translates to /dev/fd/4 or the like, and there's no way to override the relative directory in docker-compose.

[evankanderson]

(All tests now pass, I did a few unspeakable things in migrate up and migrate down to accommodate KO_DATA_PATH where needed but not require it for make run-server)

[evankanderson]

Ready for review, @JAORMX . 😁