Using with podman

[ccakes]

I'm trying to use this with podman using the Docker API-compat socket. It seems like it'll probably work but I've hit a snag and don't know enough about the Docker ecosystem to know the "right solution" for a PR.

Running ctlptl apply with config to create a registry fails. It creates the container but then can't find it to proceed with the workflow. The problem is this line

ctlptl/pkg/registry/registry.go

Line 106 in f5d2f38

filterArgs.Add("ancestor", "registry:2") // The registry everyone uses.

If I change that to use the fully-qualified path docker.io/library/registry:2 then it works fine. I'm unsure if that would also work on native Docker though, or if instead I should add a check for podman and set the filter depending on Docker v podman. Thoughts?

[nicks]

ooh nice! Ya, would love to accept a PR that gets this working with podman

I just tried that with the docker CLI and it seemed to work OK. We also have pretty good integration tests that will run on CI, so don't worry too much about breaking it.

[nicks]

@ccakes what version of podman are you using? i was playing around with this and couldn't get the podman container ls --filter to work at all 🤔 (on podman 3.0.1)

[nicks]

OK I upgraded to podman 3.4 and now see the problem. Sent out: https://github.com/tilt-dev/ctlptl/pull/new/nicks/issue146

Unfortunately, kind creation still blows up:

failed to create cluster: failed to ensure docker network: command "docker network create -d=bridge -o com.docker.network.bridge.enable_ip_masquerade=true --ipv6 --subnet fc00:f853:ccd:e793::/64 kind" failed with error: exit status 1

:\

[ccakes]

Sorry for the slow reply - I was using latest podman. I got about as far as you, I suspect that making it work for rootful containers is probably not that much work but rootless (how I was trying) would be much more difficult.

For rootless, it feels like the best option would be to create a pod and run both kind and the registry together in that but doing so would require more changes to ctlptl than I really feel comfortable tackling.

[almereyda]

Thanks to

• registry: make registry creation more robust #150

we now have a working registry with Podman. Using Podman pods to create network contexts still appears out of scope.

Meanwhile it happens that a specific networking setup for K3d clusters on Podman would also be required for connecting a K3d registry to a K3d cluster (on Podman):

• cluster: create k3d from a config file. #265 (comment)

[ncdc]

I'm using ctlptl with podman but I had to create a config file in the podman VM to make it treat the registry created by ctlptl as insecure (otherwise I couldn't use it):

$ cat /etc/containers/registries.conf.d/100-kind.conf
[[registry]]
location = "localhost:61358"
insecure = true

A few questions:

• Should I file a new issue for this discussion?
• Is there any other way to do this that doesn't require manually modifying files on the podman VM?
• If not, would it be useful to do a PR that tries to manage this config for you?

[aripollak]

Besides the registries.conf addition (I put mine in ~/.config/containers/registries.conf), for running native rootless podman under Ubuntu 23.10 (upgraded from 22.04) I also had to install the golang-github-containernetworking-plugin-dnsname package so the kind container could resolve the ctlptl-registry hostname. Since I already had the kind network created in podman, I had to run podman network rm kind and recreate the cluster/registry with ctlptl after installing the -plugin-dnsname package.

I don't think ctlptl could/should install the plugin automatically, so that might just be something that just goes in the instructions, and it might also not be needed with a fresh Ubuntu install where you've only ever used podman 4.0+, since it then uses netavark by default, which has DNS support built in. Based on https://github.com/containers/podman/blob/main/docs/tutorials/basic_networking.md#bridge, the DNS lookup issue is also be fixable by destroying all podman resources under podman 4.0+ which should switch to netavark.

[aaomidi]

Creating local k8s clusters
Creating registry "ctlptl-registry"...
registry.ctlptl.dev/ctlptl-registry created
reading docker-desktop settings: Get "http://localhost/app/settings": dial unix /Users/amir/Library/Containers/com.docker.docker/Data/backend.sock: connect: no such file or directory
make: *** [Makefile:120: dev-env-cluster] Error 1

Another fun little issue with podman. ctlptl assumes its docker-desktop and tries to do docker-desktop things to it.

[nicks]

@aaomidi ??? what command are you running? we have a lot of integration tests. the machines they run on don't even have docker-desktop installed...so that seems like an unrelated issue...