Controller-Image of develop Branch

[3deep5me]

Hi,

i think i have some trouble because i`m using the develop branch but in my cluster the v2.1 controller runs.
i think because of that i get this error after i try to use the (atm only on develop) secretref field.

The request is invalid: patch: Invalid value: "{\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta1\",\"kind\":\"ProxmoxCluster\",\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/last-applied-configuration\":\"{\\\"apiVersion\\\":\\\"infrastructure.cluster.x-k8s.io/v1beta1\\\",\\\"kind\\\":\\\"ProxmoxCluster\\\",\\\"metadata\\\":{\\\"annotations\\\":{},\\\"name\\\":\\\"proxmoxcluster-sample\\\",\\\"namespace\\\":\\\"default\\\"},\\\"spec\\\":{\\\"controlPlaneEndpoint\\\":{\\\"host\\\":\\\"192.168.2.230\\\",\\\"port\\\":6443},\\\"nodeRefs\\\":[{\\\"name\\\":\\\"mynode\\\",\\\"secretRef\\\":{\\\"name\\\":\\\"pve-mynode-credentials\\\",\\\"namespace\\\":\\\"default\\\"}}],\\\"serverRef\\\":{\\\"endpoint\\\":\\\"192.168.2.142:8006\\\",\\\"secretRef\\\":{\\\"name\\\":\\\"proxmoxcluster-sample\\\",\\\"namespace\\\":\\\"default\\\"}},\\\"storage\\\":{\\\"name\\\":\\\"proxmoxcluster-sample\\\",\\\"path\\\":\\\"\\\"}}}\\n\"},\"creationTimestamp\":\"2023-07-05T11:48:53Z\",\"finalizers\":[\"proxmoxcluster.infrastructure.cluster.x-k8s.io\"],\"generation\":5,\"labels\":{\"cluster.x-k8s.io/cluster-name\":\"proxmoxcluster-sample\"},\"managedFields\":[{\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:kubectl.kubernetes.io/last-applied-configuration\":{}}},\"f:spec\":{\".\":{},\"f:controlPlaneEndpoint\":{\".\":{},\"f:host\":{},\"f:port\":{}},\"f:nodeRefs\":{},\"f:serverRef\":{\".\":{},\"f:credentialsRef\":{\".\":{},\"f:name\":{},\"f:namespace\":{}},\"f:endpoint\":{},\"f:secretRef\":{\".\":{},\"f:name\":{},\"f:namespace\":{}}},\"f:storage\":{\".\":{},\"f:name\":{}}}},\"manager\":\"kubectl-client-side-apply\",\"operation\":\"Update\",\"time\":\"2023-07-05T11:55:48Z\"},{\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:finalizers\":{\".\":{},\"v:\\\"proxmoxcluster.infrastructure.cluster.x-k8s.io\\\"\":{}},\"f:labels\":{\".\":{},\"f:cluster.x-k8s.io/cluster-name\":{}},\"f:ownerReferences\":{\".\":{},\"k:{\\\"uid\\\":\\\"9f2522b4-5f20-4bc8-a643-63fec31c09e1\\\"}\":{}}},\"f:spec\":{\"f:storage\":{\"f:path\":{}}}},\"manager\":\"manager\",\"operation\":\"Update\",\"time\":\"2023-07-05T11:55:48Z\"},{\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:status\":{\".\":{},\"f:ready\":{}}},\"manager\":\"manager\",\"operation\":\"Update\",\"subresource\":\"status\",\"time\":\"2023-07-05T11:55:48Z\"}],\"name\":\"proxmoxcluster-sample\",\"namespace\":\"default\",\"ownerReferences\":[{\"apiVersion\":\"cluster.x-k8s.io/v1beta1\",\"blockOwnerDeletion\":true,\"controller\":true,\"kind\":\"Cluster\",\"name\":\"proxmoxcluster-sample\",\"uid\":\"9f2522b4-5f20-4bc8-a643-63fec31c09e1\"}],\"resourceVersion\":\"3459\",\"uid\":\"742c5954-15b2-49ba-9a19-e58617de4203\"},\"spec\":{\"controlPlaneEndpoint\":{\"host\":\"192.168.2.230\",\"port\":6443},\"nodeRefs\":[{\"name\":\"mynode\",\"secretRef\":{\"name\":\"pve-mynode-credentials\",\"namespace\":\"default\"}}],\"serverRef\":{\"endpoint\":\"192.168.2.142:8006\",\"secretRef\":{\"name\":\"proxmoxcluster-sample\",\"namespace\":\"default\"}},\"storage\":{\"name\":\"proxmoxcluster-sample\",\"path\":\"\"}},\"status\":{\"ready\":true}}": strict decoding error: unknown field "spec.nodeRefs[0].secretRef", unknown field "spec.serverRef.secretRef"

Error from server (BadRequest): error when creating "STDIN": ProxmoxCluster in version "v1beta1" cannot be handled as a ProxmoxCluster: strict decoding error: unknown field "spec.nodeRefs[0].credentialsRef", unknown field "spec.serverRef.credentialsRef"

or vice versa

2023-07-05T13:09:04Z    ERROR   Reconciler error        {"controller": "proxmoxcluster", "controllerGroup": "infrastructure.cluster.x-k8s.io", "controllerKind": "ProxmoxCluster", "ProxmoxCluster": {"name":"my-fancy-cluster","namespace":"default"}, "namespace": "default", "name": "my-fancy-cluster", "reconcileID": "0a465cae-7df2-46e9-a06e-40ec944baad5", "error": "failed to create scope: failed to create proxmox compute client: failed to get proxmox client form nil credentialsRef\ngithub.com/sp-yduck/cluster-api-provider-proxmox/cloud/scope.NewClusterScope\n\t/workspace/cloud/scope/cluster.go:49\ngithub.com/sp-yduck/cluster-api-provider-proxmox/controllers.(*ProxmoxClusterReconciler).Reconcile\n\t/workspace/controllers/proxmoxcluster_controller.go:84\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594", "errorVerbose": "failed to create scope: failed to create proxmox compute client: failed to get proxmox client form nil credentialsRef\ngithub.com/sp-yduck/cluster-api-provider-proxmox/cloud/scope.NewClusterScope\n\t/workspace/cloud/scope/cluster.go:49\ngithub.com/sp-yduck/cluster-api-provider-proxmox/controllers.(*ProxmoxClusterReconciler).Reconcile\n\t/workspace/controllers/proxmoxcluster_controller.go:84\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\ngithub.com/sp-yduck/cluster-api-provider-proxmox/controllers.(*ProxmoxClusterReconciler).Reconcile\n\t/workspace/controllers/proxmoxcluster_controller.go:90\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}

I briefly looked into the make file, but did not find a option to use a local image(?)

How do you work / how would you handle this?

I read something about this https://kind.sigs.k8s.io/docs/user/local-registry/

Another solution which i personally do like more, we/i could do a build for develop branch and push it into a registry but this wouldn't Adresse the problem on all scopes.
(If this is the problem here at all😅)

[sp-yduck]

once you built the image from code (by something make docker-build)
then you can just update the image of controller (e.g. kubectl -n cluster-api-provider-proxmox-system edit deploy cluster-api-provider-proxmox-controller-manager)
how to use local image is completely depends on the Kubernetes provider (kind, minikube, etc).
umm yes maybe we should provide something easier way. Current my idea is using Tilt like many other project so that we can provide a solution from creating management cluster to deploying local image.

btw Another solution which I personally use is just run the code locally (go run ./cmd) this is more easy way (no need to build container image) and it preferable for debugging use case :)

[3deep5me]

Thank you for your input.
Tilt sounds good saw it in a lot of other projects.
Could you go a little bit more into "run the code locally"? I'm new in programming especially go (did the most of the time ops stuff). How can I run the code locally and apply the .Yaml files and establish connection to proxmox etc.?