-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[admission-controller] Helm chart generates duplicate ValidatingWebhookConfiguration resources #1310
Comments
Hi @omaen , as @mavimo said, reason is explained in #1217 . TLDR, we had to create ValidatingWebhookConfiguration in post-install hook to prevent it blocking the deployment of the Admission Controller itself. But using post-install hook, it was not part of the lifecycle, so it wasn't removed on uninstall, causing some issues. The current approach, which works with helm, is creating an empty ValidatingWebhookConfiguration as part of the lifecycle, then adding the real webhook entries on post-install. Now I see this can cause a conflict when rendering as a template, and then creating the resources. I wonder if "helm post-install" does an "apply", so it will update the resource, while ArgoCD + Kustomize might be doing a "create" instead, failing to update the existing resource. I am open to suggestions. We might have a flag to control the behavior of the post-install hook. But please note that creating the ValidatingWebhookConfiguration along with all the other resources and not after the deployments, might cause delays or make the deployment fail, as it will try to validate them with an unreachable service. |
Hi @mavimo and @airadier. Thanks for the explanation! 😃 A bit tricky to set up ValidatingWebhookConfiguration I understand. I'm no expert in either helm og kustomize, so I'll have to take a deeper dive to see if there are some other options that could work as a workaround. Is the admission-controller dependant on other charts, or is it a standalone feature? Right now we are only using it for auditing-purposes, and not actually blocking any k8s actions. An alternative would be to install the admission-controller helm-chart separately from sysdig-deploy after the rest of the sysdig installation. |
Hi @omaen , admission-controller chart does not have any dependencies. You can install it as a subchart of Also, I can think of different workarounds to prevent the failure you describe:
Still, with the second approach you should be careful and try to create the Webhook registrations after deploying all the services. Otherwise might result in long delays while deploying. Regards. |
Hi @airadier, If I'm not mistanken I believe Kustomize only works in a declarative way, and that may be the problem. As the underlying helm-chart produces an output that is dependent on mutating the object. Running
A flag to the helm chart to circumvent the issue should work for us, since ArgoCD will be managing the lifecycle of the resources and do a continuous repair. Although I'm a bit curious to how other helm charts with ValidatingWebhookConfigurations handles this. Have you thought about excluding the namespace where the admission controller is installed from the ValidatingWebhookConfiguration? From the Kubernetes docs on avoiding dead-locks in self hosted webhooks it says
For example by adding something like this to the webhook template:
|
Should be addressed by #1316 . Waiting for approvals and merge |
Hi,
Previously we used terraform to deploy the
sysdig-deploy
helm chart to our cluster, but are now trying to move the setup to ArgoCD using Kustomize. This works for all our other charts, but it seems like theadmission-controller
sub-chart is generating multiple resources with the same name/id, which causes kustomization to fail with the following messageOur kustomization.yaml, and the
values.yaml
has setadmissionController.enabled: true
:Running
helm template sysdig sysdig/sysdig-deploy --values values.yaml
works, but I can see the two resources created.and
I don't know the reason for creating a empty
ValidatingWebhookConfiguration
and then overwriting it. It seemingly works with Helm, but causes problems for Kustomize. Let me know if you want me to open a PR with a suggested solution or not.The text was updated successfully, but these errors were encountered: