Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KEDA crashes when using cert-manager certificates and restricted secret access #505

Closed
zerodayyy opened this issue Aug 7, 2023 · 3 comments · Fixed by #518
Closed

KEDA crashes when using cert-manager certificates and restricted secret access #505

zerodayyy opened this issue Aug 7, 2023 · 3 comments · Fixed by #518
Labels
bug Something isn't working

Comments

@zerodayyy
Copy link
Contributor

When enabling restricted secret access feature and using cert-manager instead of self-managed certificates, KEDA operator and metrics-server fail to start.

A possible workaround for this is providing access to own-namespace secrets by creating Role and RoleBinding manually or via extraObjects in chart values.

Expected Behavior

KEDA chart renders necessary Role and RoleBinding under these conditions
OR
KEDA doesn't try to access any secrets

Actual Behavior

KEDA chart doesn't render Role and RoleBinding, leaving no access to secrets at all (which would otherwise be available through ClusterRole). KEDA components try to access secrets in own namespace, failing to do so due to missing access permissions, and crash.

Steps to Reproduce the Problem

  1. Restrict global secret access via following Helm values:
permissions:
  metricServer:
    restrict:
      secret: true
  operator:
    restrict:
      secret: true
  1. Enable cert-manager certificates instead of self-generated ones via following Helm values:
certificates:
  autoGenerated: false
  certManager:
    enabled: true
  1. Deploy KEDA to cluster
  2. KEDA metrics-server and operator crash with following error:
W0807 18:55:16.700808       1 reflector.go:424] k8s.io/client-go/informers/factory.go:150: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
E0807 18:55:16.700852       1 reflector.go:140] k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
W0807 18:55:18.950561       1 reflector.go:424] k8s.io/client-go/informers/factory.go:150: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
E0807 18:55:18.950596       1 reflector.go:140] k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
W0807 18:55:25.059600       1 reflector.go:424] k8s.io/client-go/informers/factory.go:150: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"
E0807 18:55:25.059638       1 reflector.go:140] k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:keda:keda-operator" cannot list resource "secrets" in API group "" in the namespace "keda"

Specifications

  • KEDA Version: 2.11.0
  • Chart Version: 2.11.2
@zerodayyy zerodayyy added the bug Something isn't working label Aug 7, 2023
@JorTurFer
Copy link
Member

I think that this is an error in the chart, because the secret access has to be granted if restricted mode is set.
Are you willing to open a PR fixing it?

@gjacquet
Copy link
Contributor

gjacquet commented Sep 7, 2023

I am facing the same issue and would have time to open PR.
Any pointers?

@gjacquet
Copy link
Contributor

I have opened #518 to attempt to fix this issue.
It seems to work in our test environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants