diff --git a/charts/admission-controller/Chart.yaml b/charts/admission-controller/Chart.yaml index c5b33efef..3286f5d42 100644 --- a/charts/admission-controller/Chart.yaml +++ b/charts/admission-controller/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: admission-controller description: Sysdig Admission Controller using Sysdig Secure inline image scanner type: application -version: 0.12.1 +version: 0.12.2 appVersion: 3.9.26 home: https://sysdiglabs.github.io/admission-controller/ icon: https://avatars.githubusercontent.com/u/5068817?s=200&v=4 diff --git a/charts/admission-controller/README.md b/charts/admission-controller/README.md index 7112bd412..add124d00 100644 --- a/charts/admission-controller/README.md +++ b/charts/admission-controller/README.md @@ -1,9 +1,9 @@ -# Admission Controller - -[Sysdig Admission Controller](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller/) features ActivityAudit and ImageScanning on a Kubernetes Cluster. -
This chart deploys the Sysdig Admission Controller in your Kubernetes cluster. - -## TL;DR; - -``` -$ helm repo add sysdig https://charts.sysdig.com -$ helm repo update -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller \ - --create-namespace -n sysdig-admission-controller --version=0.12.1 \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN -``` - -- [Configuration](#configuration) -- [Usages](#usages) -- [Confirm Working Status](#confirm-working-status) -- [Troubleshooting](#troubleshooting) - -

-## Introduction +# Admission Controller This chart deploys the Sysdig Admission Controller on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. +## Overview +[Sysdig Admission Controller](https://docs.sysdig.com/en/docs/installation/sysdig-secure/install-agent-components/kubernetes/install-kubernetes-audit-logging/) provides Audit Logging and optional Image Scanning capabilities to secure your Kubernetes environment. -### Prerequisites - -- Helm 3 -- Kubernetes v1.16+ -- Cluster Name (pick one to identify your Kubernetes Cluster) -- Sysdig Secure API Token +Use the [sysdig-deploy](../sysdig-deploy/README.md) parent chart to deploy the Admission Controller and any other subcomponents. Do not deploy subcharts directly. +To deploy the Admission Controller, follow the installation instructions given in [Install Kubernetes Audit Logging](https://docs.sysdig.com/en/docs/installation/sysdig-secure/install-agent-components/kubernetes/install-kubernetes-audit-logging/). +## Use Cases -### Installing the Chart - -To install the chart with the release name `admission-controller`: - -```console -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller -n sysdig-admission-controller --version=0.12.1 -``` +### Kubernetes Audit Logging -The command deploys the Sysdig Admission Controller on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. +This chart is primarily responsible for enabling Kubernetes audit logging so that Sysdig Secure can audit the following: -> **Tip**: List all releases using `helm list` +- Creation and destruction of pods, services, deployments, and DaemonSets. +- Creating, updating, and removing ConfigMaps or secrets. +- Attempts at subscribing to the changes to any endpoint. +For deployment instructions, including common deployment configurations related to proxies and certificates, see [Install Kubernetes Audit Logging](https://docs.sysdig.com/en/docs/installation/sysdig-secure/install-agent-components/kubernetes/install-kubernetes-audit-logging/). +### (Legacy Option) Image Scanning Using Scanning Engine V1 -### Uninstalling the Chart +If you use the [Legacy Scanning Engine](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/) instead of the new Vulnerability Management engine in Sysdig Secure, you can [deploy the `admission-controller` chart](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller/#installation) with old scanning options enabled and use [admission controller policies](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller/#create-admission-controller-policies) to reject container images that do not fulfill the policy requirements from the cluster before being scheduled. -To uninstall/delete the `admission-controller`: +This option is enabled by default unless you specify `--scanner.enabled=false` . -```console -$ helm uninstall sysdig-admission-controller -n sysdig-admission-controller -``` - -The command removes all the Kubernetes components associated with the chart and deletes the release. - -## Configuration - -The following table lists the configurable parameters of the `admission-controller` chart and their default values. - -| Parameter | Description | Default | -|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| global.clusterConfig | Global cluster config options. | {} | -| global.sysdig.secureAPIToken | Global API token to access Sysdig Secure. | "" | -| global.sysdig.secureAPITokenSecret | Global secret with API Token to access Sysdig Secure. | "" | -| global.sysdig.region | Global Sysdig Secure region | "us1" | -| global.proxy | Global HTTP Proxy settings. | {} | -| global.image.pullSecrets | | [] | -| global.image.pullPolicy | | IfNotPresent | -| global.ssl.ca.certs | For outbound connections (secure backend, proxy,...) A PEM-encoded x509 certificate. This can also be a bundle with multiple certificates. | [] | -| global.ssl.ca.keyName | Filename that is used when creating the secret. Required if cert is provided. | | -| global.ssl.ca.existingCaSecret | Provide the name of an existing Secret that contains the CA required | | -| global.ssl.ca.existingCaSecretKeyName | Provide the filename that is defined inside the existing Secret. Required if existingCaSecret is set. | | -| global.ssl.ca.existingCaConfigMap | Provide the name of an existing ConfigMap that contains the CA required | | -| global.ssl.ca.existingCaConfigMapKeyName | Provide the filename that is defined inside the existing ConfigMap. Required if existingCaConfigMap is set. | | -| clusterName | **required**
Cluster Name which appear on Secure UI | "" | -| namespace | Namespace to install components (Optional, will default to release namespace).

IMPORTANT: If a namespace is specified this way it must already exist otherwise installation will fail. | "" | -| sysdig.secureAPIToken | **required**
API Token to access Sysdig Secure.

If neither this value nor `sysdig.existingSecureAPITokenSecret` are configured, the user will be required to provide the deployment the `SECURE_API_TOKEN` (and `AUTH_BEARER_TOKEN` if the scanner is enabled) environment variables. Overrides `global.sysdig.secureAPIToken` | "" | -| sysdig.existingSecureAPITokenSecret | **required**
Existing secret with API Token to access Sysdig Secure
Alternatively, specify the name of a Kubernetes secret containing `SECURE_API_TOKEN` and `AUTH_BEARER_TOKEN` entry if you're also enabling scanner.

If neither this value nor `sysdig.secureAPIToken` are configured, the user will be required to provide the deployment the `SECURE_API_TOKEN` (and `AUTH_BEARER_TOKEN` if the scanner is enabled) environment variables. | "" | -| sysdig.apiEndpoint | Sysdig URL.
- Default secure.sysdig.com is for the us-east region.
- For us-west use us2.app.sysdig.com
- For European Union, use eu1.app.sysdig.com
- For APAC, use app.au1.sysdig.com
- For US4 (our west Google cloud region) use app.us4.sysdig.com
- For on-prem, your own enpoints
| "" | -| features.k8sAuditDetections | Enable K8s Audit detections with Falco rules | true | -| features.k8sAuditDetectionsRules | [Admission Webhook Configuration rules](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-rules) for the Audit Detections | [{"apiGroups":["","apps","autoscaling","batch","networking.k8s.io","rbac.authorization.k8s.io","extensions"],"apiVersions":["*"],"operations":["*"],"resources":["*/*"],"scope":"*"}] | -| verifySSL | For outbound connections (secure backend, proxy,...)
Whether to verify SSL on HTTPS connections | true | -| nameOverride | Chart name override | "" | -| fullnameOverride | Chart full name override | "" | -| labels | Additional labels, applies to both scanner and webhook | {} | -| serviceAccounts.webhook.create | Create the service account | true | -| serviceAccounts.webhook.annotations | Extra annotations for serviceAccount | {} | -| serviceAccounts.webhook.name | Use this value as serviceAccount Name | "" | -| serviceAccounts.scanner.create | Create the service account | true | -| serviceAccounts.scanner.annotations | Extra annotations for serviceAccount | {} | -| serviceAccounts.scanner.name | Use this value as serviceAccount Name | "" | -| podMonitors.webhook.enabled | Enable the webhook PodMonitor to scrape metrics | false | -| podMonitors.webhook.labels | Labels on the webhook PodMonitor | {} | -| podMonitors.webhook.annotations | Annotations on the webhook PodMonitor | {} | -| podMonitors.scanner.enabled | Enable the scanner PodMonitor to scrape metrics | false | -| podMonitors.scanner.labels | Labels on the scanner PodMonitor | {} | -| podMonitors.scanner.annotations | Annotatons on the scanner PodMonitor | {} | -| webhook.name | Service name for Webhook deployment | webhook | -| webhook.replicaCount | Amount of replicas for webhook. **Deprecated**, use `webhook.autoscaling.minReplicas` and `webhook.autoscaling.maxReplicas` instead. | 1 | -| webhook.image.registry | Webhook image registry | quay.io | -| webhook.image.repository | Webhook image registry | sysdig/admission-controller | -| webhook.image.pullPolicy | PullPolicy for Webhook image | | -| webhook.image.tag | Override the default image tag. If not specified, it defaults to appVersion in Chart.yaml | | -| webhook.image.digest | Specify the image digest value. If set, this value is used instead of the tag value. | | -| webhook.labels | Additional labels, applies to webhook only | {} | -| webhook.service.type | Use this type as webhook service | ClusterIP | -| webhook.service.port | Configure port for the webhook service | 443 | -| webhook.rbac.create | Enable the creation of ClusterRoles and the binding of these roles | true | -| webhook.httpProxy | HTTP Proxy settings for webhook.
Set to http(s)://proxyIp:proxyPort in case connection to Sysdig Secure requires a proxy | "" | -| webhook.httpsProxy | HTTPS Proxy settings for webhook.
Set to http(s)://proxyIp:proxyPort in case connection to Sysdig Secure requires a proxy | "" | -| webhook.noProxy | List of hosts, IPs, or IPs in CIDR format that should not go through the proxy. We include "kubernetes" service and typical 10.0.0.0/8 services | kubernetes,10.0.0.0/8 | -| webhook.podAnnotations | Webhook pod annotations. If empty, some annotations are automatically generated for prometheus scraping. | {} | -| webhook.podSecurityContext | Pod Security context for webhook.If empty, some security context are automatically generated. | {} | -| webhook.securityContext | Configure securityContext for webhook. If empty, some security context are automatically generated. | {} | -| webhook.hostNetwork | Specifies if the webhook should be started in hostNetwork mode.
This is required if using a custom CNI where the managed control plane nodes are unable to initiate network connections to the pods, for example using Calico CNI plugin on EKS.
This is not required or recommended in most contexts. | false | -| webhook.imagePullSecrets | The image pull secrets for webhook | [] | -| webhook.resources | Resource request and limits for webhook | {"limits":{"cpu":"250m","memory":"256Mi"},"requests":{"cpu":"100m","memory":"256Mi"}} | -| webhook.autoscaling.enabled | Enable horizontal pod autoscaling | true | -| webhook.autoscaling.minReplicas | Min replicas to use while autoscaling the webhook | 2 | -| webhook.autoscaling.maxReplicas | Max replicas to use while autoscaling the webhook | 5 | -| webhook.autoscaling.targetCPUUtilizationPercentage | Target CPU to use when the number of replicas must be increased | 80 | -| webhook.timeoutSeconds | Number of seconds for the request to time out | 5 | -| webhook.nodeSelector | Configure nodeSelector for scheduling for webhook | {} | -| webhook.priorityClassName | priorityClassName config for the webhook | | -| webhook.tolerations | Tolerations for scheduling for webhook | [] | -| webhook.affinity | Configure affinity rules for webhook | {} | -| webhook.denyOnError | Deny request when an error happened evaluating request | false | -| webhook.dryRun | Dry Run request | false | -| webhook.logLevel | Log Level - Valid Values are: error, info, debug, trace | info | -| webhook.ssl.reuseTLSSecret | Reuse existing TLS Secret during chart upgrade | false | -| webhook.ssl.ca.cert | For outbound connections (secure backend, proxy,...)
A PEM-encoded x509 certificate authority. | "" | -| webhook.ssl.ca.certs | For outbound connections (secure backend, proxy,...) A PEM-encoded x509 certificate. This can also be a bundle with multiple certificates. | [] | -| webhook.ssl.ca.keyName | Filename that is used when creating the secret. Required if cert is provided. | | -| webhook.ssl.ca.existingCaSecret | Provide the name of an existing Secret that contains the CA required | | -| webhook.ssl.ca.existingCaSecretKeyName | Provide the filename that is defined inside the existing Secret. Required if existingCaSecret is set. | | -| webhook.ssl.ca.existingCaConfigMap | Provide the name of an existing ConfigMap that contains the CA required | | -| webhook.ssl.ca.existingCaConfigMapKeyName | Provide the filename that is defined inside the existing ConfigMap. Required if existingCaConfigMap is set. | | -| webhook.customEntryPoint | Custom entrypoint for the webhook
Remember to provide the webhook valid arguments with `--tls_cert_file` and `--tls_private_key_file`.
default: /bin/webhook --tls_cert_file /cert/tls.crt --tls_private_key_file /cert/tls.key | [] | -| webhook.http.port | HTTP serve port where the requests will be served from | 5000 | -| scc.create | Enable the creation of Security Context Constraints in Openshift | true | -| scanner.enabled | If you only want the Kubernetes Audit Log functionality then disable this, and it will disable the Admission Controller Scanning Policy functionality. | true | -| scanner.name | Service name for Scanner deployment | scanner | -| scanner.replicaCount | Amount of replicas for scanner | 1 | -| scanner.image.registry | Scanner image registry | quay.io | -| scanner.image.repository | Scanner image repository | sysdig/inline-scan-service | -| scanner.image.pullPolicy | PullPolicy for Scanner image | | -| scanner.image.tag | Scanner image tag | 0.0.13 | -| scanner.image.digest | Specify the image digest value. If set, this value is used instead of the tag value. | | -| scanner.labels | Additional labels, applies to scanner only | {} | -| scanner.service.port | Configure port for the webhook service | 8080 | -| scanner.authWithSecureToken | Authenticate with Secure token | false | -| scanner.httpProxy | HTTP Proxy settings for scanner.
Set to http(s)://proxyIp:proxyPort in case connection to Sysdig Secure requires a proxy | "" | -| scanner.httpsProxy | HTTPS Proxy settings for scanner.
Set to http(s)://proxyIp:proxyPort in case connection to Sysdig Secure requires a proxy | "" | -| scanner.noProxy | List of hosts, IPs, or IPs in CIDR format that should not go through the proxy. We include "kubernetes" service and typical 10.0.0.0/8 services | kubernetes,10.0.0.0/8 | -| scanner.podAnnotations | Scanner pod annotations | {"prometheus.io/path":"/metrics","prometheus.io/port":"8080","prometheus.io/scrape":"true"} | -| scanner.psp.create | Whether to create a psp policy and role / role-binding | false | -| scanner.podSecurityContext | PSP's for scanner | {} | -| scanner.verifyRegistryTLS | Verify TLS on image pull from registries | true | -| scanner.dockerCfgSecretName | Docker config secret. Use a provided secret containing a .dockercfg for registry authentication (i.e. Openshift internal registry) | "" | -| scanner.securityContext | Configure securityContext for scanner | {"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsNonRoot":true} | -| scanner.imagePullSecrets | The image pull secrets for scanner | [] | -| scanner.resources | Resource requests and limits for scanner | {} | -| scanner.nodeSelector | Configure nodeSelector for scheduling for the scanner | {} | -| scanner.priorityClassName | priorityClassName config for the scanner | | -| scanner.tolerations | Tolerations for scheduling for the scanner | [] | -| scanner.affinity | Configure affinity rules for the scanner | {} | -| scanner.ssl.ca.cert | For outbound connections (secure backend, proxy,...).
A PEM-encoded x509 certificate authority. | "" | -| scanner.ssl.ca.certs | For outbound connections (secure backend, proxy,...) A PEM-encoded x509 certificate. This can also be a bundle with multiple certificates. | [] | -| scanner.ssl.ca.keyName | Filename that is used when creating the secret. Required if cert is provided. | | -| scanner.ssl.ca.existingCaSecret | Provide the name of an existing Secret that contains the CA required | | -| scanner.ssl.ca.existingCaSecretKeyName | Provide the filename that is defined inside the existing Secret. Required if existingCaSecret is set. | | -| scanner.ssl.ca.existingCaConfigMap | Provide the name of an existing ConfigMap that contains the CA required | | -| scanner.ssl.ca.existingCaConfigMapKeyName | Provide the filename that is defined inside the existing ConfigMap. Required if existingCaConfigMap is set. | | -| scanner.customEntryPoint | Custom entrypoint for the scanner.
Remember to provide the scanner valid arguments with `--server_port` and optionally `--auth_secure_token`
default: /inline-scan-service --server_port=8080 | [] | - - -Specify each parameter using the **`--set key=value[,key=value]`** argument to `helm upgrade --install`. For example: - -```console -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller \ - --create-namespace -n sysdig-admission-controller --version=0.12.1 \ - --set sysdig.secureAPIToken=YOUR-KEY-HERE,clusterName=YOUR-CLUSTER-NAME -``` - -**Alternatively, a YAML file** that specifies the values for the parameters can be provided while -installing the chart. For example: - -```console -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller \ - --create-namespace -n sysdig-admission-controller --version=0.12.1 \ - --values values.yaml -``` - -### Verify the integrity and origin -Sysdig Helm Charts are signed so users can verify the integrity and origin of each chart, the steps are as follows: - -#### Import the Public Key - -```console -$ curl -o "/tmp/sysdig_public.gpg" "https://charts.sysdig.com/public.gpg" -$ gpg --import /tmp/sysdig_public.gpg -``` - -#### Verify the chart - -To check the integrity and the origin of the charts you can now append the `--verify` flag to the `install`, `upgrade` and `pull` helm commands. - -## Examples -- [Default `values.yaml`](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/values.yaml) -- Find some [examples of these values](https://github.com/sysdiglabs/charts/tree/master/charts/admission-controller/ci) - -### Custom Admission Controller Rules to be detected - -In case you don't want to detect some resources you can create your custom rules. - -To achieve this, you can change the **k8sAuditDetectionsRules** variable in the [values.yaml](./values.yaml) file. -For example, if you want to filter out secrets from the AC you can try with these rules: - -``` -- apiGroups: - - "" - apiVersions: [ "*" ] - operations: [ "*" ] - resources: - - bindings - - componentstatuses - - configmaps - - endpoints - - events - - limitranges - - namespaces - - nodes - - persistentvolumeclaims - - persistentvolumes - - pods/* - - podtemplates - - replicationcontrollers - - resourcequotas - - serviceaccounts - - services - scope: "*" -- apiGroups: - - apps - - autoscaling - - batch - - networking.k8s.io - - rbac.authorization.k8s.io - - extensions - apiVersions: [ "*" ] - operations: [ "*" ] - resources: [ "*/*" ] - scope: "*" -``` - -### Proxy Usage - -There are several configuration parameters for the proxy usage - -- Two involved components `webhook.*` and `scanner.*`; reference to the first for communications to the Sysdig backend, while -second communicates with the registry from where to pull the image to be scanned. -- configuration values `*.httpProxy`, `*.httpsProxy` and `*.noProxy`. Make sure to use at least `https` version for Sysdig Secure Backend. - -If your Proxy is served with TLS -- The url for those `*.httpProxy` and `*.httpsProxy` must be `https://` -- If using a self-signed certificate you will need to also configure one of the following two options - 1. Set the `verifySSL=false` parameter - 2. Or set `*.ssl.ca.cert` for both components `webhook` and `scanner` - -## Usages - - -### Basic - -``` -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller \ - --create-namespace -n sysdig-admission-controller \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN -``` +## Verify the Integrity and Origin +Sysdig Helm charts are signed so you can confirm the integrity and origin of each chart. To do so: -### On Prem +1. Import the Public Key: -Sysdig On-Prem installations might use a TLS self-signed server certificate or one from an untrusted CA, so it requires an extra configuration. + ```bash + $ curl -o "/tmp/sysdig_public.gpg" "https://charts.sysdig.com/public.gpg" + $ gpg --import /tmp/sysdig_public.gpg + ``` -#### Ignore TLS certificate verification +2. Verify the chart by appending the `--verify` flag to the `install`, `upgrade`, and `pull` helm commands. -Use the following command to deploy in an on-prem and ignore the untrusted certificate using `verifySSL=false`: -``` -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller \ - --create-namespace -n sysdig-admission-controller \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN \ - --set verifySSL=false -``` - -#### Custom CA Provided - -The following command will deploy the admission controller with a custom CA. The custom CA certificate is added to the trusted certificates store. - -``` -$ helm upgrade --install sysdig-admission-controller sysdig/admission-controller \ - --create-namespace -n sysdig-admission-controller \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN \ - --set webhook.ssl.ca.cert=YOUR_CA_CERT_AS_PEM_ENCODED \ - --set scanner.ssl.ca.cert=YOUR_CA_CERT_AS_PEM_ENCODED -``` - -## Confirm Working Status - - -### Activity Audit - -1. Install Admission Controller on your Kubernetes Cluster following one of the (use-cases)(#usage) described - - This feature is enabled by default through `features.k8sAuditDetections` value -2. Check your current "Kubernetes Audit" policies in `Sysdig Secure > Policies > Threat Detection | Runtime Policies` as we will be triggering one of those to prove it's working correctly. - - We suggest using "Create Privileged Pod" but you can choose any. -3. If possible, let's activate just installed component logs to have them at sight - ``` - $ kubectl logs -f -n sysdig-admission-controller -l app.kubernetes.io/component=webhook - ``` -4. Trigger following command to force an unwanted audit detection - ``` - $ kubectl run nginx --image nginx --privileged - ``` -5. If you had a chance to activate logs, take a look at them. You should see something like this - ``` - {"level":"info","component":"console-notifier","message":"Pod started with privileged container (user=** pod=nginx ns=default images=nginx)"} - ``` -6. Confirm that event reached Sysdig Secure, looking at `Events` - - - -### Image Scanning - -1. Install Admission Controller on your Kubernetes Cluster following one of the (use-cases)(#usage) described - - In the chart, this feature is enabled by default through `scanner.enabled` value -2. Enable Admission-Controller on your Sysdig Secure > Image Scanning > Admission Controller > Policy Assignments -This section can only be accessed by a user with Administrator permissions -3. Add some an assignment to Allow or Deny images within a namespace -4. Tail to the logs from the Admission Controller - ``` - $ kubectl logs -f -n sysdig-admission-controller -l app.kubernetes.io/component=webhook - ``` -5. Push some deployment into your Kubernetes Cluster to watch the result, for example an nginx image - ``` - $ kubectl run nginx --image=nginx - ``` - -If policy is set to allow, the deployment will be successful. - -Either way, you should see some logs in Admission Controller tail - - -- allow assignment result - {"level":"info","component":"scanning-evaluator","message":"checking pod=nginx in namespace=default"} - {"level":"info","component":"scanning-evaluator","message":"evaluating container with name=nginx and image=nginx"} - {"level":"info","component":"scanning-evaluator","time":"","message":"matched policy=Allow always for namespace=default and image=nginx"} - {"level":"info","component":"scanning-evaluator","message":"allowing container with name=nginx and image=nginx"} - - -- reject assignment result - {"level":"info","component":"scanning-evaluator","message":"checking pod=nginx in namespace=default"} - {"level":"info","component":"scanning-evaluator","message":"evaluating container with name=nginx and image=nginx"} - {"level":"info","component":"scanning-evaluator","message":"matched policy=Reject Allways for namespace=default and image=nginx"} - {"level":"info","component":"scanning-evaluator","message":"denying container with name=nginx and image=nginx reason=\"Reject Always\""} - -## Running helm unit tests - -The sysdiglabs/charts repository uses the following helm unittest plugin: https://github.com/quintush/helm-unittest - -You can test the changes to your chart by running the test suites as follows: - -``` -helm unittest --helm3 . -``` - -The helm unit tests are in the tests folder. It is recommended to add new tests as new features are added here. - -## Troubleshooting - - -### Q: I'm not able to get an alert for an event with the `ka.verb=get` condition. - -A: Despite [Kubernetes Extensible Admission Controller webhook allows it](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-rules), Sysdig Admission Controller does only handle `CREATE`, `UPDATE`, `DELETE` and `CONNECT` type of events. -Also, beware Kubernetes [`apiGroups`](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/values.yaml#L41-L54) are scoped - -S: Still, if required, you can make use of the [legacy Sysdig Kubernetes Audit Log](https://docs.sysdig.com/en/docs/sysdig-secure/secure-events/kubernetes-audit-logging/#legacy-installation-instructions) which do support more verbs. - - +## Configuration -### Q: I get tons of "TLS handshake error" +### Using the Key-Value Pair -A: This happens when DEBUG is enabled but Admission Controller will behave as expected. Those calls are some non-sysdig direct calls to the Admission Controller without TLS, which raises this informational log by Go internal library. +Specify each parameter using the `--set key=value[,key=value]` argument to the `helm install`command. +For example: -### Q: I need to troubleshoot, any way to switch to `debug` verbose? -A: If you used helm to install, you can edit the helm `values.yaml` to set `webhook.logLevel=debug` -
Alternatively, you can edit the webhook configmap - add the `LOG_LEVEL=debug` key-value and restart the webhook -``` - $ kubectl edit configmaps -n sysdig-admission-controller sysdig-admission-controller-webhook - $ kubectl rollout restart deployment -n sysdig-admission-controller sysdig-admission-controller-webhook +```bash +helm upgrade --install admission-controller sysdig/admission-controller \ + --create-namespace -n sysdig-admission-controller --version=0.12.2 \ + --set sysdig.secureAPIToken=YOUR-KEY-HERE,clusterName=YOUR-CLUSTER-NAME ``` -### Q: I don't see `Policy Rules` honored -S: Review the [Admission Controller - Understanding:How Policy Conditions are applied](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller//#understanding-how-policy-conditions-are-applied) - -### Q: I don't see `Policy Assignments` order honored -A: It may be that you're using same namespace and image prefix on more than one assignment
-S: Review the [Admission Controller - Understanding:Evaluation Order](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller//#understanding-evaluation-order) - -### Q: I don't see changes on `Policy Assignments` being applied on my cluster -A: Admission Controller pull changes from the Sysdig Secure platform every 5 minutes
-S: You can wait those five minutes, or force the admission controller webhook restart +### Using values.yaml - $ kubectl rollout restart deployment -n sysdig-admission-controller sysdig-admission-controller-webhook +The `values.yaml` file specifies the values for the admission controller configuration parameters. You can add the configuration to the `values.yaml` file, then use it in the `helm install` command. -### Q: I am deploying it in a GKE Cluster, with Private Network enabled, and everything is slow or I cannot scale the cluster correctly. +For example: -```text -"Failed calling webhook, failing open audit.secure.sysdig.com: failed calling webhook "audit.secure.sysdig.com": Post "https://sysdig-ac-webhook.sysdig-agent.svc:443/k8s-audit?timeout=10s ": context canceled" -``` - -A: GKE clusters run the K8s API outside from the cluster. If Private Network is enabled, the K8s API may be unable to reach the Admission Controller's webhook that validates each API request, so eventually every API request times out and is processed, but the performance is impacted in the process. -

S: As specified in [GKE Private Cluster Webhook Timeouts](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#api_request_that_triggers_admission_webhook_timing_out), the default firewall configuration does not allow TCP connections for ports other than 443 and 10250. -Admission Controller's webhook run on `5000 TCP port`, so you need to enable a new rule that allows the Control Plane's network to access it. -
Follow the instructions in [GKE-Adding firewall rules to cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#add_firewall_rules) to enable inbound connections to our webhook. - - -### Q: Getting "error getting the cluster id from kubernetes: open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied" +```bash +helm upgrade --install admission-controller sysdig/admission-controller \ + --create-namespace -n sysdig-admission-controller --version=0.12.2 \ + --values values.yaml -A: Some users (old versions of GKE) reported that the permissions to access serviceAccount token, mounted in the filesystem, was set to [`0600` permissions](https://discuss.hashicorp.com/t/wrong-permission-on-being-set-on-serviceaccount-token/28777), not allowing the pods to actually read from it. -

S: [Recommend](https://github.com/kubernetes/kubernetes/issues/82573) to change the `securityContext.fsGroup` to the value `65534` on the pod. -
You can specify this through our helm chart with the parameter -``` ---set webhook.podSecurityContext.fsGroup=65534 ``` -### Q: Getting readiness probe errors and cannot startup +See the default [`values.yaml`](./values.yaml) file for more information. -``` -13m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/sysdig-admission-controller-webhook invalid metrics (1 invalid out of 1), first error is: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io) -``` -A: [HorizontalAutoScaller](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/templates/webhook/autoscaler.yaml) requires your kubernetes cluster to be able to use [metrics API](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-metrics-apis), which in some lightweight installations, such as minikube, must be enabled through a plugin -
S: For minikube, enable `metric-server` plugin -``` - $ minikube addons list | grep metrics-server - $ minikube addons enable metrics-server - ``` +## Configuration Parameters -### Q: Getting error "x509: certificate signed by unknown authority" +The following table lists the configurable parameters of the `admission-controller` chart and their default values. -A: Sysdig installation is made with an unverfied certificate, such as self-signed, `SECURE_URL` being `https` -
S: Add `--set verifySSL=false` to your installation parameters or configure a trusted CA certificate +| Parameter | Description | Default | +|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| global.clusterConfig | The global cluster configuration options. | {} | +| global.sysdig.secureAPIToken | The global API token to access Sysdig Secure. | "" | +| global.sysdig.secureAPITokenSecret | The global secret with API Token to access Sysdig Secure. | "" | +| global.sysdig.region | The global Sysdig Secure region. | "us1" | +| global.proxy | Global HTTP Proxy settings. | {} | +| global.image.pullSecrets | | [] | +| global.image.pullPolicy | | IfNotPresent | +| global.ssl.ca.certs | For outbound connections (secure backend, proxy,...) A PEM-encoded x509 certificate. This can also be a bundle with multiple certificates. | [] | +| global.ssl.ca.keyName | Filename that is used when creating the secret. Required if cert is provided. | | +| global.ssl.ca.existingCaSecret | Provide the name of an existing Secret that contains the CA required | | +| global.ssl.ca.existingCaSecretKeyName | Provide the filename that is defined inside the existing Secret. Required if existingCaSecret is set. | | +| global.ssl.ca.existingCaConfigMap | Provide the name of an existing ConfigMap that contains the CA required | | +| global.ssl.ca.existingCaConfigMapKeyName | Provide the filename that is defined inside the existing ConfigMap. Required if existingCaConfigMap is set. | | +| clusterName | **required**
The cluster Name which appear on Secure UI | "" | +| namespace | The namespace to install components. An optional field. If not specified, it will default to the release namespace.
IMPORTANT: Ensure that a namespace is already exist, otherwise installation will fail. | "" | +| sysdig.secureAPIToken | **required**
The API Token to access Sysdig Secure.

If neither this value nor `sysdig.existingSecureAPITokenSecret` is configured, you are required to provide the deployment with the `SECURE_API_TOKEN` (and `AUTH_BEARER_TOKEN` if the scanner is enabled) environment variable. Overrides the `global.sysdig.secureAPIToken` parameter. | "" | +| sysdig.existingSecureAPITokenSecret | **required**
TheeExisting secret with API Token to access Sysdig Secure.
Alternatively, specify the name of a Kubernetes secret containing `SECURE_API_TOKEN` and `AUTH_BEARER_TOKEN` entry if you're also enabling scanner.
If neither this value nor `sysdig.secureAPIToken` is configured, you are required to provide the deployment with the `SECURE_API_TOKEN` (and `AUTH_BEARER_TOKEN` if the scanner is enabled) environment variable. | "" | +| sysdig.apiEndpoint | Sysdig URL.
- The default for the us-east region is secure.sysdig.com.
- For us-west use us2.app.sysdig.com
- For European Union, use eu1.app.sysdig.com
- For APAC, use app.au1.sysdig.com
- For US4 (our west Google cloud region) use app.us4.sysdig.com
- For on-prem, your own enpoints
| "" | +| features.k8sAuditDetections | Enable Kubernetes Audit detections with Falco rules. | true | +| features.k8sAuditDetectionsRules | [Admission Webhook Configuration rules](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-rules) for the Audit Detections | [{"apiGroups":["","apps","autoscaling","batch","networking.k8s.io","rbac.authorization.k8s.io","extensions"],"apiVersions":["*"],"operations":["*"],"resources":["*/*"],"scope":"*"}] | +| verifySSL | Used for outbound connections, such as Secure backend and proxy.
Specifies whether to verify SSL on HTTPS connections. | true | +| nameOverride | The chart name override. | "" | +| fullnameOverride | The chart full name override. | "" | +| labels | Additional labels. It applies to both scanner and webhook. | {} | +| serviceAccounts.webhook.create | Creates the service account. | true | +| serviceAccounts.webhook.annotations | The additional annotations for serviceAccount. | {} | +| serviceAccounts.webhook.name | Use this value as serviceAccount Name. | "" | +| serviceAccounts.scanner.create | Creates the service account. | true | +| serviceAccounts.scanner.annotations | The additional annotations for serviceAccount. | {} | +| serviceAccounts.scanner.name | Use this value as serviceAccount Name. | "" | +| podMonitors.webhook.enabled | Enable the webhook PodMonitor to scrape metrics. | false | +| podMonitors.webhook.labels | Specifies the labels on the webhook PodMonitor. | {} | +| podMonitors.webhook.annotations | The annotations on the webhook PodMonitor. | {} | +| podMonitors.scanner.enabled | Enable the scanner PodMonitor to scrape metrics. | false | +| podMonitors.scanner.labels | Specifies the labels on the scanner PodMonitor. | {} | +| podMonitors.scanner.annotations | The annotatons on the scanner PodMonitor | {} | +| webhook.name | The service name for Webhook deployment | webhook | +| webhook.replicaCount | The number of replicas for webhook. **Deprecated**, use `webhook.autoscaling.minReplicas` and `webhook.autoscaling.maxReplicas` instead. | 1 | +| webhook.image.registry | The webhook image registry. | quay.io | +| webhook.image.repository | The webhook image registry. | sysdig/admission-controller | +| webhook.image.pullPolicy | The PullPolicy for Webhook image. | | +| webhook.image.tag | Overrides the default image tag. If not specified, it defaults to appVersion in Chart.yaml | | +| webhook.image.digest | Specifies the image digest value. If set, this value is used instead of the tag value. | | +| webhook.labels | Specifies the additional labels; applies to webhook only. | {} | +| webhook.service.type | Use this type as webhook service. | ClusterIP | +| webhook.service.port | Configure port for the webhook service. | 443 | +| webhook.rbac.create | Enable the creation of ClusterRoles and the binding of these roles. | true | +| webhook.httpProxy | The HTTP Proxy settings for webhook.
Set to http(s)://proxyIp:proxyPort if the connections to Sysdig Secure requires a proxy. | "" | +| webhook.httpsProxy | The HTTPS Proxy settings for webhook.
Set to http(s)://proxyIp:proxyPort if the connection to Sysdig Secure requires a proxy. | "" | +| webhook.noProxy | List of hosts, IPs, or IPs in CIDR format that should not go through the proxy. Sysdig includes "kubernetes" service and typical 10.0.0.0/8 services. | kubernetes,10.0.0.0/8 | +| webhook.podAnnotations | The webhook pod annotations. If empty, some annotations are automatically generated for prometheus scraping. | {} | +| webhook.podSecurityContext | The Pod Security context for webhook.If empty, some security context are automatically generated. | {} | +| webhook.securityContext | Configure securityContext for webhook. If empty, some security context are automatically generated. | {} | +| webhook.hostNetwork | Specifies if the webhook should be started in hostNetwork mode.
This field is required if you are using a custom CNI where the managed control plane nodes are unable to initiate network connections to the pods, for example, using Calico CNI plugin on EKS.
This is not required or recommended in most contexts. | false | +| webhook.imagePullSecrets | The image pull secrets for webhook. | [] | +| webhook.resources | Resource request and limits for webhook. | {"limits":{"cpu":"250m","memory":"256Mi"},"requests":{"cpu":"100m","memory":"256Mi"}} | +| webhook.autoscaling.minReplicas | The minimum replicas to use while autoscaling the webhook. | 2 | +| webhook.autoscaling.maxReplicas | The maximum replicas to use while autoscaling the webhook. | 5 | +| webhook.autoscaling.targetCPUUtilizationPercentage | The target CPU to use when the number of replicas must be increased. | 80 | +| webhook.timeoutSeconds | The number of seconds for the request to time out. | 5 | +| webhook.nodeSelector | Configure nodeSelector for scheduling for webhook. | {} | +| webhook.priorityClassName | The priorityClassName configuration for the webhook. | | +| webhook.tolerations | Tolerations for scheduling for webhook. | [] | +| webhook.affinity | Configure affinity rules for webhook. | {} | +| webhook.denyOnError | Deny request when an error happened evaluating request. | false | +| webhook.dryRun | Dry Run request | false | +| webhook.logLevel | Specifies the log level. The valid values are error, info, debug, trace. | info | +| webhook.ssl.reuseTLSSecret | Reuse existing TLS Secret during chart upgrade. | false | +| webhook.ssl.ca.cert | Used for outbound connections, such as Secure backend and proxy.
Used also for inbound connections to serve HttpRequests as Kubernetes Webhook.
A PEM-encoded x509 certificate authority. | "" | +| webhook.ssl.ca.certs | For outbound connections (secure backend, proxy,...) A PEM-encoded x509 certificate. This can also be a bundle with multiple certificates. | [] | +| webhook.ssl.ca.keyName | Filename that is used when creating the secret. Required if cert is provided. | | +| webhook.ssl.ca.existingCaSecret | Provide the name of an existing Secret that contains the CA required | | +| webhook.ssl.ca.existingCaSecretKeyName | Provide the filename that is defined inside the existing Secret. Required if existingCaSecret is set. | | +| webhook.ssl.ca.existingCaConfigMap | Provide the name of an existing ConfigMap that contains the CA required | | +| webhook.ssl.ca.existingCaConfigMapKeyName | Provide the filename that is defined inside the existing ConfigMap. Required if existingCaConfigMap is set. | | +| webhook.customEntryPoint | The custom entrypoint for the webhook
Remember to provide the webhook valid arguments with `--tls_cert_file` and `--tls_private_key_file`.
default: `/bin/webhook --tls_cert_file /cert/tls.crt --tls_private_key_file /cert/tls.key` | [] | +| webhook.http.port | The HTTP serve port where the requests will be served from. | 5000 | +| scc.create | Enable the creation of Security Context Constraints in Openshift. | true | +| scanner.enabled | If you only want the Kubernetes Audit Log functionality then disable this option and it will disable the Admission Controller Scanning Policy functionality. | true | +| scanner.name | The service name for Scanner deployment. | scanner | +| scanner.replicaCount | The amount of replicas for scanner. | 1 | +| scanner.image.registry | The Scanner image registry. | quay.io | +| scanner.image.repository | The Scanner image repository. | sysdig/inline-scan-service | +| scanner.image.pullPolicy | The PullPolicy for Scanner image. | | +| scanner.image.tag | The Scanner image tag. | 0.0.13 | +| scanner.image.digest | Specify the image digest value. If set, this value is used instead of the tag value. | | +| scanner.labels | Specifies additional labels. It applies to Scanner only. | {} | +| scanner.service.port | Configure port for the webhook service. | 8080 | +| scanner.authWithSecureToken | Authenticate with Secure token. | false | +| scanner.httpProxy | The HTTP Proxy settings for Scanner.
Set to `http(s)://proxyIp:proxyPort` if the connection to Sysdig Secure requires a proxy. | "" | +| scanner.httpsProxy | The HTTPS Proxy settings for Scanner.
Set to `http(s)://proxyIp:proxyPort` if connection to Sysdig Secure requires a proxy. | "" | +| scanner.noProxy | Specifies the list of hosts, IPs, or IPs in CIDR format that should not go through the proxy. Sysdig includes "kubernetes" service and typical 10.0.0.0/8 services. | kubernetes,10.0.0.0/8 | +| scanner.podAnnotations | Specifies the Scanner pod annotations. | {"prometheus.io/path":"/metrics","prometheus.io/port":"8080","prometheus.io/scrape":"true"} | +| scanner.psp.create | Specifies whether to create a psp policy and role / role-binding. | false | +| scanner.podSecurityContext | The PSPs for scanner | {} | +| scanner.verifyRegistryTLS | Verify the TLS on image pull from registries. | true | +| scanner.dockerCfgSecretName | The Docker config secret. Use a provided secret containing a `.dockercfg` for registry authentication (i.e. Openshift internal registry). | "" | +| scanner.securityContext | Configure securityContext for scanner. | {"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsNonRoot":true} | +| scanner.imagePullSecrets | The image pull secrets for Scanner. | [] | +| scanner.resources | Specifies resource requests and limits for Scanner. | {} | +| scanner.nodeSelector | Configure nodeSelector for scheduling for the Scanner. | {} | +| scanner.priorityClassName | Specifies the priorityClassName configuration for the Scanner. | | +| scanner.tolerations | Specifies the sheduling tolerations for the Scanner. | [] | +| scanner.affinity | Configure affinity rules for the Scanner. | {} | +| scanner.ssl.ca.cert | For outbound connections, such as Secure backend and proxy.
A PEM-encoded x509 certificate authority. | "" | +| scanner.ssl.ca.certs | For outbound connections, for example, the Secure backend and proxy. A PEM-encoded x509 certificate. This can also be a bundle with multiple certificates. | [] | +| scanner.ssl.ca.keyName | A filename that is used when creating the secret. Required if cert is provided. | | +| scanner.ssl.ca.existingCaSecret | Provide the name of an existing Secret that contains the CA required | | +| scanner.ssl.ca.existingCaSecretKeyName | Provide the filename that is defined inside the existing Secret. Required if existingCaSecret is set. | | +| scanner.ssl.ca.existingCaConfigMap | Provide the name of an existing ConfigMap that contains the CA required | | +| scanner.ssl.ca.existingCaConfigMapKeyName | Provide the filename that is defined inside the existing ConfigMap. Required if existingCaConfigMap is set. | | +| scanner.customEntryPoint | Custom entrypoint for the scanner.
Remember to provide the scanner valid arguments with `--server_port` and optionally `--auth_secure_token`
default: `/inline-scan-service --server_port=8080` | [] | -### Q: Why is there no support for `ka.sourceips`? -AdmissionController is unable to retrieve the source IP of the events, because this information is not provided by the [Kubernetes AdmissionReview](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#request). -If you really require this field, as a workaround, you can use the legacy [Sysdig Agent + Kubernetes Audit](https://docs.sysdig.com/en/docs/sysdig-secure/secure-events/kubernetes-audit-logging/#legacy-installation-instructions) +## Examples +- [Default `values.yaml`](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/values.yaml) +- Find some [examples of these values](https://github.com/sysdiglabs/charts/tree/master/charts/admission-controller/ci) -# Admission Controller - -[{{ .Project.Name }}]({{ .Project.URL }}) features ActivityAudit and ImageScanning on a Kubernetes Cluster. -
{{ .Project.Description }} - -## TL;DR; - -``` -$ helm repo add {{ .Repository.Name }} {{ .Repository.URL }} -$ helm repo update -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ - --create-namespace -n {{ .Release.Namespace }}{{ with .Chart.Version }} --version={{.}} {{ end }} \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN -``` - -- [Configuration](#configuration) -- [Usages](#usages) -- [Confirm Working Status](#confirm-working-status) -- [Troubleshooting](#troubleshooting) - -

-## Introduction +# Admission Controller This chart deploys {{ .Project.App }} on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. +## Overview +[{{ .Project.Name }}]({{ .Project.URL }}) provides Audit Logging and optional Image Scanning capabilities to secure your Kubernetes environment. -### Prerequisites -{{ range .Prerequisites }} -- {{ . }} -{{- end }} - - - -### Installing the Chart - -To install the chart with the release name `{{ .Release.Name }}`: +Use the [sysdig-deploy](../sysdig-deploy/README.md) parent chart to deploy the Admission Controller and any other subcomponents. Do not deploy subcharts directly. -```console -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} -n {{ .Release.Namespace }}{{ with .Chart.Version }} --version={{.}}{{ end }} -``` - -The command deploys {{ .Project.App }} on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. - -> **Tip**: List all releases using `helm list` - - - -### Uninstalling the Chart - -To uninstall/delete the `{{ .Release.Name }}`: +To deploy the Admission Controller, follow the installation instructions given in [Install Kubernetes Audit Logging](https://docs.sysdig.com/en/docs/installation/sysdig-secure/install-agent-components/kubernetes/install-kubernetes-audit-logging/). -```console -$ helm uninstall sysdig-{{ .Release.Name }} -n {{ .Release.Namespace }} -``` +## Use Cases -The command removes all the Kubernetes components associated with the chart and deletes the release. +### Kubernetes Audit Logging -{{ if .Chart.Values -}} +This chart is primarily responsible for enabling Kubernetes audit logging so that Sysdig Secure can audit the following: -## Configuration +- Creation and destruction of pods, services, deployments, and DaemonSets. +- Creating, updating, and removing ConfigMaps or secrets. +- Attempts at subscribing to the changes to any endpoint. -The following table lists the configurable parameters of the `{{ .Chart.Name }}` chart and their default values. +For deployment instructions, including common deployment configurations related to proxies and certificates, see [Install Kubernetes Audit Logging](https://docs.sysdig.com/en/docs/installation/sysdig-secure/install-agent-components/kubernetes/install-kubernetes-audit-logging/). -{{ .Chart.Values }} +### (Legacy Option) Image Scanning Using Scanning Engine V1 -Specify each parameter using the **`--set key=value[,key=value]`** argument to `helm upgrade --install`. For example: +If you use the [Legacy Scanning Engine](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/) instead of the new Vulnerability Management engine in Sysdig Secure, you can [deploy the `admission-controller` chart](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller/#installation) with old scanning options enabled and use [admission controller policies](https://docs.sysdig.com/en/docs/sysdig-secure/scanning/admission-controller/#create-admission-controller-policies) to reject container images that do not fulfill the policy requirements from the cluster before being scheduled. -```console -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ - --create-namespace -n {{ .Release.Namespace }}{{ with .Chart.Version }} --version={{.}}{{ end }} \ - --set {{ .Chart.ValuesExample }} -``` +This option is enabled by default unless you specify `--scanner.enabled=false` . -**Alternatively, a YAML file** that specifies the values for the parameters can be provided while -installing the chart. For example: -```console -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ - --create-namespace -n {{ .Release.Namespace }}{{ with .Chart.Version }} --version={{.}}{{ end }} \ - --values values.yaml -``` - -### Verify the integrity and origin -Sysdig Helm Charts are signed so users can verify the integrity and origin of each chart, the steps are as follows: - -#### Import the Public Key - -```console -$ curl -o "/tmp/sysdig_public.gpg" "https://charts.sysdig.com/public.gpg" -$ gpg --import /tmp/sysdig_public.gpg -``` - -#### Verify the chart - -To check the integrity and the origin of the charts you can now append the `--verify` flag to the `install`, `upgrade` and `pull` helm commands. - -## Examples -- [Default `values.yaml`](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/values.yaml) -- Find some [examples of these values](https://github.com/sysdiglabs/charts/tree/master/charts/admission-controller/ci) - -{{- end }} - -### Custom Admission Controller Rules to be detected - -In case you don't want to detect some resources you can create your custom rules. - -To achieve this, you can change the **k8sAuditDetectionsRules** variable in the [values.yaml](./values.yaml) file. -For example, if you want to filter out secrets from the AC you can try with these rules: - -``` -- apiGroups: - - "" - apiVersions: [ "*" ] - operations: [ "*" ] - resources: - - bindings - - componentstatuses - - configmaps - - endpoints - - events - - limitranges - - namespaces - - nodes - - persistentvolumeclaims - - persistentvolumes - - pods/* - - podtemplates - - replicationcontrollers - - resourcequotas - - serviceaccounts - - services - scope: "*" -- apiGroups: - - apps - - autoscaling - - batch - - networking.k8s.io - - rbac.authorization.k8s.io - - extensions - apiVersions: [ "*" ] - operations: [ "*" ] - resources: [ "*/*" ] - scope: "*" -``` - -### Proxy Usage - -There are several configuration parameters for the proxy usage - -- Two involved components `webhook.*` and `scanner.*`; reference to the first for communications to the Sysdig backend, while -second communicates with the registry from where to pull the image to be scanned. -- configuration values `*.httpProxy`, `*.httpsProxy` and `*.noProxy`. Make sure to use at least `https` version for Sysdig Secure Backend. - -If your Proxy is served with TLS -- The url for those `*.httpProxy` and `*.httpsProxy` must be `https://` -- If using a self-signed certificate you will need to also configure one of the following two options - 1. Set the `verifySSL=false` parameter - 2. Or set `*.ssl.ca.cert` for both components `webhook` and `scanner` - -## Usages - - -### Basic - -``` -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ - --create-namespace -n {{ .Release.Namespace }} \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN -``` +## Verify the Integrity and Origin +Sysdig Helm charts are signed so you can confirm the integrity and origin of each chart. To do so: +1. Import the Public Key: -### On Prem + ```bash + $ curl -o "/tmp/sysdig_public.gpg" "https://charts.sysdig.com/public.gpg" + $ gpg --import /tmp/sysdig_public.gpg + ``` -Sysdig On-Prem installations might use a TLS self-signed server certificate or one from an untrusted CA, so it requires an extra configuration. +2. Verify the chart by appending the `--verify` flag to the `install`, `upgrade`, and `pull` helm commands. -#### Ignore TLS certificate verification -Use the following command to deploy in an on-prem and ignore the untrusted certificate using `verifySSL=false`: - -``` -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ - --create-namespace -n {{ .Release.Namespace }} \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN \ - --set verifySSL=false -``` - -#### Custom CA Provided - -The following command will deploy the admission controller with a custom CA. The custom CA certificate is added to the trusted certificates store. - -``` -$ helm upgrade --install sysdig-{{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ - --create-namespace -n {{ .Release.Namespace }} \ - --set clusterName=CLUSTER_NAME \ - --set sysdig.secureAPIToken=SECURE_API_TOKEN \ - --set webhook.ssl.ca.cert=YOUR_CA_CERT_AS_PEM_ENCODED \ - --set scanner.ssl.ca.cert=YOUR_CA_CERT_AS_PEM_ENCODED -``` - -## Confirm Working Status - - -### Activity Audit - -1. Install Admission Controller on your Kubernetes Cluster following one of the (use-cases)(#usage) described - - This feature is enabled by default through `features.k8sAuditDetections` value -2. Check your current "Kubernetes Audit" policies in `Sysdig Secure > Policies > Threat Detection | Runtime Policies` as we will be triggering one of those to prove it's working correctly. - - We suggest using "Create Privileged Pod" but you can choose any. -3. If possible, let's activate just installed component logs to have them at sight - ``` - $ kubectl logs -f -n sysdig-admission-controller -l app.kubernetes.io/component=webhook - ``` -4. Trigger following command to force an unwanted audit detection - ``` - $ kubectl run nginx --image nginx --privileged - ``` -5. If you had a chance to activate logs, take a look at them. You should see something like this - ``` - {"level":"info","component":"console-notifier","message":"Pod started with privileged container (user=** pod=nginx ns=default images=nginx)"} - ``` -6. Confirm that event reached Sysdig Secure, looking at `Events` - - - -### Image Scanning - -1. Install Admission Controller on your Kubernetes Cluster following one of the (use-cases)(#usage) described - - In the chart, this feature is enabled by default through `scanner.enabled` value -2. Enable Admission-Controller on your Sysdig Secure > Image Scanning > Admission Controller > Policy Assignments -This section can only be accessed by a user with Administrator permissions -3. Add some an assignment to Allow or Deny images within a namespace -4. Tail to the logs from the Admission Controller - ``` - $ kubectl logs -f -n {{ .Release.Namespace }} -l app.kubernetes.io/component=webhook - ``` -5. Push some deployment into your Kubernetes Cluster to watch the result, for example an nginx image - ``` - $ kubectl run nginx --image=nginx - ``` - -If policy is set to allow, the deployment will be successful. - -Either way, you should see some logs in Admission Controller tail - - -- allow assignment result - {"level":"info","component":"scanning-evaluator","message":"checking pod=nginx in namespace=default"} - {"level":"info","component":"scanning-evaluator","message":"evaluating container with name=nginx and image=nginx"} - {"level":"info","component":"scanning-evaluator","time":"","message":"matched policy=Allow always for namespace=default and image=nginx"} - {"level":"info","component":"scanning-evaluator","message":"allowing container with name=nginx and image=nginx"} - - -- reject assignment result - {"level":"info","component":"scanning-evaluator","message":"checking pod=nginx in namespace=default"} - {"level":"info","component":"scanning-evaluator","message":"evaluating container with name=nginx and image=nginx"} - {"level":"info","component":"scanning-evaluator","message":"matched policy=Reject Allways for namespace=default and image=nginx"} - {"level":"info","component":"scanning-evaluator","message":"denying container with name=nginx and image=nginx reason=\"Reject Always\""} - -## Running helm unit tests - -The sysdiglabs/charts repository uses the following helm unittest plugin: https://github.com/quintush/helm-unittest - -You can test the changes to your chart by running the test suites as follows: - -``` -helm unittest --helm3 . -``` - -The helm unit tests are in the tests folder. It is recommended to add new tests as new features are added here. - -## Troubleshooting - - -### Q: I'm not able to get an alert for an event with the `ka.verb=get` condition. - -A: Despite [Kubernetes Extensible Admission Controller webhook allows it](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-rules), Sysdig Admission Controller does only handle `CREATE`, `UPDATE`, `DELETE` and `CONNECT` type of events. -Also, beware Kubernetes [`apiGroups`](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/values.yaml#L41-L54) are scoped - -S: Still, if required, you can make use of the [legacy Sysdig Kubernetes Audit Log](https://docs.sysdig.com/en/docs/sysdig-secure/secure-events/kubernetes-audit-logging/#legacy-installation-instructions) which do support more verbs. - - +## Configuration -### Q: I get tons of "TLS handshake error" +### Using the Key-Value Pair -A: This happens when DEBUG is enabled but Admission Controller will behave as expected. Those calls are some non-sysdig direct calls to the Admission Controller without TLS, which raises this informational log by Go internal library. +Specify each parameter using the `--set key=value[,key=value]` argument to the `helm install`command. +For example: -### Q: I need to troubleshoot, any way to switch to `debug` verbose? -A: If you used helm to install, you can edit the helm `values.yaml` to set `webhook.logLevel=debug` -
Alternatively, you can edit the webhook configmap - add the `LOG_LEVEL=debug` key-value and restart the webhook +```bash +helm upgrade --install {{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ + --create-namespace -n {{ .Release.Namespace }}{{ with .Chart.Version }} --version={{.}} {{ end }} \ + --set sysdig.secureAPIToken=YOUR-KEY-HERE,clusterName=YOUR-CLUSTER-NAME ``` - $ kubectl edit configmaps -n {{ .Release.Namespace }} sysdig-admission-controller-webhook - $ kubectl rollout restart deployment -n {{ .Release.Namespace }} sysdig-admission-controller-webhook -``` - -### Q: I don't see `Policy Rules` honored -S: Review the [Admission Controller - Understanding:How Policy Conditions are applied]({{ .Project.URL }}/#understanding-how-policy-conditions-are-applied) -### Q: I don't see `Policy Assignments` order honored -A: It may be that you're using same namespace and image prefix on more than one assignment
-S: Review the [Admission Controller - Understanding:Evaluation Order]({{ .Project.URL }}/#understanding-evaluation-order) +### Using values.yaml -### Q: I don't see changes on `Policy Assignments` being applied on my cluster -A: Admission Controller pull changes from the Sysdig Secure platform every 5 minutes
-S: You can wait those five minutes, or force the admission controller webhook restart +The `values.yaml` file specifies the values for the admission controller configuration parameters. You can add the configuration to the `values.yaml` file, then use it in the `helm install` command. - $ kubectl rollout restart deployment -n {{ .Release.Namespace }} sysdig-admission-controller-webhook - -### Q: I am deploying it in a GKE Cluster, with Private Network enabled, and everything is slow or I cannot scale the cluster correctly. - -```text -"Failed calling webhook, failing open audit.secure.sysdig.com: failed calling webhook "audit.secure.sysdig.com": Post "https://sysdig-ac-webhook.sysdig-agent.svc:443/k8s-audit?timeout=10s ": context canceled" -``` +For example: -A: GKE clusters run the K8s API outside from the cluster. If Private Network is enabled, the K8s API may be unable to reach the Admission Controller's webhook that validates each API request, so eventually every API request times out and is processed, but the performance is impacted in the process. -

S: As specified in [GKE Private Cluster Webhook Timeouts](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#api_request_that_triggers_admission_webhook_timing_out), the default firewall configuration does not allow TCP connections for ports other than 443 and 10250. -Admission Controller's webhook run on `5000 TCP port`, so you need to enable a new rule that allows the Control Plane's network to access it. -
Follow the instructions in [GKE-Adding firewall rules to cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#add_firewall_rules) to enable inbound connections to our webhook. - - -### Q: Getting "error getting the cluster id from kubernetes: open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied" +```bash +helm upgrade --install {{ .Release.Name }} {{ .Repository.Name }}/{{ .Chart.Name }} \ + --create-namespace -n {{ .Release.Namespace }}{{ with .Chart.Version }} --version={{.}} {{ end }} \ + --values values.yaml -A: Some users (old versions of GKE) reported that the permissions to access serviceAccount token, mounted in the filesystem, was set to [`0600` permissions](https://discuss.hashicorp.com/t/wrong-permission-on-being-set-on-serviceaccount-token/28777), not allowing the pods to actually read from it. -

S: [Recommend](https://github.com/kubernetes/kubernetes/issues/82573) to change the `securityContext.fsGroup` to the value `65534` on the pod. -
You can specify this through our helm chart with the parameter -``` ---set webhook.podSecurityContext.fsGroup=65534 ``` -### Q: Getting readiness probe errors and cannot startup +See the default [`values.yaml`](./values.yaml) file for more information. -``` -13m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/sysdig-admission-controller-webhook invalid metrics (1 invalid out of 1), first error is: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io) -``` -A: [HorizontalAutoScaller](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/templates/webhook/autoscaler.yaml) requires your kubernetes cluster to be able to use [metrics API](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-metrics-apis), which in some lightweight installations, such as minikube, must be enabled through a plugin -
S: For minikube, enable `metric-server` plugin -``` - $ minikube addons list | grep metrics-server - $ minikube addons enable metrics-server - ``` +## Configuration Parameters -### Q: Getting error "x509: certificate signed by unknown authority" - -A: Sysdig installation is made with an unverfied certificate, such as self-signed, `SECURE_URL` being `https` -
S: Add `--set verifySSL=false` to your installation parameters or configure a trusted CA certificate +The following table lists the configurable parameters of the `{{ .Chart.Name }}` chart and their default values. +{{ .Chart.Values }} -### Q: Why is there no support for `ka.sourceips`? -AdmissionController is unable to retrieve the source IP of the events, because this information is not provided by the [Kubernetes AdmissionReview](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#request). -If you really require this field, as a workaround, you can use the legacy [Sysdig Agent + Kubernetes Audit](https://docs.sysdig.com/en/docs/sysdig-secure/secure-events/kubernetes-audit-logging/#legacy-installation-instructions) +## Examples +- [Default `values.yaml`](https://github.com/sysdiglabs/charts/blob/master/charts/admission-controller/values.yaml) +- Find some [examples of these values](https://github.com/sysdiglabs/charts/tree/master/charts/admission-controller/ci)