Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various fixes, mostly on form and organization #1603

Merged
merged 5 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions docs/content/en/docs/advanced-config/_index.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/content/en/docs/concepts/enforcement/_index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Enforcement"
icon: "overview"
weight: 5
weight: 4
description: "Documentation for Tetragon enforcement system"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Tetragon Events"
title: "Events"
icon: "overview"
weight: 3
weight: 1
description: "Documentation for Tetragon event system"
---

Expand Down
8 changes: 8 additions & 0 deletions docs/content/en/docs/concepts/events/grpc-events.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: "gRPC Events"
weight: 3
icon: "reference"
description: "Tetragon gRPC events"
---

A gRPC endpoint is exposed by the agent and is configurable.
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
---
title: "Tetragon Metrics"
weight: 1
title: "Metrics"
icon: "overview"
description: "Fetching and understanding Tetragon metrics"
weight: 2
description: "Documentation for Tetragon metrics"
---

Tetragon's metrics are exposed to the system through an HTTP endpoint. These
are used to expose event summaries and information about the state of the
Tetragon agent.

## Kubernetes

Tetragon pods expose a metrics endpoint by default. The chart also creates a service named `tetragon`
that exposes metrics on the specified port.
Tetragon pods exposes a metrics endpoint by default. The chart also creates a
service named `tetragon` that exposes metrics on the specified port.

### Getting metrics port

Expand All @@ -25,8 +29,8 @@ tetragon ClusterIP 10.96.54.218 <none> 2112/TCP 3m
```

{{< note >}}
In the previous output it shows, 2112 is the port on which the service is
listening. It is also the port on which the Tetragon metrics server listens
In the previous output it shows, 2112 is the port on which the service is
listening. It is also the port on which the Tetragon metrics server listens
with the default Helm values.
{{< /note >}}

Expand All @@ -40,11 +44,11 @@ kubectl -n kube-system port-forward service/tetragon 2112:2112

## Package

By default, metrics are disabled, which can be enabled using `--metrics-server`
By default, metrics are disabled, which can be enabled using `--metrics-server`
flag, by specifying the address.

Alternatively, the [examples/configuration/tetragon.yaml](https://github.com/cilium/tetragon/blob/main/examples/configuration/tetragon.yaml)
file contains example entries showing the defaults for the address of
file contains example entries showing the defaults for the address of
metrics-server. Local overrides can be created by editing and copying this file
into `/etc/tetragon/tetragon.yaml`, or by editing and copying "drop-ins" from
the [examples/configuration/tetragon.conf.d](https://github.com/cilium/tetragon/tree/main/examples/configuration/tetragon.conf.d)
Expand All @@ -62,25 +66,25 @@ sudo tetragon --metrics-server localhost:2112
The output should be similar to this:

```
time="2023-09-21T13:17:08+05:30" level=info msg="Starting tetragon"
time="2023-09-21T13:17:08+05:30" level=info msg="Starting tetragon"
version=v0.11.0
time="2023-09-21T13:17:08+05:30" level=info msg="config settings"
time="2023-09-21T13:17:08+05:30" level=info msg="config settings"
config="mapeased
time="2023-09-22T23:16:24+05:30" level=info msg="Starting metrics server"
addr="localhost:2112"
time="2023-09-22T23:16:24+05:30" level=info msg="Starting metrics server"
addr="localhost:2112"
[...]
time="2023-09-21T13:17:08+05:30" level=info msg="Listening for events..."
```

Alternatively, a file named `server-address` can be created in `etc/tetragon/tetragon.conf.d/metrics-server` with content specifying
a port like this `localhost:2112`, or any port of your choice as mentioned
Alternatively, a file named `server-address` can be created in `etc/tetragon/tetragon.conf.d/metrics-server` with content specifying
a port like this `localhost:2112`, or any port of your choice as mentioned
above.

## Fetch the Metrics

After the metrics are exposed, either by port forwarding in case of
Kubernetes installation or by setting metrics address in case of Package
installation, the metrics can be fetched using
After the metrics are exposed, either by port forwarding in case of
Kubernetes installation or by setting metrics address in case of Package
installation, the metrics can be fetched using
`curl` on `localhost:2112/metrics`:

```shell-session
Expand Down
10 changes: 0 additions & 10 deletions docs/content/en/docs/concepts/metrics/_index.md

This file was deleted.

8 changes: 0 additions & 8 deletions docs/content/en/docs/concepts/tetragon-events/grpc-events.md

This file was deleted.

139 changes: 70 additions & 69 deletions docs/content/en/docs/getting-started/enforcement.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
---
title: "Policy Enforcement"
weight: 2
weight: 6
description: "Policy Enforcement"
---

This adds a network and file policy enforcement on top of execution, file tracing
and networking policy already deployed in the quick start. In this use case we use
a namespace filter to limit the scope of the enforcement policy to just the 'darkstar'
a namespace filter to limit the scope of the enforcement policy to just the `darkstar`
cluster we installed the demo application in from the
[Quick Kubernetes Install]({{< ref "docs/getting-started/install-k8s" >}}).

This highlights two important concepts of Tetragon. First in kernel filtering
provides a key performance improvement by limiting events from kernel to user
space. But, also allows for enforcing policies in the kernel. By issueing a
SIGKILL to the binary at this point the application will be stopped from
`SIGKILL` to the process at this point the application will be stopped from
continuing to run. If the operation is triggered through a syscall this means
the application will not return from the syscall and will be terminated.

Expand All @@ -22,20 +22,21 @@ segment a policy to apply to targeted namespaces and pods. This is critical
for effective policy segmentation.

For implementation details see the [Enforcement]({{< ref "/docs/concepts/enforcement" >}})
section.
concept section.

## Kubernetes Enforcement

The following section is layed out with the following: A guide to promote the
network observation policy that observer all network traffic egressing the
cluster to enforce this policy. A guide to promote the file access monitoring
policy to block write and read operations to sensitive files.
The following section is layed out with the following:
- A guide to promote the network observation policy that observer all network
traffic egressing the cluster to enforce this policy.
- A guide to promote the file access monitoring policy to block write and read
operations to sensitive files.

### Kubernetes Block TCP Connect outside Cluster
### Block TCP Connect outside Cluster

First we will deploy the [Network Monitoring]({{< ref "docs/getting-started/network" >}})
policy with enforcement on. For this case the policy is written to only apply
against the 'empire' namespace. This limits the scope of the policy for the
against the `empire` namespace. This limits the scope of the policy for the
getting started guide.

Ensure we have the proper Pod CIDRs
Expand All @@ -46,86 +47,81 @@ export PODCIDR=`kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'`

and Service CIDRs configured.

{{< tabpane text=true >}}
{{% tab GKE %}}

```shell-session
{{< tabpane lang=shell-session >}}
{{< tab GKE >}}
export SERVICECIDR=$(gcloud container clusters describe ${NAME} --zone ${ZONE} --project ${PROJECT} | awk '/servicesIpv4CidrBlock/ { print $2; }')
```
{{% /tab %}}
{{< /tab >}}

{{% tab Kind %}}
```shell-session
{{< tab Kind >}}
export SERVICECIDR=$(kubectl describe pod -n kube-system kube-apiserver-kind-control-plane | awk -F= '/--service-cluster-ip-range/ {print $2; }')
```
{{< /tab >}}
{{< /tabpane >}}

Then we can apply the egress cluster enforcement policy

```shell-session
wget http://github.com/cilium/tetragon/quickstart/network_egress_cluster_enforce.yaml
wget http://github.com/cilium/tetragon/examples/quickstart/network_egress_cluster_enforce.yaml
envsubst < network_egress_cluster_enforce.yaml | kubectl apply -n default -f -
```

With the enforcement policy applied we can attach tetra to observe events again,
With the enforcement policy applied we can attach tetra to observe events again:

```shell-session
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
```

And once again execute a curl command in the xwing,
And once again execute a curl command in the xwing:

```shell-session
kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon'
```

The command returns an error code because the egress TCP connects are blocked shown here
```shell-session
$ kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon'
The command returns an error code because the egress TCP connects are blocked shown here.
```
command terminated with exit code 137
```

connect inside the cluster will work as expected,
Connect inside the cluster will work as expected,

```shell-session
kubectl exec -ti xwing -- bash -c 'curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing'
```

The Tetra CLI will print the curl and annotate that the process that was issued a Sigkill. The successful internal connect is filtered and will not be shown.
The Tetra CLI will print the curl and annotate that the process that was issued
a Sigkill. The successful internal connect is filtered and will not be shown.

``` shell
🚀 process default/xwing /bin/bash -c "curl https://ebpf.io/applications/#tetragon"
🚀 process default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon
🔌 connect default/xwing /usr/bin/curl tcp 10.32.0.28:45200 -> 104.198.14.52:443
💥 exit default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon SIGKILL
🚀 process default/xwing /bin/bash -c "curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing"
🚀 process default/xwing /usr/bin/curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
```
🚀 process default/xwing /bin/bash -c "curl https://ebpf.io/applications/#tetragon"
🚀 process default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon
🔌 connect default/xwing /usr/bin/curl tcp 10.32.0.28:45200 -> 104.198.14.52:443
💥 exit default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon SIGKILL
🚀 process default/xwing /bin/bash -c "curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing"
🚀 process default/xwing /usr/bin/curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
```

The enforces TCP connects see [Enforce Sandbox]({{< ref "#enforce-common-security-policy" >}}) below to further restrict possible
workaround such as writing through /dev devices and raw sockets application may
attempt.
The enforces TCP connects see [Enforce Sandbox]({{< ref "#enforce-common-security-policy" >}})
below to further restrict possible workaround such as writing through /dev
devices and raw sockets application may attempt.

### Enforce File Access Monitoring

The following extends the example from [File Access Monitoring]({{< ref "docs/getting-started/file-events" >}})
with enforcement to ensure sensitive files are not read. The policy used is the
[`file-monitoring-enforce.yaml`](https://github.com/cilium/tetragon/blob/main/quickstart/file-monitoring-enforce.yaml)
[`file-monitoring-enforce.yaml`](https://github.com/cilium/tetragon/blob/main/examples/quickstart/file-monitoring-enforce.yaml)
it can be reviewed and extended as needed. The only difference between the
observation policy and the enforce policy is the addition of an action block
to sigkill the application and return an error on the op.

To apply the policy.
To apply the policy:

{{< tabpane lang=shell-session >}}

{{< tab Kubernetes >}}
kubectl delete -f http://github.com/cilium/tetragon/quickstart/file_monitoring.yaml
kubectl apply -f http://github.com/cilium/tetragon/quickstart/file_monitoring_enforce.yaml
{{< /tab >}}
{{< tab Docker >}}
wget http://github.com/cilium/tetragon/quickstart/file-monitoring.yaml
{{< tab Kubernetes >}}
kubectl delete -f http://github.com/cilium/tetragon/examples/quickstart/file_monitoring.yaml
kubectl apply -f http://github.com/cilium/tetragon/examples/quickstart/file_monitoring_enforce.yaml
{{< /tab >}}
{{< tab Docker >}}
wget http://github.com/cilium/tetragon/examples/quickstart/file-monitoring.yaml
docker stop tetragon-container
docker run --name tetragon-container --rm --pull always \
--pid=host --cgroupns=host --privileged \
Expand All @@ -138,58 +134,63 @@ docker run --name tetragon-container --rm --pull always \
With the file applied we can attach tetra to observe events again,

{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
{{< tab Kubernetes >}}
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
{{< /tab >}}
{{< tab Docker >}}
{{< tab Docker >}}
docker exec tetragon-container tetra getevents -o compact
{{< /tab >}}
{{< /tabpane >}}

Then reading a sensitive file,

{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
{{< tab Kubernetes >}}
kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
{{< /tab >}}
{{< tab Docker >}}
{{< tab Docker >}}
cat /etc/shadow
{{< /tab >}}
{{< /tabpane >}}

The command will fail with an error code because this is one of our sensitive files,
```shell-session
kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
```

The output should be similar to:

```
$ kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
command terminated with exit code 137
```

This will generate a read event (Docker events will omit Kubernetes metadata),

```shell-session
🚀 process default/xwing /bin/bash -c "cat /etc/shadow"
🚀 process default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
```
🚀 process default/xwing /bin/bash -c "cat /etc/shadow"
🚀 process default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
💥 exit default/xwing /bin/cat /etc/shadow SIGKILL
```

Writes and reads to files not part of the enforced file policy will not be
impacted.

```shell-session
🚀 process default/xwing /bin/bash -c "echo foo >> bar; cat bar"
🚀 process default/xwing /bin/cat bar
💥 exit default/xwing /bin/cat bar 0
💥 exit default/xwing /bin/bash -c "echo foo >> bar; cat bar" 0
```
🚀 process default/xwing /bin/bash -c "echo foo >> bar; cat bar"
🚀 process default/xwing /bin/cat bar
💥 exit default/xwing /bin/cat bar 0
💥 exit default/xwing /bin/bash -c "echo foo >> bar; cat bar" 0
```

# What's next
## What's next

The completes the quick start guides. At this point we should be able to observe
execution traces in a Kubernetes cluster and extend the base deployment of
Tetragon with policies to observe and enforce different aspects of a Kubernetes
system.
The completes the quick start guides. At this point we should be able to
observe execution traces in a Kubernetes cluster and extend the base deployment
of Tetragon with policies to observe and enforce different aspects of a
Kubernetes system.

The rest of the docs provide further documentation about installation and
using policies. Some useful links:
Expand Down
Loading
Loading