Add tetra policyfilter listpolicies command #3122

tpapagian · 2024-11-15T15:21:24Z

Add tetra policyfilter listpolicies to determine which Kubernetes Identity Aware policies should be applied on a specific container.

Example:

$ kubectl exec -it ds/tetragon -n kube-system -c tetragon -- tetra policyfilter -r "unix:///procRoot/1/root/run/containerd/containerd.sock" listpolicies ff433e9e16467787a60ac853d9b313150091968731f620776d6d7c514b1e8d6c
ID   NAME                    STATE     FILTERID   NAMESPACE   SENSORS          KERNELMEMORY
5    lseek-podfilter-usage   enabled   5          (global)    generic_kprobe   1.72 MB
1    lseek-podfilter-app     enabled   1          (global)    generic_kprobe   1.72 MB

Details on how this works can be found on specific commits.

netlify · 2024-12-13T09:36:35Z

✅ Deploy Preview for tetragon ready!

Name	Link
🔨 Latest commit	`fca9fe4`
🔍 Latest deploy log	https://app.netlify.com/sites/tetragon/deploys/6784d80de61d9c000878d158
😎 Deploy Preview	https://deploy-preview-3122--tetragon.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

bpf/process/policy_filter.h

kkourt · 2024-12-19T14:31:20Z

cmd/tetra/debug/dump.go

 		ids := make([]string, 0, len(cgIDs))
 		for id := range cgIDs {
 			ids = append(ids, strconv.FormatUint(uint64(id), 10))
 		}
 		fmt.Printf("%d: %s\n", polId, strings.Join(ids, ","))
 	}
+
+	fmt.Println("--- Reverse Map ---")


Even if we do not rename the map I find that "Direct" and "Reverse" would make things harder to understand. Let's describe what is the key and what is the value here to make it easier to understand the output.

I have renamed those to more descriptive headers.

mtardy

A few comments :)

pkg/policyfilter/map.go

cmd/tetra/policyfilter/policyfilter.go

mtardy

Looks good to me! I'll let Kornilios review

kkourt

Thanks!

Please find some comments below.

kkourt · 2025-01-13T08:23:26Z

bpf/process/policy_filter.h

@@ -9,6 +9,7 @@

 #define POLICY_FILTER_MAX_POLICIES   128
 #define POLICY_FILTER_MAX_NAMESPACES 1024
+#define POLICY_FILTER_MAX_CGROUP_IDS 32768 /* same as polMapSize in policyfilter/state.go */


This seems high to me. According to https://kubernetes.io/docs/setup/best-practices/cluster-large/, a good limit for the number of pods per node is ~100. How about having something 512 or 1024 entries?

Yes, this makes sense. Just tried to keep that consistent with what we had in policyfilter/state.go.

Changed that to 1024.

kkourt · 2025-01-13T08:24:01Z

pkg/policyfilter/map.go

+		return PfMap{}, fmt.Errorf("opening map %s failed: %w", MapName, err)
+	}
+	if ret.cgroupMap, err = openMap(spec, CgroupMapName, polMaxPolicies); err != nil {
+		return PfMap{}, fmt.Errorf("opening cgroup map %s failed: %w", MapName, err)


Do we need to release policyMap here?

Correct, fixed that.

This patch introduces an eBPF map that maps cgroupIds to policyIds. This is handled from the user-space in a similar way to policy_filter_maps. This can be used on later PRs to quickly indentify policies that match a specific container or optimize tracing policies. Signed-off-by: Anastasios Papagiannis <[email protected]>

Signed-off-by: Anastasios Papagiannis <[email protected]>

It is useful to have a debug command to indentify which Kubernetes Identity Aware policies should be applied on a specific container. An example can be found here: Create a pod with "app: ubuntu" and "usage: dev" labels. $ cat << EOF | kubectl apply -f - apiVersion: v1 kind: Pod metadata: name: ubuntu labels: app: ubuntu usage: dev spec: containers: - name: ubuntu image: ubuntu:24.10 command: ["/bin/sleep", "3650d"] imagePullPolicy: IfNotPresent restartPolicy: Always EOF And apply several policies where some of them match while others don't. $ cat << EOF | kubectl apply -f - apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: "lseek-podfilter-app" spec: podSelector: matchLabels: app: "ubuntu" kprobes: [...] EOF $ cat << EOF | kubectl apply -f - apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: "lseek-podfilter-usage" spec: podSelector: matchLabels: usage: "dev" kprobes: [...] EOF $ cat << EOF | kubectl apply -f - apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: "lseek-podfilter-prod" spec: podSelector: matchLabels: prod: "true" kprobes: [...] EOF $ cat << EOF | kubectl apply -f - apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: "lseek-podfilter-info" spec: podSelector: matchLabels: info: "broken" kprobes: [...] EOF $ cat << EOF | kubectl apply -f - apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: "lseek-podfilter-global" spec: kprobes: [...] EOF Based on the labels we expect that policies lseek-podfilter-app and lseek-podfilter-usage to match on that pod. lseek-podfilter-global is not a Kubernetes Identity Aware policy so this will be applied in all cases and we do not report that. First step is to find the container ID that we care about. $ kubectl describe pod/ubuntu | grep containerd Container ID: containerd://ff433e9e16467787a60ac853d9b313150091968731f620776d6d7c514b1e8d6c And then use it to report all Kubernetes Identity Aware policies that match. $ kubectl exec -it ds/tetragon -n kube-system -c tetragon -- tetra policyfilter -r "unix:///procRoot/1/root/run/containerd/containerd.sock" listpolicies ff433e9e16467787a60ac853d9b313150091968731f620776d6d7c514b1e8d6c ID NAME STATE FILTERID NAMESPACE SENSORS KERNELMEMORY 5 lseek-podfilter-usage enabled 5 (global) generic_kprobe 1.72 MB 1 lseek-podfilter-app enabled 1 (global) generic_kprobe 1.72 MB We also provide --debug flag to provide more details i.e.: $ kubectl exec -it ds/tetragon -n kube-system -c tetragon -- tetra policyfilter -r "unix:///procRoot/1/root/run/containerd/containerd.sock" listpolicies ff433e9e16467787a60ac853d9b313150091968731f620776d6d7c514b1e8d6c --debug time="2024-12-13T09:47:38Z" level=info msg=cgroup path=/run/tetragon/cgroup2/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod189a8053_9f36_4250_bcae_9ed167172920.slice/cri-containerd-ff433e9e16467787a60ac853d9b313150091968731f620776d6d7c514b1e8d6c.scope time="2024-12-13T09:47:38Z" level=info msg=cgroup id=5695 time="2024-12-13T09:47:39Z" level=debug msg="resolved server address using info file" InitInfoFile=/var/run/tetragon/tetragon-info.json ServerAddress="localhost:54321" ID NAME STATE FILTERID NAMESPACE SENSORS KERNELMEMORY 1 lseek-podfilter-app enabled 1 (global) generic_kprobe 1.72 MB 5 lseek-podfilter-usage enabled 5 (global) generic_kprobe 1.72 MB This uses the cgroup-based policy filter map that introduced in a previous commit that maps cgroupIds to policyIds. Signed-off-by: Anastasios Papagiannis <[email protected]>

By adding a command line argument (and the appropriate configmap option). Signed-off-by: Anastasios Papagiannis <[email protected]>

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch 3 times, most recently from d62cd19 to 7ed2acf Compare November 16, 2024 20:00

tpapagian added the release-note/misc This PR makes changes that have no direct user impact. label Nov 16, 2024

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch 2 times, most recently from a6e138f to 0350886 Compare November 21, 2024 09:43

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch from 0350886 to 9a9bfd1 Compare November 28, 2024 08:20

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch 4 times, most recently from edb12b0 to 1833233 Compare December 13, 2024 09:35

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch from 1833233 to 6d7bb31 Compare December 13, 2024 10:15

tpapagian changed the title ~~Test~~ Add tetra policyfilter listpolicies command Dec 13, 2024

tpapagian requested a review from kkourt December 13, 2024 10:43

tpapagian marked this pull request as ready for review December 13, 2024 10:43

tpapagian requested a review from a team as a code owner December 13, 2024 10:43

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch from 6d7bb31 to 0de3361 Compare December 16, 2024 16:14

tpapagian requested a review from mtardy as a code owner December 16, 2024 16:14

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch from 0de3361 to 06d4488 Compare December 17, 2024 14:13

kkourt reviewed Dec 19, 2024

View reviewed changes

mtardy reviewed Jan 6, 2025

View reviewed changes

pkg/policyfilter/map.go Outdated Show resolved Hide resolved

pkg/policyfilter/map.go Outdated Show resolved Hide resolved

pkg/policyfilter/map.go Outdated Show resolved Hide resolved

cmd/tetra/policyfilter/policyfilter.go Outdated Show resolved Hide resolved

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch from 06d4488 to 83d092c Compare January 8, 2025 10:22

tpapagian requested review from mtardy and kkourt January 8, 2025 11:01

mtardy approved these changes Jan 10, 2025

View reviewed changes

kkourt reviewed Jan 13, 2025

View reviewed changes

tpapagian added 3 commits January 13, 2025 11:07

Add tests for the cgroup-based policy filter mapping

7fb7fde

Signed-off-by: Anastasios Papagiannis <[email protected]>

Make cgroup-based policy filter mapping optional

fca9fe4

By adding a command line argument (and the appropriate configmap option). Signed-off-by: Anastasios Papagiannis <[email protected]>

tpapagian force-pushed the pr/apapag/reverse_policy_id_map branch from 83d092c to fca9fe4 Compare January 13, 2025 09:08

tpapagian requested a review from kkourt January 13, 2025 09:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add tetra policyfilter listpolicies command #3122

Add tetra policyfilter listpolicies command #3122

tpapagian commented Nov 15, 2024 •

edited

Loading

netlify bot commented Dec 13, 2024 •

edited

Loading

kkourt Dec 19, 2024

tpapagian Jan 8, 2025

mtardy left a comment

mtardy left a comment

kkourt left a comment

kkourt Jan 13, 2025

tpapagian Jan 13, 2025

kkourt Jan 13, 2025

tpapagian Jan 13, 2025

Add tetra policyfilter listpolicies command #3122

Are you sure you want to change the base?

Add tetra policyfilter listpolicies command #3122

Conversation

tpapagian commented Nov 15, 2024 • edited Loading

netlify bot commented Dec 13, 2024 • edited Loading

✅ Deploy Preview for tetragon ready!

kkourt Dec 19, 2024

Choose a reason for hiding this comment

tpapagian Jan 8, 2025

Choose a reason for hiding this comment

mtardy left a comment

Choose a reason for hiding this comment

mtardy left a comment

Choose a reason for hiding this comment

kkourt left a comment

Choose a reason for hiding this comment

kkourt Jan 13, 2025

Choose a reason for hiding this comment

tpapagian Jan 13, 2025

Choose a reason for hiding this comment

kkourt Jan 13, 2025

Choose a reason for hiding this comment

tpapagian Jan 13, 2025

Choose a reason for hiding this comment

tpapagian commented Nov 15, 2024 •

edited

Loading

netlify bot commented Dec 13, 2024 •

edited

Loading