systemd leaks file descriptors

[waldheinz]

Describe the bug

Since 89700ef, systemd leaks file descriptors. Those are all of similar form:

# ls -l /proc/1/fd
lrwx------ 1 root root 64 Nov 11 11:43 345 -> anon_inode:bpf-prog
lrwx------ 1 root root 64 Nov 11 11:43 346 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 347 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 348 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 349 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 35 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 350 -> anon_inode:bpf-prog
lrwx------ 1 root root 64 Nov 11 11:43 351 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 352 -> anon_inode:bpf-prog
lrwx------ 1 root root 64 Nov 11 11:43 353 -> anon_inode:bpf-prog
lrwx------ 1 root root 64 Nov 11 11:43 354 -> 'anon_inode:[pidfd]'
lrwx------ 1 root root 64 Nov 11 11:43 355 -> anon_inode:bpf-map
lrwx------ 1 root root 64 Nov 11 11:43 356 -> anon_inode:bpf-prog
...

Steps To Reproduce

Just run something like this:

watch "ssh [email protected] 'ls /proc/1/fd | wc -l'"

And see the number increase by two FDs for every SSH connect.

Additional context

Because of the nature of these FDs and that AFAIK systemd uses BPF magic to do this cgroup stuff, I assume that dropping this patchset: systemd/systemd#33258 is problematic. But it does not apply cleanly anymore. Also, it seems other Linux distros running the same systemd version as NixOS are not affected.

Notify maintainers

Hi, @arianvp , @flokli and @kloenk !

[waldheinz]

This commit seems to fix the issue:

systemd/systemd@5a8c2c9

[emhamm]

systemd v257-rc1 fixes it as well.

And it is not only ssh, it seems every systemd service unit which restarts leaks FDs.

[arianvp]

I don't understand why this is NixOS-specific. Can you elaborate? It sounds like an upstream bug that should be backported?

[emhamm]

I don't understand why this is NixOS-specific. Can you elaborate? It sounds like an upstream bug that should be backported?

I have tested it with fedora 41 - systemd 256.7, kernel 6.11.7 - this distribution does not suffer from this bug. The package they build does not contain any patches targeting bpf as far as I can tell https://src.fedoraproject.org/rpms/systemd/tree/f41

[arianvp]

We don't ship any patches targeting bpf either. Hence me aksing if you have any clue why this is only popping up for us

https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/systemd/default.nix#L218