Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow libbpf-tools to run on old kernels #4231

Open
6 of 14 tasks
chenhengqi opened this issue Sep 9, 2022 · 12 comments · Fixed by #4233
Open
6 of 14 tasks

Allow libbpf-tools to run on old kernels #4231

chenhengqi opened this issue Sep 9, 2022 · 12 comments · Fixed by #4233

Comments

@chenhengqi
Copy link
Collaborator

chenhengqi commented Sep 9, 2022

We have a bunch of libbpf tools require features like fentry / tp_btf which only available on kernel v5.5+.
Running these tools on old kernels result in EINVAL. Instead of error out, we can fallback to kprobe.
Open this ticket to keep track of this process:

  • biolatency
  • biosnoop
  • biostacks
  • bitesize
  • cachestat
  • cpudist
  • cpufreq
  • drsnoop
  • hardirqs
  • mdflush
  • numamove
  • offcputime
  • readahead
  • softirqs

If you are interested, please feel free to pick it up.

@chenhengqi chenhengqi pinned this issue Sep 9, 2022
chenhengqi added a commit to chenhengqi/bcc that referenced this issue Sep 13, 2022
This is part of efforts towards iovisor#4231. Fallback to raw tracepoints
if tp_btf is not available. With this patch, we also remove manual
attach and merge the two irq_handler_entry handlers into one.

Signed-off-by: Hengqi Chen <[email protected]>
chenhengqi added a commit to chenhengqi/bcc that referenced this issue Sep 13, 2022
This is part of efforts towards iovisor#4231. Fallback to raw tracepoints
if tp_btf is not available.

Signed-off-by: Hengqi Chen <[email protected]>
@chenhengqi chenhengqi linked a pull request Sep 13, 2022 that will close this issue
@eiffel-fl
Copy link
Contributor

biolatency

I wanted to give a shot at biolatency but I hit a wall as this is not easy to replace the tracepoint by kprobes.
Indeed, the three tracepoints of this tools are called from different kernel functions, so this is not trivial to replace it with kprobes.

I suggest more using raw_tp instead (which I was able to make work).
But I am not sure when the raw_tp were introduced... Were they in 4.17 with this commit?

@chenhengqi
Copy link
Collaborator Author

Using raw_tp is enough, this will make the tools running on older LTS kernels (like v4.19/v5.4).

yonghong-song pushed a commit that referenced this issue Oct 8, 2022
This is part of efforts towards #4231. Fallback to raw tracepoints
if tp_btf is not available. With this patch, we also remove manual
attach and merge the two irq_handler_entry handlers into one.

Signed-off-by: Hengqi Chen <[email protected]>
yonghong-song pushed a commit that referenced this issue Oct 8, 2022
This is part of efforts towards #4231. Fallback to raw tracepoints
if tp_btf is not available.

Signed-off-by: Hengqi Chen <[email protected]>
@chenhengqi chenhengqi reopened this Oct 9, 2022
@woodpenker
Copy link
Contributor

Do we consider providing a cmd flag such as -btf to let user provide their BTF file for the kernel, so that we can use tools even on 4.x kernels?

@chenhengqi
Copy link
Collaborator Author

Do we consider providing a cmd flag such as -btf to let user provide their BTF file for the kernel, so that we can use tools even on 4.x kernels?

We have such support at build time. See #3889 .

@woodpenker
Copy link
Contributor

Do we consider providing a cmd flag such as -btf to let user provide their BTF file for the kernel, so that we can use tools even on 4.x kernels?

We have such support at build time. See #3889 .

It seems to rely on the BTF hub and won't support self-built kernels or some old kernels which can't be found in the hub, eg: Ubuntu 4.18.0-22. Since LIBBPF_OPTS is supported, how about letting the user make a choice?
example:

if (env.btf_path) {
  LIBBPF_OPTS(bpf_object_open_opts, bpf_opts, .btf_custom_path = env.btf_path);
  skel = xxx_bpf__open_opts(&bpf_opts);
} else {
  skel = xxx_bpf__open();
}

@chenhengqi
Copy link
Collaborator Author

Instead of touching every tool, I'd suggest providing the BTF path through environment variable and teach ensure_core_btf() to recognize it.

@woodpenker
Copy link
Contributor

Instead of touching every tool, I'd suggest providing the BTF path through environment variable and teach ensure_core_btf() to recognize it.

Thanks for explaining, I may try that. But I realize another problem with global values, only legacy codes are supported on old kernels like this. To support them, maybe we need some mechanisms to auto-generate legacy codes instead of global values?

@woodpenker
Copy link
Contributor

BTW, some tools such as softirqs seems not using ensure_core_btf(), is it intended?

@chenhengqi
Copy link
Collaborator Author

Yes, tp_btf relies on BPF trampoline which support /sys/kernel/btf/vmlinux only.

@woodpenker
Copy link
Contributor

Yes, tp_btf relies on BPF trampoline which support /sys/kernel/btf/vmlinux only.

Thanks, I want to try to teach some tools listed above to fall back to raw_tp or Kprobe, maybe start from offcputime which I am familiar with. Another question, it seems kernel support auto falls back to tp now after this patch, am I right?

@chenhengqi
Copy link
Collaborator Author

Another question, it seems kernel support auto falls back to tp now after this patch, am I right?

No.

@woodpenker
Copy link
Contributor

Since cpufreq need Memory-mapping BPF maps support starting from v5.7(b97e12e594eb & 333291ce5055), I think there is no need to change it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants