rusty.bpf.c has a few small places where we can improve either the
formatting of the code, or the logic. In rusty_select_cpu(), we declare
the idle_smtmask as struct cpumask *, when it could be const. Also, when
initializing the pcpu_ctx, we're using an actual for-loop instead of
bpf_for. Let's just fix up these small issues.

Signed-off-by: David Vernet <[email protected]>

scx: Adjust a couple of small things in rusty.bpf.c

When used from a bpf scheduler that is launched via libbpf-rs this
naming runs into issues because "type" is a reserved keyword in Rust.

Signed-off-by: Dan Schatzberg <[email protected]>

scx: Rename "type" -> "exit_type"

The struct cpumask * argument to the ops.set_cpumask() op isn't const.
It doesn't really matter in terms of mutability in a BPF program, but
let's make it const just because it really is.

Signed-off-by: David Vernet <[email protected]>

scx: Make cpumask arg to ops.set_cpumask() const

Andrea Righi reports that smp_load_acquire() can't be used on u64's on some
32bit architectures. pnt_seq is used to close a very short race window and
32bit should be more than enough. Use unsigned long instead of u64.

Some of the sched_ext kfuncs are fine to call from tracepoints. For
example, we may want to call scx_bpf_error_bstr() if some error
condition is detected in a tracepoint rather than a sched_ext ops
callback. This patch therefore separates the scx_kfunc_ids_any kfunc BTF
set into two sets: one of which includes kfuncs that can only be called
from struct_ops, and the other which can be called from both struct_ops
and tracepoint progs.

Signed-off-by: David Vernet <[email protected]>

scx: Allow calling some kfuncs from tracepoints

atomic64_t can be pretty inefficient in 32bit archs and the counter being
32bit on 32bit arch is fine. Let's use atomic_long_t instead.

Some 32bit archs can't do 64bit store_release/load_acquire. Use
atomic_long_t instead.

Use unsigned longs for atomics

…ples

use resizing of datasec maps in examples

…cesses

The function is currently returning 0 for unknown accesses which means
allowing writes to anything. Fix the default return value.

scx: bpf_scx_btf_struct_access() should return -EACCES for unknown accesses

…TASK_ON_DSQ_PRIQ

p->scx.flag is protected by the task's rq lock but one of the flags,
SCX_TASK_ON_DSQ_PRIQ, is protected by p->dsq->lock, not its rq lock. This
could lead to corruption of p->scx.flags through RMW races triggering
watchdog and other sanity checks. Fix it moving it to its own flag field
p->scx.dsq_flags which is protected by the dsq lock.

scx: Fix p->scx.flags corruption due to unsynchronized writes of SCX_TASK_ON_DSQ_PRIQ

scx_rusty: keep .bpf.o files for debugging

This reverts commit 664d650, reversing
changes made to ee9077a.

(cherry picked from commit be81498)

Fix incorrect merge of #48

Instead, collect all per-dom cpumasks into all_cpumask and test whether
that's a subset of a task's cpumask. bpf_cpumask_full() can incorrectly
indicate that a task's affinity is restricted when it's not depending on the
machine configuration.

Researchers at Inria-Paris are experimenting with the central
scheduler, and want to try setting different slice lengths to see how
they affect performance for VMs running the NAS benchmarks. Let's make
this convenient by allowing it to be passed as a parameter from user
space.

Signed-off-by: David Vernet <[email protected]>

The scx_central scheduler specifies an infinite slice for all cores
other than a "central" core where scheduling decisions are made. This
scheduler currently suffers from the fact that the BPF timer may be
invoked on a different core than the central scheduler, due to BPF
timers not supporting being pinned to specific CPUs.

That capability was proposed upstream for BPF in [0]. If and when it
lands, we would need to invoke bpf_timer_start() from the core that we
want the timer pinned to, because the API does not support specifying a
core to have the timer invoked from. To accommodate this, we can
affinitize the loading thread to the central CPU before loading the
scheduler, and then pin from there.

[0]: https://lore.kernel.org/bpf/[email protected]/T/

Though the BPF timer pinning feature has not yet landed, we can still
set the stage for leveraging it by adding the logic to affinitize the
loading thread to the central CPU. While we won't yet have a guarantee
that the timer will be pinned to the same core throughout the runtime
of the scheduler, in practice, it seems that affinitizing in this manner
does make it very likely regardless. In addition, the user space
component of the central scheduler doesn't benefit from running on a
tickless core, so keeping it affinitized to the central CPU avoids it
from preempting a task on a tickless core that would otherwise benefit
from less preemption.

Signed-off-by: David Vernet <[email protected]>

There's a comment that says can_stop_tick_scx(). The function is
scx_can_stop_tick().

Signed-off-by: David Vernet <[email protected]>

scx: Fix typo in tickless comment

central: Pin timer callbacks to central CPU

Bpf next merge

Forgot to git add a small conflict to resolve

Signed-off-by: David Vernet <[email protected]>

- Includes the latest timer pinning feature

In commit d6247ec ("bpf: Add ability to pin bpf timer to calling
CPU"), BPF added the ability to be able to pin a BPF timer to the
calling CPU. Let's use this capability from the central scheduler.

Signed-off-by: David Vernet <[email protected]>

Bpf next merge

The current scx build system is a bit hacky. We put some build artifacts
in a tools/ directory, and others (skel files and .bpf.o files) we leave
in the current directory. This isn't conducive to environments that want
to package sched_ext schedulers. This patch therefore updates the
Makefile to have the build put all build artifacts (including the
compiled binaries for the schedulers into an build/ directory
(previously tools/). All artifacts will be deployed as follows:

build/bin: Compiled binaries (e.g. scx_simple, scx_central, etc)
build/sbin: Compiled binaries that are used as part of the build
            process, e.g. bpftool
build/include: Headers that are visible from .c files
build/obj: Contains object files and libraries that are used as part of
           the build process
build/obj/bpftool: Build artifacts from compiling bpftool from source
build/obj/libbpf: Build artifacts from compiling libbpf from source
build/obj/sched_ext: Build artifacts from compiling and linking BPF
                     programs and their user space counterparts.
build/release: Build output from Cargo for Rust schedulers

This patch also adds the following enhancement:

- Support for changing the build directory output by specifying the O
  environment variable, as in:

$ O=/tmp/sched_ext make CC=clang LLVM=1 -j

to output all artifacts for that build job to /tmp/sched_ext/build

- Removing code duplication by defining a ccsched make function for
  compiling schedulers, and an $(SCX_COMMON_DEPS) variable for common
  dependencies.

Signed-off-by: David Vernet <[email protected]>

Another requirement of packaging systems is to be able to install
compiled schedulers in some reachable PATH endpoint so they can be
accessed easily. This patch adds a new install target in Make for this,
which installs the schedulers on the system at /usr/bin. The user also
has the option of specifying DESTDIR to indicate a prefix of /usr/bin.

Signed-off-by: David Vernet <[email protected]>

It's mostly self evident, but now that we support environment variables
to dictate build behavior, we should document them in a clean and easy
to consume way.

Signed-off-by: David Vernet <[email protected]>

Cargo supports the cargo fetch command to fetch dependencies via the
network before compiling with cargo build. Let's put it into a separate
Makefile target so that packaging systems can separate steps that
require network access from just building.

Signed-off-by: David Vernet <[email protected]>

We were previously under the impression that the rustup nightly
toolchain was required to build the schedulers. Daan pointed out in [0]
that he was able to build with stable, and I similarly was able to build
with rust stable 1.70.0. Let's update the README accordingly.

[0]: sched-ext/sched_ext#57

We also update the README to not explicitly require compiling the
schedulers with

$ make LLVM=1 CC=clang

The BPF schedulers are automatically compiled with clang. If you compile
this way, the user space portions will be compiled with gcc, which is
fine.

Signed-off-by: David Vernet <[email protected]>

scx: Don't specify nightly rustup as dependency

Update and refactor scheduler build system

We previously separated the scx_rusty build into two steps -- a step to
download dependencies, and another to build. That mostly works, except
that the download-dependencies step is always run before the build step
as it's a dependency. Even though it doesn't download any cargo
dependencies, it still accesses the network.

Let's add a way for builders to pass --offline to cargo via a
CARGO_OFFLINE make variable so that we don't need scx_rusty_deps to be a
dependency of scx_rusty.

Signed-off-by: David Vernet <[email protected]>

rusty: Further tweak build system

Signed-off-by: David Vernet <[email protected]>

We're missing an entry in .gitignore for the build-generated files when
building the example schedulers.

We've gotten some feedback that it's confusing and/or inconvenient to
know what needs to be clean built in order to be able to correctly
compile and run the example schedulers. Let's update the build targets
to make this simpler by:

1. Always cleaning sched_ext schedulers on make mrproper in the tree
   root
2. Adding a make fullclean target to the sched_ext tools directory which
   also invokes the root make clean target.

Signed-off-by: David Vernet <[email protected]>

Update README, and improve build usability

…untime calculation

Calculating runtime from the amount consumed from slice punishes
yield(2)ers. There's nothing fundamentally wrong with it but it doesn't
align well with how cfs does it and can have unexpected effects on
applications.

Note the caveat in the example schedulers and switch scx_rusty to use
timestamp based one.

… task_ctx var name

As it was pointed out by Yonghong Song [1], in the bpf selftests the use
of the ASSERT_* series of macros is preferred over the CHECK macro.
This patch replaces all CHECK calls in bpf_iter with the appropriate
ASSERT_* macros.

[1] https://lore.kernel.org/lkml/[email protected]

Suggested-by: Yonghong Song <[email protected]>
Signed-off-by: Yuran Pereira <[email protected]>
Acked-by: Yonghong Song <[email protected]>
Acked-by: Kui-Feng Lee <[email protected]>
Link: https://lore.kernel.org/r/DB3PR10MB6835E9C8DFCA226DD6FEF914E8A3A@DB3PR10MB6835.EURPRD10.PROD.OUTLOOK.COM
Signed-off-by: Alexei Starovoitov <[email protected]>

Since some malloc calls in bpf_iter may at times fail,
this patch adds the appropriate fail checks, and ensures that
any previously allocated resource is appropriately destroyed
before returning the function.

Signed-off-by: Yuran Pereira <[email protected]>
Acked-by: Yonghong Song <[email protected]>
Acked-by: Kui-Feng Lee <[email protected]>
Link: https://lore.kernel.org/r/DB3PR10MB6835F0ECA792265FA41FC39BE8A3A@DB3PR10MB6835.EURPRD10.PROD.OUTLOOK.COM
Signed-off-by: Alexei Starovoitov <[email protected]>

Compiler complains about malloc(). We also don't need to dynamically
allocate anything, so make the life easier by using statically sized
buffer.

Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Some compilers complain about get_pprint_mapv_size() not returning value
in some code paths. Fix with explicit return.

Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Add smin/smax derivation from appropriate umin/umax values. Previously the
logic was surprisingly asymmetric, trying to derive umin/umax from smin/smax
(if possible), but not trying to do the same in the other direction. A simple
addition to __reg64_deduce_bounds() fixes this.

Added also generic comment about u64/s64 ranges and their relationship.
Hopefully that helps readers to understand all the bounds deductions
a bit better.

Acked-by: Eduard Zingerman <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

All the logic that applies to u64 vs s64, equally applies for u32 vs s32
relationships (just taken in a smaller 32-bit numeric space). So do the
same deduction of smin32/smax32 from umin32/umax32, if we can.

Acked-by: Eduard Zingerman <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

…stant

Comments in code try to explain the idea behind why this is correct.
Please check the code and comments.

Acked-by: Eduard Zingerman <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Add a special case where we can derive valid s32 bounds from umin/umax
or smin/smax by stitching together negative s32 subrange and
non-negative s32 subrange. That requires upper 32 bits to form a [N, N+1]
range in u32 domain (taking into account wrap around, so 0xffffffff
to 0x00000000 is a valid [N, N+1] range in this sense). See code comment
for concrete examples.

Eduard Zingerman also provided an alternative explanation ([0]) for more
mathematically inclined readers:

Suppose:
. there are numbers a, b, c
. 2**31 <= b < 2**32
. 0 <= c < 2**31
. umin = 2**32 * a + b
. umax = 2**32 * (a + 1) + c

The number of values in the range represented by [umin; umax] is:
. N = umax - umin + 1 = 2**32 + c - b + 1
. min(N) = 2**32 + 0 - (2**32-1) + 1 = 2, with b = 2**32-1, c = 0
. max(N) = 2**32 + (2**31 - 1) - 2**31 + 1 = 2**32, with b = 2**31, c = 2**31-1

Hence [(s32)b; (s32)c] forms a valid range.

  [0] https://lore.kernel.org/bpf/[email protected]/

Acked-by: Eduard Zingerman <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Add a few interesting cases in which we can tighten 64-bit bounds based
on newly learnt information about 32-bit bounds. E.g., when full u64/s64
registers are used in BPF program, and then eventually compared as
u32/s32. The latter comparison doesn't change the value of full
register, but it does impose new restrictions on possible lower 32 bits
of such full registers. And we can use that to derive additional full
register bounds information.

Acked-by: Eduard Zingerman <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

There are cases (caught by subsequent reg_bounds tests in selftests/bpf)
where performing one round of __reg_deduce_bounds() doesn't propagate
all the information from, say, s32 to u32 bounds and than from newly
learned u32 bounds back to u64 and s64. So perform __reg_deduce_bounds()
twice to make sure such derivations are propagated fully after
reg_bounds_sync().

One such example is test `(s64)[0xffffffff00000001; 0] (u64)<
0xffffffff00000000` from selftest patch from this patch set. It demonstrates an
intricate dance of u64 -> s64 -> u64 -> u32 bounds adjustments, which requires
two rounds of __reg_deduce_bounds(). Here are corresponding refinement log from
selftest, showing evolution of knowledge.

REFINING (FALSE R1) (u64)SRC=[0xffffffff00000000; U64_MAX] (u64)DST_OLD=[0; U64_MAX] (u64)DST_NEW=[0xffffffff00000000; U64_MAX]
REFINING (FALSE R1) (u64)SRC=[0xffffffff00000000; U64_MAX] (s64)DST_OLD=[0xffffffff00000001; 0] (s64)DST_NEW=[0xffffffff00000001; -1]
REFINING (FALSE R1) (s64)SRC=[0xffffffff00000001; -1] (u64)DST_OLD=[0xffffffff00000000; U64_MAX] (u64)DST_NEW=[0xffffffff00000001; U64_MAX]
REFINING (FALSE R1) (u64)SRC=[0xffffffff00000001; U64_MAX] (u32)DST_OLD=[0; U32_MAX] (u32)DST_NEW=[1; U32_MAX]

R1 initially has smin/smax set to [0xffffffff00000001; -1], while umin/umax is
unknown. After (u64)< comparison, in FALSE branch we gain knowledge that
umin/umax is [0xffffffff00000000; U64_MAX]. That causes smin/smax to learn that
zero can't happen and upper bound is -1. Then smin/smax is adjusted from
umin/umax improving lower bound from 0xffffffff00000000 to 0xffffffff00000001.
And then eventually umin32/umax32 bounds are drived from umin/umax and become
[1; U32_MAX].

Selftest in the last patch is actually implementing a multi-round fixed-point
convergence logic, but so far all the tests are handled by two rounds of
reg_bounds_sync() on the verifier state, so we keep it simple for now.

Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

When performing 32-bit conditional operation operating on lower 32 bits
of a full 64-bit register, register full value isn't changed. We just
potentially gain new knowledge about that register's lower 32 bits.

Unfortunately, __reg_combine_{32,64}_into_{64,32} logic that
reg_set_min_max() performs as a last step, can lose information in some
cases due to __mark_reg64_unbounded() and __reg_assign_32_into_64().
That's bad and completely unnecessary. Especially __reg_assign_32_into_64()
looks completely out of place here, because we are not performing
zero-extending subregister assignment during conditional jump.

So this patch replaced __reg_combine_* with just a normal
reg_bounds_sync() which will do a proper job of deriving u64/s64 bounds
from u32/s32, and vice versa (among all other combinations).

__reg_combine_64_into_32() is also used in one more place,
coerce_reg_to_size(), while handling 1- and 2-byte register loads.
Looking into this, it seems like besides marking subregister as
unbounded before performing reg_bounds_sync(), we were also performing
deduction of smin32/smax32 and umin32/umax32 bounds from respective
smin/smax and umin/umax bounds. It's now redundant as reg_bounds_sync()
performs all the same logic more generically (e.g., without unnecessary
assumption that upper 32 bits of full register should be zero).

Long story short, we remove __reg_combine_64_into_32() completely, and
coerce_reg_to_size() now only does resetting subreg to unbounded and then
performing reg_bounds_sync() to recover as much information as possible
from 64-bit umin/umax and smin/smax bounds, set explicitly in
coerce_reg_to_size() earlier.

Acked-by: Eduard Zingerman <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Just taking mundane refactoring bits out into a separate patch. No
functional changes.

Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

While still assuming that second register is a constant, generalize
is_branch_taken-related code to accept two registers instead of register
plus explicit constant value. This also, as a side effect, allows to
simplify check_cond_jmp_op() by unifying BPF_K case with BPF_X case, for
which we use a fake register to represent BPF_K's imm constant as
a register.

Acked-by: Eduard Zingerman <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: Shung-Hsi Yu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Move is_branch_taken() slightly down. In subsequent patched we'll need
both flip_opcode() and is_pkt_ptr_branch_taken() for is_branch_taken(),
but instead of sprinkling forward declarations around, it makes more
sense to move is_branch_taken() lower below is_pkt_ptr_branch_taken(),
and also keep it closer to very tightly related reg_set_min_max(), as
they are two critical parts of the same SCALAR range tracking logic.

Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

…e place

Make is_branch_taken() a single entry point for branch pruning decision
making, handling both pointer vs pointer, pointer vs scalar, and scalar
vs scalar cases in one place. This also nicely cleans up check_cond_jmp_op().

Acked-by: Eduard Zingerman <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Combine 32-bit and 64-bit is_branch_taken logic for SCALAR_VALUE
registers. It makes it easier to see parallels between two domains
(32-bit and 64-bit), and makes subsequent refactoring more
straightforward.

No functional changes.

Acked-by: Eduard Zingerman <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Similarly to is_branch_taken()-related refactorings, start preparing
reg_set_min_max() to handle more generic case of two non-const
registers. Start with renaming arguments to accommodate later addition
of second register as an input argument.

Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Change reg_set_min_max() to take FALSE/TRUE sets of two registers each,
instead of assuming that we are always comparing to a constant. For now
we still assume that right-hand side registers are constants (and make
sure that's the case by swapping src/dst regs, if necessary), but
subsequent patches will remove this limitation.

reg_set_min_max() is now called unconditionally for any register
comparison, so that might include pointer vs pointer. This makes it
consistent with is_branch_taken() generality. But we currently only
support adjustments based on SCALAR vs SCALAR comparisons, so
reg_set_min_max() has to guard itself againts pointers.

Taking two by two registers allows to further unify and simplify
check_cond_jmp_op() logic. We utilize fake register for BPF_K
conditional jump case, just like with is_branch_taken() part.

Acked-by: Eduard Zingerman <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

Andrii Nakryiko says:

====================
BPF register bounds logic and testing improvements

This patch set adds a big set of manual and auto-generated test cases
validating BPF verifier's register bounds tracking and deduction logic. See
details in the last patch.

We start with building a tester that validates existing <range> vs <scalar>
verifier logic for range bounds. To make all this work, BPF verifier's logic
needed a bunch of improvements to handle some cases that previously were not
covered. This had no implications as to correctness of verifier logic, but it
was incomplete enough to cause significant disagreements with alternative
implementation of register bounds logic that tests in this patch set
implement. So we need BPF verifier logic improvements to make all the tests
pass. This is what we do in patches #3 through #9.

The end goal of this work, though, is to extend BPF verifier range state
tracking such as to allow to derive new range bounds when comparing non-const
registers. There is some more investigative work required to investigate and
fix existing potential issues with range tracking as part of ALU/ALU64
operations, so <range> x <range> part of v5 patch set ([0]) is dropped until
these issues are sorted out.

For now, we include preparatory refactorings and clean ups, that set up BPF
verifier code base to extend the logic to <range> vs <range> logic in
subsequent patch set. Patches #10-#16 perform preliminary refactorings without
functionally changing anything. But they do clean up check_cond_jmp_op() logic
and generalize a bunch of other pieces in is_branch_taken() logic.

  [0] https://patchwork.kernel.org/project/netdevbpf/list/?series=797178&state=*

v5->v6:
  - dropped <range> vs <range> patches (original patches #18 through #23) to
    add more register range sanity checks and fix preexisting issues;
  - comments improvements, addressing other feedback on first 17 patches
    (Eduard, Alexei);
v4->v5:
  - added entirety of verifier reg bounds tracking changes, now handling
    <range> vs <range> cases (Alexei);
  - added way more comments trying to explain why deductions added are
    correct, hopefully they are useful and clarify things a bit (Daniel,
    Shung-Hsi);
  - added two preliminary selftests fixes necessary for RELEASE=1 build to
    work again, it keeps breaking.
v3->v4:
  - improvements to reg_bounds tester (progress report, split 32-bit and
    64-bit ranges, fix various verbosity output issues, etc);
v2->v3:
  - fix a subtle little-endianness assumption inside parge_reg_state() (CI);
v1->v2:
  - fix compilation when building selftests with llvm-16 toolchain (CI).
====================

Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>

When updating per-cpu map in map_percpu_stats test, patch_map_thread()
only passes 4-bytes-sized value to bpf_map_update_elem(). The expected
size of the value is 8 * num_possible_cpus(), so fix it by passing a
value with enough-size for per-cpu map update.

Signed-off-by: Hou Tao <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Export map_update_retriable() to make it usable for other map_test
cases. These cases may only need retry for specific errno, so add
a new callback parameter to let map_update_retriable() decide whether or
not the errno is retriable.

Signed-off-by: Hou Tao <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

BPF CI failed due to map_percpu_stats_percpu_hash from time to time [1].
It seems that the failure reason is per-cpu bpf memory allocator may not
be able to allocate per-cpu pointer successfully and it can not refill
free llist timely, and bpf_map_update_elem() will return -ENOMEM.

So mitigate the problem by retrying the update operation for
non-preallocated per-cpu map.

[1]: https://github.com/kernel-patches/bpf/actions/runs/6713177520/job/18244865326?pr=5909

Signed-off-by: Hou Tao <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Hou Tao says:

====================
List-Subscribe: <mailto:[email protected]>
List-Unsubscribe: <mailto:[email protected]>
MIME-Version: 1.0
X-CM-TRANSID: gCh0CgCHHt6+xEFlSpMJEg--.58519S4
X-Coremail-Antispam: 1UD129KBjvJXoW7Jr1UJrW3JFyUXw4rJrWxCrg_yoW8JrW5pF
	WrK3WrKrZ7tryaqw13tanrW3yrtrs5W3WjkF13tr4YvF1UJ34xKr48KF1jgrZxCrZYqr1a
	yay8tF1xWa1xZrUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2
	9KBjDU0xBIdaVrnRJUUUk2b4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2
	6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rwA2F7IY1VAKz4
	vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7Cj
	xVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x
	0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG
	6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFV
	Cjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4IIrI8v6xkF7I0E8cxan2IY04v7MxAIw28I
	cxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2
	IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x0EwIxGrwCI
	42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwCI42
	IY6xAIw20EY4v20xvaj40_WFyUJVCq3wCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E
	87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjxUrR6zUUUUU
X-CM-SenderInfo: xkrx3t3r6k3tpzhluzxrxghudrp/
X-Patchwork-Delegate: [email protected]

From: Hou Tao <[email protected]>

Hi,

BPF CI failed due to map_percpu_stats_percpu_hash from time to time [1].
It seems that the failure reason is per-cpu bpf memory allocator may not
be able to allocate per-cpu pointer successfully and it can not refill
free llist timely, and bpf_map_update_elem() will return -ENOMEM.

Patch #1 fixes the size of value passed to per-cpu map update API. The
problem was found when fixing the ENOMEM problem, so also post it in
this patchset. Patch #2 & #3 mitigates the ENOMEM problem by retrying
the update operation for non-preallocated per-cpu map.

Please see individual patches for more details. And comments are always
welcome.

Regards,
Tao

[1]: https://github.com/kernel-patches/bpf/actions/runs/6713177520/job/18244865326?pr=5909
====================

Signed-off-by: Andrii Nakryiko <[email protected]>

Those configs are needed to be able to run VM somewhat consistently.
For instance, ATM, s390x is missing the `CONFIG_VIRTIO_CONSOLE` which
prevents s390x kernels built in CI to leverage qemu-guest-agent.

By moving them to `config,vm`, we should have selftest kernels which are
equal in term of VM functionalities when they include this file.

The set of config unabled were picked using

    grep -h -E '(_9P|_VIRTIO)' config.x86_64 config | sort | uniq

added to `config.vm` and then
    grep -vE '(_9P|_VIRTIO)' config.{x86_64,aarch64,s390x}

as a side-effect, some config may have disappeared to the aarch64 and
s390x kernels, but they should not be needed. CI will tell.

Signed-off-by: Manu Bretelle <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

…ges.

With the recent Makefile refactor that puts all build artifacts into a
build/ directory output, there was a regression in that Make would now
always rebuild schedulers even if they were unchanged. This is happening
because when Make looks at a target, it looks to see if that file
exists. If it doesn't, it executes the target. There are a few targets
that are improperly tracked:

1. We were taking a dependency on the sched.skel.h target (e.g.
   scx_simple.skel.h). In the old build system this was an actual file,
   but now it's just a target as the target name was never updated to
   point to the full path to the include file output.

2. The same goes for sched.bpf.o, which is a dependency of the skel
   file.

3. The scheduler itself, which now resides in build/bin.

The first two we can fix by updating the targets to include the build
directories. The latter we'll have to fix with some more complex Make
magic, which we'll do in the subsequent commit.

Signed-off-by: David Vernet <[email protected]>

Now that the scheduler binaries are written to the build/bin/ directory,
Make gets confused because it doesn't see the binary file in the same
directory anymore and tries to rebuild it. This makes things kind of
tricky, because make will always execute the recipe for the target,
which is to compile it.

We could add a layer of indirection by instead having the base scheduler
target be empty, and just take a dependency on the actual binary that's
created the compiler. This patch does that, and also cleans up the build
to avoid copy-pasted scheduler recipes.

Signed-off-by: David Vernet <[email protected]>

scx_rusty currently defines several build targets and recipes that would
have to be duplicated by any other rust scheduler we may add. Let's add
some build scaffolding to avoid people having to copy paste.

Note that we can't fully avoid running any make logic if we take the
same approach as with the C schedulers. The C schedulers add a layer of
indirection where the "base" target (e.g. scx_simple) do nothing but
take a dependency on the binary output file. This doesn't work with rust
schedulers, because we're relying on Cargo to tell us when it needs to
be rebuilt.

Signed-off-by: David Vernet <[email protected]>

bpftool's man page lists "program" as one of possible values for OBJECT,
while in fact bpftool accepts "prog" instead.

Reported-by: Jerry Snitselaar <[email protected]>
Signed-off-by: Artem Savkov <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: Yonghong Song <[email protected]>
Acked-by: Quentin Monnet <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Fix Makefile dependency tracking

…on.bpf.h