kernel/linux.git/arch/arm/net, branch v6.6.141

bpf, arm32: Reject BPF-to-BPF calls and callbacks in the JIT

2026-05-23T11:03:19+00:00

[ Upstream commit e1d486445af3c392628532229f7ce5f5cf7891b6 ] The ARM32 BPF JIT does not support BPF-to-BPF function calls (BPF_PSEUDO_CALL) or callbacks (BPF_PSEUDO_FUNC), but it does not reject them either. When a program with subprograms is loaded (e.g. libxdp's XDP dispatcher uses __noinline__ subprograms, or any program using callbacks like bpf_loop or bpf_for_each_map_elem), the verifier invokes bpf_jit_subprogs() which calls bpf_int_jit_compile() for each subprogram. For BPF_PSEUDO_CALL, since ARM32 does not reject it, the JIT silently emits code using the wrong address computation: func = __bpf_call_base + imm where imm is a pc-relative subprogram offset, producing a bogus function pointer. For BPF_PSEUDO_FUNC, the ldimm64 handler ignores src_reg and loads the immediate as a normal 64-bit value without error. In both cases, build_body() reports success and a JIT image is allocated. ARM32 lacks the jit_data/extra_pass mechanism needed for the second JIT pass in bpf_jit_subprogs(). On the second pass, bpf_int_jit_compile() performs a full fresh compilation, allocating a new JIT binary and overwriting prog->bpf_func. The first allocation is never freed. bpf_jit_subprogs() then detects the function pointer changed and aborts with -ENOTSUPP, but the original JIT binary has already been leaked. Each program load/unload cycle leaks one JIT binary allocation, as reported by kmemleak: unreferenced object 0xbf0a1000 (size 4096): backtrace: bpf_jit_binary_alloc+0x64/0xfc bpf_int_jit_compile+0x14c/0x348 bpf_jit_subprogs+0x4fc/0xa60 Fix this by rejecting both BPF_PSEUDO_CALL in the BPF_CALL handler and BPF_PSEUDO_FUNC in the BPF_LD_IMM64 handler, falling through to the existing 'notyet' path. This causes build_body() to fail before any JIT binary is allocated, so bpf_int_jit_compile() returns the original program unjitted. bpf_jit_subprogs() then sees !prog->jited and cleanly falls back to the interpreter with no leak. Acked-by: Daniel Borkmann Fixes: 1c2a088a6626 ("bpf: x64: add JIT support for multi-function programs") Reported-by: Jonas Rebmann Closes: https://lore.kernel.org/bpf/b63e9174-7a3d-4e22-8294-16df07a4af89@pengutronix.de Tested-by: Jonas Rebmann Signed-off-by: Puranjay Mohan Reviewed-by: Emil Tsalapatis Link: https://lore.kernel.org/r/20260417143353.838911-1-puranjay@kernel.org Signed-off-by: Alexei Starovoitov Signed-off-by: Sasha Levin

Revert "bpf: Take return from set_memory_rox() into account with bpf_jit_binary_lock_ro()"

2024-07-09T09:44:29+00:00

This reverts commit 08f6c05feb1db21653e98ca84ea04ca032d014c7 which is commit e60adf513275c3a38e5cb67f7fd12387e43a3ff5 upstream. It is part of a series that is reported to both break the arm64 builds and instantly crashes the powerpc systems at the first load of a bpf program. So revert it for now until it can come back in a safe way. Reported-by: matoro Reported-by: Vitaly Chikunov Reported-by: WangYuli Link: https://lore.kernel.org/r/5A29E00D83AB84E3+20240706031101.637601-1-wangyuli@uniontech.com Link: https://lore.kernel.org/r/cf736c5e37489e7dc7ffd67b9de2ab47@matoro.tk Cc: Hari Bathini Cc: Song Liu Cc: Michael Ellerman Cc: Christophe Leroy Cc: Kees Cook Cc: Puranjay Mohan Cc: Ilya Leoshkevich # s390x Cc: Tiezhu Yang # LoongArch Cc: Johan Almbladh # MIPS Part Cc: Alexei Starovoitov Cc: Sasha Levin Signed-off-by: Greg Kroah-Hartman

bpf: Take return from set_memory_rox() into account with bpf_jit_binary_lock_ro()

2024-07-05T07:33:50+00:00

[ Upstream commit e60adf513275c3a38e5cb67f7fd12387e43a3ff5 ] set_memory_rox() can fail, leaving memory unprotected. Check return and bail out when bpf_jit_binary_lock_ro() returns an error. Link: https://github.com/KSPP/linux/issues/7 Signed-off-by: Christophe Leroy Cc: linux-hardening@vger.kernel.org Reviewed-by: Kees Cook Reviewed-by: Puranjay Mohan Reviewed-by: Ilya Leoshkevich # s390x Acked-by: Tiezhu Yang # LoongArch Reviewed-by: Johan Almbladh # MIPS Part Message-ID: <036b6393f23a2032ce75a1c92220b2afcb798d5d.1709850515.git.christophe.leroy@csgroup.eu> Signed-off-by: Alexei Starovoitov Signed-off-by: Sasha Levin

bpf, arm: Remove unused function emit_a32_alu_r()

2022-06-14T20:04:16+00:00

Since commit b18bea2a45b1 ("ARM: net: bpf: improve 64-bit ALU implementation") this is unused anymore, so can remove it. Signed-off-by: YueHaibing Signed-off-by: Daniel Borkmann Link: https://lore.kernel.org/bpf/20220611040904.8976-1-yuehaibing@huawei.com

bpf, arm: Fix various typos in comments

2022-03-21T15:20:26+00:00

Various spelling mistakes in comments. Detected with the help of Coccinelle. Signed-off-by: Julia Lawall Signed-off-by: Daniel Borkmann Link: https://lore.kernel.org/bpf/20220318103729.157574-9-Julia.Lawall@inria.fr

bpf, docs: Prune all references to "internal BPF"

2021-11-30T18:52:11+00:00

The eBPF name has completely taken over from eBPF in general usage for the actual eBPF representation, or BPF for any general in-kernel use. Prune all remaining references to "internal BPF". Signed-off-by: Christoph Hellwig Signed-off-by: Alexei Starovoitov Acked-by: Song Liu Link: https://lore.kernel.org/bpf/20211119163215.971383-4-hch@lst.de

bpf: Change value of MAX_TAIL_CALL_CNT from 32 to 33

2021-11-16T13:03:15+00:00

In the current code, the actual max tail call count is 33 which is greater than MAX_TAIL_CALL_CNT (defined as 32). The actual limit is not consistent with the meaning of MAX_TAIL_CALL_CNT and thus confusing at first glance. We can see the historical evolution from commit 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs") and commit f9dabe016b63 ("bpf: Undo off-by-one in interpreter tail call count limit"). In order to avoid changing existing behavior, the actual limit is 33 now, this is reasonable. After commit 874be05f525e ("bpf, tests: Add tail call test suite"), we can see there exists failed testcase. On all archs when CONFIG_BPF_JIT_ALWAYS_ON is not set: # echo 0 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf # dmesg | grep -w FAIL Tail call error path, max count reached jited:0 ret 34 != 33 FAIL On some archs: # echo 1 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf # dmesg | grep -w FAIL Tail call error path, max count reached jited:1 ret 34 != 33 FAIL Although the above failed testcase has been fixed in commit 18935a72eb25 ("bpf/tests: Fix error in tail call limit tests"), it would still be good to change the value of MAX_TAIL_CALL_CNT from 32 to 33 to make the code more readable. The 32-bit x86 JIT was using a limit of 32, just fix the wrong comments and limit to 33 tail calls as the constant MAX_TAIL_CALL_CNT updated. For the mips64 JIT, use "ori" instead of "addiu" as suggested by Johan Almbladh. For the riscv JIT, use RV_REG_TCC directly to save one register move as suggested by Björn Töpel. For the other implementations, no function changes, it does not change the current limit 33, the new value of MAX_TAIL_CALL_CNT can reflect the actual max tail call count, the related tail call testcases in test_bpf module and selftests can work well for the interpreter and the JIT. Here are the test results on x86_64: # uname -m x86_64 # echo 0 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf test_suite=test_tail_calls # dmesg | tail -1 test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [0/8 JIT'ed] # rmmod test_bpf # echo 1 > /proc/sys/net/core/bpf_jit_enable # modprobe test_bpf test_suite=test_tail_calls # dmesg | tail -1 test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed] # rmmod test_bpf # ./test_progs -t tailcalls #142 tailcalls:OK Summary: 1/11 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Tiezhu Yang Signed-off-by: Daniel Borkmann Tested-by: Johan Almbladh Tested-by: Ilya Leoshkevich Acked-by: Björn Töpel Acked-by: Johan Almbladh Acked-by: Ilya Leoshkevich Link: https://lore.kernel.org/bpf/1636075800-3264-1-git-send-email-yangtiezhu@loongson.cn

Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

2021-11-02T02:59:46+00:00

Alexei Starovoitov says: ==================== pull-request: bpf-next 2021-11-01 We've added 181 non-merge commits during the last 28 day(s) which contain a total of 280 files changed, 11791 insertions(+), 5879 deletions(-). The main changes are: 1) Fix bpf verifier propagation of 64-bit bounds, from Alexei. 2) Parallelize bpf test_progs, from Yucong and Andrii. 3) Deprecate various libbpf apis including af_xdp, from Andrii, Hengqi, Magnus. 4) Improve bpf selftests on s390, from Ilya. 5) bloomfilter bpf map type, from Joanne. 6) Big improvements to JIT tests especially on Mips, from Johan. 7) Support kernel module function calls from bpf, from Kumar. 8) Support typeless and weak ksym in light skeleton, from Kumar. 9) Disallow unprivileged bpf by default, from Pawan. 10) BTF_KIND_DECL_TAG support, from Yonghong. 11) Various bpftool cleanups, from Quentin. * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (181 commits) libbpf: Deprecate AF_XDP support kbuild: Unify options for BTF generation for vmlinux and modules selftests/bpf: Add a testcase for 64-bit bounds propagation issue. bpf: Fix propagation of signed bounds from 64-bit min/max into 32-bit. bpf: Fix propagation of bounds from 64-bit min/max into 32-bit and var_off. selftests/bpf: Fix also no-alu32 strobemeta selftest bpf: Add missing map_delete_elem method to bloom filter map selftests/bpf: Add bloom map success test for userspace calls bpf: Add alignment padding for "map_extra" + consolidate holes bpf: Bloom filter map naming fixups selftests/bpf: Add test cases for struct_ops prog bpf: Add dummy BPF STRUCT_OPS for test purpose bpf: Factor out helpers for ctx access checking bpf: Factor out a helper to prepare trampoline for struct_ops prog selftests, bpf: Fix broken riscv build riscv, libbpf: Add RISC-V (RV64) support to bpf_tracing.h tools, build: Add RISC-V to HOSTARCH parsing riscv, bpf: Increase the maximum number of iterations selftests, bpf: Add one test for sockmap with strparser selftests, bpf: Fix test_txmsg_ingress_parser error ... ==================== Link: https://lore.kernel.org/r/20211102013123.9005-1-alexei.starovoitov@gmail.com Signed-off-by: Jakub Kicinski

bpf, arm: Remove dummy bpf_jit_compile stub

2021-10-06T19:34:02+00:00

The BPF core defines a __weak bpf_jit_compile() dummy function already which should only be overridden by JITs if they actually implement a legacy cBPF JIT. Given arm implements an eBPF JIT, this stub is not needed. Now that MIPS cBPF JIT is finally gone, the only JIT left that is still implementing bpf_jit_compile() is the sparc32 one. Signed-off-by: Daniel Borkmann Signed-off-by: Andrii Nakryiko

bpf, arm: Fix register clobbering in div/mod implementation

2021-09-29T14:04:50+00:00

On ARM CPUs that lack div/mod instructions, ALU32 BPF_DIV and BPF_MOD are implemented using a call to a helper function. Before, the emitted code for those function calls failed to preserve caller-saved ARM registers. Since some of those registers happen to be mapped to BPF registers, it resulted in eBPF register values being overwritten. This patch emits code to push and pop the remaining caller-saved ARM registers r2-r3 into the stack during the div/mod function call. ARM registers r0-r1 are used as arguments and return value, and those were already saved and restored correctly. Fixes: 39c13c204bb1 ("arm: eBPF JIT compiler") Signed-off-by: Johan Almbladh Signed-off-by: Daniel Borkmann