summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)AuthorFilesLines
2023-01-25bpf: Allow BPF_PROG_TYPE_STRUCT_OPS programs to be sleepableDavid Vernet1-1/+1
BPF struct_ops programs currently cannot be marked as sleepable. This need not be the case -- struct_ops programs can be sleepable, and e.g. invoke kfuncs that export the KF_SLEEPABLE flag. So as to allow future struct_ops programs to invoke such kfuncs, this patch updates the verifier to allow struct_ops programs to be sleepable. A follow-on patch will add support to libbpf for specifying struct_ops.s as a sleepable struct_ops program, and then another patch will add testcases to the dummy_st_ops selftest suite which test sleepable struct_ops behavior. Signed-off-by: David Vernet <void@manifault.com> Link: https://lore.kernel.org/r/20230125164735.785732-2-void@manifault.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-25selftests/bpf: Fix vmtest static compilation errorDaniel T. Lee1-2/+3
As stated in README.rst, in order to resolve errors with linker errors, 'LDLIBS=-static' should be used. Most problems will be solved by this option, but in the case of urandom_read, this won't fix the problem. So the Makefile is currently implemented to strip the 'static' option when compiling the urandom_read. However, stripping this static option isn't configured properly on $(LDLIBS) correctly, which is now causing errors on static compilation. # LDLIBS=-static ./vmtest.sh ld.lld: error: attempted static link of dynamic object liburandom_read.so clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [Makefile:190: /linux/tools/testing/selftests/bpf/urandom_read] Error 1 make: *** Waiting for unfinished jobs.... This commit fixes this problem by configuring the strip with $(LDLIBS). Fixes: 68084a136420 ("selftests/bpf: Fix building bpf selftests statically") Signed-off-by: Daniel T. Lee <danieltimlee@gmail.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230125100440.21734-1-danieltimlee@gmail.com
2023-01-25tools/resolve_btfids: Alter how HOSTCC is forcedIan Rogers1-10/+7
HOSTCC is always wanted when building. Setting CC to HOSTCC happens after tools/scripts/Makefile.include is included, meaning flags are set assuming say CC is gcc, but then it can be later set to HOSTCC which may be clang. tools/scripts/Makefile.include is needed for host set up and common macros in objtool's Makefile. Rather than override CC to HOSTCC, just pass CC as HOSTCC to Makefile.build, the libsubcmd builds and the linkage step. This means the Makefiles don't see things like CC changing and tool flag determination, and similar, work properly. Also, clear the passed subdir as otherwise an outer build may break by inadvertently passing an inappropriate value. Signed-off-by: Ian Rogers <irogers@google.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20230124064324.672022-2-irogers@google.com
2023-01-25tools/resolve_btfids: Install subcmd headersIan Rogers2-6/+15
Previously tools/lib/subcmd was added to the include path, switch to installing the headers and then including from that directory. This avoids dependencies on headers internal to tools/lib/subcmd. Add the missing subcmd directory to the affected #include. Signed-off-by: Ian Rogers <irogers@google.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20230124064324.672022-1-irogers@google.com
2023-01-25selftests/bpf: Add selftest suite for cpumask kfuncsDavid Vernet5-0/+741
A recent patch added a new set of kfuncs for allocating, freeing, manipulating, and querying cpumasks. This patch adds a new 'cpumask' selftest suite which verifies their behavior. Signed-off-by: David Vernet <void@manifault.com> Link: https://lore.kernel.org/r/20230125143816.721952-5-void@manifault.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-25selftests/bpf: Add nested trust selftests suiteDavid Vernet5-0/+77
Now that defining trusted fields in a struct is supported, we should add selftests to verify the behavior. This patch adds a few such testcases. Signed-off-by: David Vernet <void@manifault.com> Link: https://lore.kernel.org/r/20230125143816.721952-4-void@manifault.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-25bpf: Disallow NULLable pointers for trusted kfuncsDavid Vernet2-4/+4
KF_TRUSTED_ARGS kfuncs currently have a subtle and insidious bug in validating pointers to scalars. Say that you have a kfunc like the following, which takes an array as the first argument: bool bpf_cpumask_empty(const struct cpumask *cpumask) { return cpumask_empty(cpumask); } ... BTF_ID_FLAGS(func, bpf_cpumask_empty, KF_TRUSTED_ARGS) ... If a BPF program were to invoke the kfunc with a NULL argument, it would crash the kernel. The reason is that struct cpumask is defined as a bitmap, which is itself defined as an array, and is accessed as a memory address by bitmap operations. So when the verifier analyzes the register, it interprets it as a pointer to a scalar struct, which is an array of size 8. check_mem_reg() then sees that the register is NULL and returns 0, and the kfunc crashes when it passes it down to the cpumask wrappers. To fix this, this patch adds a check for KF_ARG_PTR_TO_MEM which verifies that the register doesn't contain a possibly-NULL pointer if the kfunc is KF_TRUSTED_ARGS. Signed-off-by: David Vernet <void@manifault.com> Link: https://lore.kernel.org/r/20230125143816.721952-2-void@manifault.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-23libbpf: Clean up now not needed __PT_PARM{1-6}_SYSCALL_REG defaultsAndrii Nakryiko1-18/+0
Each architecture supports at least 6 syscall argument registers, so now that specs for each architecture is defined in bpf_tracing.h, remove unnecessary macro overrides, which previously were required to keep existing BPF_KSYSCALL() uses compiling and working. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-26-andrii@kernel.org
2023-01-23selftests/bpf: Add 6-argument syscall tracing testAndrii Nakryiko2-0/+43
Turns out splice() is one of the syscalls that's using current maximum number of arguments (six). This is perfect for testing, so extend bpf_syscall_macro selftest to also trace splice() syscall, using BPF_KSYSCALL() macro. This makes sure all the syscall argument register definitions are correct. Suggested-by: Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> # arm64 Tested-by: Ilya Leoshkevich <iii@linux.ibm.com> # s390x Link: https://lore.kernel.org/bpf/20230120200914.3008030-25-andrii@kernel.org
2023-01-23libbpf: Define loongarch syscall regs spec in bpf_tracing.hAndrii Nakryiko1-2/+10
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-24-andrii@kernel.org
2023-01-23libbpf: Define arc syscall regs spec in bpf_tracing.hAndrii Nakryiko1-2/+10
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-23-andrii@kernel.org
2023-01-23libbpf: Define riscv syscall regs spec in bpf_tracing.hAndrii Nakryiko1-2/+10
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Pu Lehui <pulehui@huawei.com> # RISC-V Link: https://lore.kernel.org/bpf/20230120200914.3008030-22-andrii@kernel.org
2023-01-23libbpf: Define sparc syscall regs spec in bpf_tracing.hAndrii Nakryiko1-0/+8
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-21-andrii@kernel.org
2023-01-23libbpf: Define powerpc syscall regs spec in bpf_tracing.hAndrii Nakryiko1-2/+13
Define explicit table of registers used for syscall argument passing. Note that 7th arg is supported on 32-bit powerpc architecture, by not on powerpc64. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-20-andrii@kernel.org
2023-01-23libbpf: Define mips syscall regs spec in bpf_tracing.hAndrii Nakryiko1-0/+8
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-19-andrii@kernel.org
2023-01-23libbpf: Define arm64 syscall regs spec in bpf_tracing.hAndrii Nakryiko1-2/+11
Define explicit table of registers used for syscall argument passing. We need PT_REGS_PARM1_[CORE_]SYSCALL macros overrides, similarly to s390x, due to orig_x0 not being present in UAPI's pt_regs, so we need to utilize BPF CO-RE and custom pt_regs___arm64 definition. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> # arm64 Link: https://lore.kernel.org/bpf/20230120200914.3008030-18-andrii@kernel.org
2023-01-23libbpf: Define arm syscall regs spec in bpf_tracing.hAndrii Nakryiko1-0/+8
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-17-andrii@kernel.org
2023-01-23libbpf: Define s390x syscall regs spec in bpf_tracing.hAndrii Nakryiko1-2/+15
Define explicit table of registers used for syscall argument passing. Note that we need custom overrides for PT_REGS_PARM1_[CORE_]SYSCALL macros due to the need to use BPF CO-RE and custom local pt_regs definitions to fetch orig_gpr2, storing 1st argument. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Ilya Leoshkevich <iii@linux.ibm.com> # s390x Acked-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/bpf/20230120200914.3008030-16-andrii@kernel.org
2023-01-23libbpf: Define i386 syscall regs spec in bpf_tracing.hAndrii Nakryiko1-0/+8
Define explicit table of registers used for syscall argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-15-andrii@kernel.org
2023-01-23libbpf: Define x86-64 syscall regs spec in bpf_tracing.hAndrii Nakryiko1-6/+19
Define explicit table of registers used for syscall argument passing. Remove now unnecessary overrides of PT_REGS_PARM5_[CORE_]SYSCALL macros. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-14-andrii@kernel.org
2023-01-23libbpf: Improve syscall tracing support in bpf_tracing.hAndrii Nakryiko1-13/+58
Set up generic support in bpf_tracing.h for up to 7 syscall arguments tracing with BPF_KSYSCALL, which seems to be the limit according to syscall(2) manpage. Also change the way that syscall convention is specified to be more explicit. Subsequent patches will adjust and define proper per-architecture syscall conventions. __PT_PARM1_SYSCALL_REG through __PT_PARM6_SYSCALL_REG is added temporarily to keep everything working before each architecture has syscall reg tables defined. They will be removed afterwards. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> # arm64 Link: https://lore.kernel.org/bpf/20230120200914.3008030-13-andrii@kernel.org
2023-01-23selftests/bpf: Validate arch-specific argument registers limitsAndrii Nakryiko3-7/+99
Update uprobe_autoattach selftest to validate architecture-specific argument passing through registers. Use new BPF_UPROBE and BPF_URETPROBE, and construct both BPF-side and user-space side in such a way that for different architectures we are fetching and checking different number of arguments, matching architecture-specific limit of how many registers are available for argument passing. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> # arm64 Tested-by: Ilya Leoshkevich <iii@linux.ibm.com> # s390x Link: https://lore.kernel.org/bpf/20230120200914.3008030-12-andrii@kernel.org
2023-01-23libbpf: Add BPF_UPROBE and BPF_URETPROBE macro aliasesAndrii Nakryiko1-0/+7
Add BPF_UPROBE and BPF_URETPROBE macros, aliased to BPF_KPROBE and BPF_KRETPROBE, respectively. This makes uprobe-based BPF program code much less confusing, especially to people new to tracing, at no cost in terms of maintainability. We'll use this macro in selftests in subsequent patch. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-11-andrii@kernel.org
2023-01-23libbpf: Complete LoongArch (loongarch) spec in bpf_tracing.hAndrii Nakryiko1-1/+7
Add PARM6 through PARM8 definitions. Add kernel docs link describing ABI for LoongArch. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-10-andrii@kernel.org
2023-01-23libbpf: Fix and complete ARC spec in bpf_tracing.hAndrii Nakryiko1-1/+9
Add PARM6 through PARM8 definitions. Also fix frame pointer (FP) register definition. Also leave a link to where to find ABI spec. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-9-andrii@kernel.org
2023-01-23libbpf: Complete riscv arch spec in bpf_tracing.hAndrii Nakryiko1-0/+7
Add PARM6 through PARM8 definitions for RISC V (riscv) arch. Leave the link for ABI doc for future reference. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Pu Lehui <pulehui@huawei.com> # RISC-V Link: https://lore.kernel.org/bpf/20230120200914.3008030-8-andrii@kernel.org
2023-01-23libbpf: Complete sparc spec in bpf_tracing.hAndrii Nakryiko1-0/+5
Add PARM6 definition for sparc architecture. Leave a link to calling convention documentation. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-7-andrii@kernel.org
2023-01-23libbpf: Complete powerpc spec in bpf_tracing.hAndrii Nakryiko1-0/+8
Add definitions of PARM6 through PARM8 for powerpc architecture. Add also a link to a functiona call sequence documentation for future reference. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-6-andrii@kernel.org
2023-01-23libbpf: Complete mips spec in bpf_tracing.hAndrii Nakryiko1-0/+8
Add registers for PARM6 through PARM8. Add a link to an ABI. We don't distinguish between O32, N32, and N64, so document that we assume N64 right now. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-5-andrii@kernel.org
2023-01-23libbpf: Fix arm and arm64 specs in bpf_tracing.hAndrii Nakryiko1-1/+11
Remove invalid support for PARM5 on 32-bit arm, as per ABI. Add three more argument registers for arm64. Also leave links to ABI specs for future reference. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> # arm64 Link: https://lore.kernel.org/bpf/20230120200914.3008030-4-andrii@kernel.org
2023-01-23libbpf: Add 6th argument support for x86-64 in bpf_tracing.hAndrii Nakryiko1-0/+6
Add r9 as register containing 6th argument on x86-64 architecture, as per its ABI. Add also a link to a page describing ABI for easier future reference. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230120200914.3008030-3-andrii@kernel.org
2023-01-23libbpf: Add support for fetching up to 8 arguments in kprobesAndrii Nakryiko1-4/+37
Add BPF_KPROBE() and PT_REGS_PARMx() support for up to 8 arguments, if target architecture supports this. Currently all architectures are limited to only 5 register-placed arguments, which is limiting even on x86-64. This patch adds generic macro machinery to support up to 8 arguments both when explicitly fetching it from pt_regs through PT_REGS_PARMx() macros, as well as more ergonomic access in BPF_KPROBE(). Also, for i386 architecture we now don't have to define fake PARM4 and PARM5 definitions, they will be generically substituted, just like for PARM6 through PARM8. Subsequent patches will fill out architecture-specific definitions, where appropriate. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Alan Maguire <alan.maguire@oracle.com> # arm64 Tested-by: Ilya Leoshkevich <iii@linux.ibm.com> # s390x Link: https://lore.kernel.org/bpf/20230120200914.3008030-2-andrii@kernel.org
2023-01-23selftests/bpf: Use __failure macro in task kfunc testsuiteDavid Vernet2-70/+19
In commit 537c3f66eac1 ("selftests/bpf: add generic BPF program tester-loader"), a new mechanism was added to the BPF selftest framework to allow testsuites to use macros to define expected failing testcases. This allows any testsuite which tests verification failure to remove a good amount of boilerplate code. This patch updates the task_kfunc selftest suite to use these new macros. Signed-off-by: David Vernet <void@manifault.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20230120021844.3048244-1-void@manifault.com
2023-01-23selftests/bpf: Simple program to dump XDP RX metadataStanislav Fomichev4-1/+491
To be used for verification of driver implementations. Note that the skb path is gone from the series, but I'm still keeping the implementation for any possible future work. $ xdp_hw_metadata <ifname> On the other machine: $ echo -n xdp | nc -u -q1 <target> 9091 # for AF_XDP $ echo -n skb | nc -u -q1 <target> 9092 # for skb Sample output: # xdp xsk_ring_cons__peek: 1 0x19f9090: rx_desc[0]->addr=100000000008000 addr=8100 comp_addr=8000 rx_timestamp_supported: 1 rx_timestamp: 1667850075063948829 0x19f9090: complete idx=8 addr=8000 # skb found skb hwtstamp = 1668314052.854274681 Decoding: # xdp rx_timestamp=1667850075.063948829 $ date -d @1667850075 Mon Nov 7 11:41:15 AM PST 2022 $ date Mon Nov 7 11:42:05 AM PST 2022 # skb $ date -d @1668314052 Sat Nov 12 08:34:12 PM PST 2022 $ date Sat Nov 12 08:37:06 PM PST 2022 Cc: John Fastabend <john.fastabend@gmail.com> Cc: David Ahern <dsahern@gmail.com> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Willem de Bruijn <willemb@google.com> Cc: Jesper Dangaard Brouer <brouer@redhat.com> Cc: Anatoly Burakov <anatoly.burakov@intel.com> Cc: Alexander Lobakin <alexandr.lobakin@intel.com> Cc: Magnus Karlsson <magnus.karlsson@gmail.com> Cc: Maryam Tahhan <mtahhan@redhat.com> Cc: xdp-hints@xdp-project.net Cc: netdev@vger.kernel.org Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20230119221536.3349901-18-sdf@google.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
2023-01-23selftests/bpf: Verify xdp_metadata xdp->af_xdp pathStanislav Fomichev6-1/+514
- create new netns - create veth pair (veTX+veRX) - setup AF_XDP socket for both interfaces - attach bpf to veRX - send packet via veTX - verify the packet has expected metadata at veRX Cc: John Fastabend <john.fastabend@gmail.com> Cc: David Ahern <dsahern@gmail.com> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Willem de Bruijn <willemb@google.com> Cc: Jesper Dangaard Brouer <brouer@redhat.com> Cc: Anatoly Burakov <anatoly.burakov@intel.com> Cc: Alexander Lobakin <alexandr.lobakin@intel.com> Cc: Magnus Karlsson <magnus.karlsson@gmail.com> Cc: Maryam Tahhan <mtahhan@redhat.com> Cc: xdp-hints@xdp-project.net Cc: netdev@vger.kernel.org Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20230119221536.3349901-12-sdf@google.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
2023-01-23selftests/bpf: Update expected test_offload.py messagesStanislav Fomichev1-5/+5
Generic check has a different error message, update the selftest. Cc: John Fastabend <john.fastabend@gmail.com> Cc: David Ahern <dsahern@gmail.com> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Willem de Bruijn <willemb@google.com> Cc: Jesper Dangaard Brouer <brouer@redhat.com> Cc: Anatoly Burakov <anatoly.burakov@intel.com> Cc: Alexander Lobakin <alexandr.lobakin@intel.com> Cc: Magnus Karlsson <magnus.karlsson@gmail.com> Cc: Maryam Tahhan <mtahhan@redhat.com> Cc: xdp-hints@xdp-project.net Cc: netdev@vger.kernel.org Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20230119221536.3349901-7-sdf@google.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
2023-01-23bpf: Introduce device-bound XDP programsStanislav Fomichev1-0/+5
New flag BPF_F_XDP_DEV_BOUND_ONLY plus all the infra to have a way to associate a netdev with a BPF program at load time. netdevsim checks are dropped in favor of generic check in dev_xdp_attach. Cc: John Fastabend <john.fastabend@gmail.com> Cc: David Ahern <dsahern@gmail.com> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Willem de Bruijn <willemb@google.com> Cc: Jesper Dangaard Brouer <brouer@redhat.com> Cc: Anatoly Burakov <anatoly.burakov@intel.com> Cc: Alexander Lobakin <alexandr.lobakin@intel.com> Cc: Magnus Karlsson <magnus.karlsson@gmail.com> Cc: Maryam Tahhan <mtahhan@redhat.com> Cc: xdp-hints@xdp-project.net Cc: netdev@vger.kernel.org Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20230119221536.3349901-6-sdf@google.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
2023-01-21selftests/bpf: Add dynptr helper testsKumar Kartikeya Dwivedi1-0/+192
First test that we allow overwriting dynptr slots and reinitializing them in unreferenced case, and disallow overwriting for referenced case. Include tests to ensure slices obtained from destroyed dynptrs are being invalidated on their destruction. The destruction needs to be scoped, as in slices of dynptr A should not be invalidated when dynptr B is destroyed. Next, test that MEM_UNINIT doesn't allow writing dynptr stack slots. Acked-by: Joanne Koong <joannelkoong@gmail.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-13-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21selftests/bpf: Add dynptr partial slot overwrite testsKumar Kartikeya Dwivedi1-0/+66
Try creating a dynptr, then overwriting second slot with first slot of another dynptr. Then, the first slot of first dynptr should also be invalidated, but without our fix that does not happen. As a consequence, the unfixed case allows passing first dynptr (as the kernel check only checks for slot_type and then first_slot == true). Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-12-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21selftests/bpf: Add dynptr var_off testsKumar Kartikeya Dwivedi1-0/+40
Ensure that variable offset is handled correctly, and verifier takes both fixed and variable part into account. Also ensures that only constant var_off is allowed. Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-11-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21selftests/bpf: Add dynptr pruning testsKumar Kartikeya Dwivedi1-0/+141
Add verifier tests that verify the new pruning behavior for STACK_DYNPTR slots, and ensure that state equivalence takes into account changes to the old and current verifier state correctly. Also ensure that the stacksafe changes are actually enabling pruning in case states are equivalent from pruning PoV. Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-10-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21selftests/bpf: convenience macro for use with 'asm volatile' blocksEduard Zingerman1-0/+7
A set of macros useful for writing naked BPF functions using inline assembly. E.g. as follows: struct map_struct { ... } map SEC(".maps"); SEC(...) __naked int foo_test(void) { asm volatile( "r0 = 0;" "*(u64*)(r10 - 8) = r0;" "r1 = %[map] ll;" "r2 = r10;" "r2 += -8;" "call %[bpf_map_lookup_elem];" "r0 = 0;" "exit;" : : __imm(bpf_map_lookup_elem), __imm_addr(map) : __clobber_all); } Acked-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> [ Kartikeya: Add acks, include __clobber_common from Andrii ] Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-9-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21bpf: Invalidate slices on destruction of dynptrs on stackKumar Kartikeya Dwivedi1-2/+2
The previous commit implemented destroy_if_dynptr_stack_slot. It destroys the dynptr which given spi belongs to, but still doesn't invalidate the slices that belong to such a dynptr. While for the case of referenced dynptr, we don't allow their overwrite and return an error early, we still allow it and destroy the dynptr for unreferenced dynptr. To be able to enable precise and scoped invalidation of dynptr slices in this case, we must be able to associate the source dynptr of slices that have been obtained using bpf_dynptr_data. When doing destruction, only slices belonging to the dynptr being destructed should be invalidated, and nothing else. Currently, dynptr slices belonging to different dynptrs are indistinguishible. Hence, allocate a unique id to each dynptr (CONST_PTR_TO_DYNPTR and those on stack). This will be stored as part of reg->id. Whenever using bpf_dynptr_data, transfer this unique dynptr id to the returned PTR_TO_MEM_OR_NULL slice pointer, and store it in a new per-PTR_TO_MEM dynptr_id register state member. Finally, after establishing such a relationship between dynptrs and their slices, implement precise invalidation logic that only invalidates slices belong to the destroyed dynptr in destroy_if_dynptr_stack_slot. Acked-by: Joanne Koong <joannelkoong@gmail.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-5-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21bpf: Fix partial dynptr stack slot reads/writesKumar Kartikeya Dwivedi1-3/+3
Currently, while reads are disallowed for dynptr stack slots, writes are not. Reads don't work from both direct access and helpers, while writes do work in both cases, but have the effect of overwriting the slot_type. While this is fine, handling for a few edge cases is missing. Firstly, a user can overwrite the stack slots of dynptr partially. Consider the following layout: spi: [d][d][?] 2 1 0 First slot is at spi 2, second at spi 1. Now, do a write of 1 to 8 bytes for spi 1. This will essentially either write STACK_MISC for all slot_types or STACK_MISC and STACK_ZERO (in case of size < BPF_REG_SIZE partial write of zeroes). The end result is that slot is scrubbed. Now, the layout is: spi: [d][m][?] 2 1 0 Suppose if user initializes spi = 1 as dynptr. We get: spi: [d][d][d] 2 1 0 But this time, both spi 2 and spi 1 have first_slot = true. Now, when passing spi 2 to dynptr helper, it will consider it as initialized as it does not check whether second slot has first_slot == false. And spi 1 should already work as normal. This effectively replaced size + offset of first dynptr, hence allowing invalid OOB reads and writes. Make a few changes to protect against this: When writing to PTR_TO_STACK using BPF insns, when we touch spi of a STACK_DYNPTR type, mark both first and second slot (regardless of which slot we touch) as STACK_INVALID. Reads are already prevented. Second, prevent writing to stack memory from helpers if the range may contain any STACK_DYNPTR slots. Reads are already prevented. For helpers, we cannot allow it to destroy dynptrs from the writes as depending on arguments, helper may take uninit_mem and dynptr both at the same time. This would mean that helper may write to uninit_mem before it reads the dynptr, which would be bad. PTR_TO_MEM: [?????dd] Depending on the code inside the helper, it may end up overwriting the dynptr contents first and then read those as the dynptr argument. Verifier would only simulate destruction when it does byte by byte access simulation in check_helper_call for meta.access_size, and fail to catch this case, as it happens after argument checks. The same would need to be done for any other non-trivial objects created on the stack in the future, such as bpf_list_head on stack, or bpf_rb_root on stack. A common misunderstanding in the current code is that MEM_UNINIT means writes, but note that writes may also be performed even without MEM_UNINIT in case of helpers, in that case the code after handling meta && meta->raw_mode will complain when it sees STACK_DYNPTR. So that invalid read case also covers writes to potential STACK_DYNPTR slots. The only loophole was in case of meta->raw_mode which simulated writes through instructions which could overwrite them. A future series sequenced after this will focus on the clean up of helper access checks and bugs around that. Fixes: 97e03f521050 ("bpf: Add verifier support for dynptrs") Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-4-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-21bpf: Fix missing var_off check for ARG_PTR_TO_DYNPTRKumar Kartikeya Dwivedi2-3/+3
Currently, the dynptr function is not checking the variable offset part of PTR_TO_STACK that it needs to check. The fixed offset is considered when computing the stack pointer index, but if the variable offset was not a constant (such that it could not be accumulated in reg->off), we will end up a discrepency where runtime pointer does not point to the actual stack slot we mark as STACK_DYNPTR. It is impossible to precisely track dynptr state when variable offset is not constant, hence, just like bpf_timer, kptr, bpf_spin_lock, etc. simply reject the case where reg->var_off is not constant. Then, consider both reg->off and reg->var_off.value when computing the stack pointer index. A new helper dynptr_get_spi is introduced to hide over these details since the dynptr needs to be located in multiple places outside the process_dynptr_func checks, hence once we know it's a PTR_TO_STACK, we need to enforce these checks in all places. Note that it is disallowed for unprivileged users to have a non-constant var_off, so this problem should only be possible to trigger from programs having CAP_PERFMON. However, its effects can vary. Without the fix, it is possible to replace the contents of the dynptr arbitrarily by making verifier mark different stack slots than actual location and then doing writes to the actual stack address of dynptr at runtime. Fixes: 97e03f521050 ("bpf: Add verifier support for dynptrs") Acked-by: Joanne Koong <joannelkoong@gmail.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20230121002241.2113993-3-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-20selftests/bpf: Add serial_test_kprobe_multi_bench_attach_kernel/module testsJiri Olsa1-5/+14
Add bench test for module portion of the symbols as well. # ./test_progs -v -t kprobe_multi_bench_attach_module bpf_testmod.ko is already unloaded. Loading bpf_testmod.ko... Successfully loaded bpf_testmod.ko. test_kprobe_multi_bench_attach:PASS:get_syms 0 nsec test_kprobe_multi_bench_attach:PASS:kprobe_multi_empty__open_and_load 0 nsec test_kprobe_multi_bench_attach:PASS:bpf_program__attach_kprobe_multi_opts 0 nsec test_kprobe_multi_bench_attach: found 26620 functions test_kprobe_multi_bench_attach: attached in 0.182s test_kprobe_multi_bench_attach: detached in 0.082s #96 kprobe_multi_bench_attach_module:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Successfully unloaded bpf_testmod.ko. It's useful for testing kprobe multi link modules resolving. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230116101009.23694-3-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-19selftests/bpf: Fix build errors if CONFIG_NF_CONNTRACK=mTiezhu Yang1-3/+8
If CONFIG_NF_CONNTRACK=m, there are no definitions of NF_NAT_MANIP_SRC and NF_NAT_MANIP_DST in vmlinux.h, build test_bpf_nf.c failed. $ make -C tools/testing/selftests/bpf/ CLNG-BPF [test_maps] test_bpf_nf.bpf.o progs/test_bpf_nf.c:160:42: error: use of undeclared identifier 'NF_NAT_MANIP_SRC' bpf_ct_set_nat_info(ct, &saddr, sport, NF_NAT_MANIP_SRC); ^ progs/test_bpf_nf.c:163:42: error: use of undeclared identifier 'NF_NAT_MANIP_DST' bpf_ct_set_nat_info(ct, &daddr, dport, NF_NAT_MANIP_DST); ^ 2 errors generated. Copy the definitions in include/net/netfilter/nf_nat.h to test_bpf_nf.c, in order to avoid redefinitions if CONFIG_NF_CONNTRACK=y, rename them with ___local suffix. This is similar with commit 1058b6a78db2 ("selftests/bpf: Do not fail build if CONFIG_NF_CONNTRACK=m/n"). Fixes: b06b45e82b59 ("selftests/bpf: add tests for bpf_ct_set_nat_info kfunc") Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Acked-by: Jiri Olsa <jolsa@kernel.org> Tested-by: Jiri Olsa <jolsa@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/r/1674028604-7113-1-git-send-email-yangtiezhu@loongson.cn Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
2023-01-18bpf/selftests: Add verifier tests for loading sleepable programsJiri Olsa1-0/+91
Adding verifier tests for loading all types od allowed sleepable programs plus reject for tp_btf type. Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230117223705.440975-2-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-01-15selftests/bpf: add ipip6 and ip6ip decap to test_tc_tunnelZiyang Xuan2-8/+98
Add ipip6 and ip6ip decap testcases. Verify that bpf_skb_adjust_room() correctly decapsulate ipip6 and ip6ip tunnel packets. Signed-off-by: Ziyang Xuan <william.xuanziyang@huawei.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/dfd2d8cfdf9111bd129170d4345296f53bee6a67.1673574419.git.william.xuanziyang@huawei.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
2023-01-15bpf: Add ipip6 and ip6ip decap support for bpf_skb_adjust_room()Ziyang Xuan1-0/+7
Add ipip6 and ip6ip decap support for bpf_skb_adjust_room(). Main use case is for using cls_bpf on ingress hook to decapsulate IPv4 over IPv6 and IPv6 over IPv4 tunnel packets. Add two new flags BPF_F_ADJ_ROOM_DECAP_L3_IPV{4,6} to indicate the new IP header version after decapsulating the outer IP header. Suggested-by: Willem de Bruijn <willemb@google.com> Signed-off-by: Ziyang Xuan <william.xuanziyang@huawei.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/b268ec7f0ff9431f4f43b1b40ab856ebb28cb4e1.1673574419.git.william.xuanziyang@huawei.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
generated by cgit v1.2.3 (git 2.39.0) at 2025-09-26 00:38:23 +0300