kernel/linux.git/net/xdp, branch v7.1.8

xsk: reclaim invalid Tx descriptors in ZC batch path

2026-08-09T18:26:28+00:00

[ Upstream commit 72f2b4516faf55d4dfac2414649d3cffa5fd2c5e ] The zero-copy Tx batch parser stops when it encounters an invalid descriptor. If this happens after one or more continuation descriptors, the Tx consumer can be advanced past fragments that are neither submitted to the driver nor returned to userspace through the completion ring. A similar problem occurs when a packet exceeds xdp_zc_max_segs. The descriptors consumed up to the limit are released without completion, and the remaining continuation descriptors can subsequently be interpreted as the beginning of another packet. Parse Tx batches in packet units and distinguish descriptors belonging to complete valid packets from descriptors consumed while draining an invalid or oversized packet. Return the former to the driver and append the latter to the CQ address area so userspace can reclaim their UMEM frames. Treat a standalone invalid descriptor as a one-descriptor reclaim-only packet. Advancing the Tx-ring consumer releases the ring slot, but does not by itself return ownership of the referenced UMEM frame to userspace. Once draining starts, continue until the packet's end-of-packet descriptor is consumed. Preserve the drain state on the socket when EOP has not yet been supplied, so draining can continue during a later call. Leave incomplete but otherwise valid packets on the Tx ring. Shared-UMEM pools using multi-buffer Tx also need packet-framed parsing. Walk their Tx sockets one packet at a time, preserving the existing per-socket fairness scheme, instead of using the legacy one-descriptor fallback. Keep that fallback for shared pools that do not use multi-buffer Tx. Since the drain state is maintained per socket and both the singular and shared paths can resume an interrupted drain, changing the socket list from singular to shared requires no special bind-time transition. CQ entries are positional, and drivers may complete only part of the Tx work returned by xsk_tx_peek_release_desc_batch(). Therefore, reclaim-only entries cannot be published immediately when earlier driver-visible descriptors are still outstanding. Track the number of driver-visible CQ entries preceding the reclaim entries. Let xsk_tx_completed() publish partial hardware Tx completions, and publish the reclaim entries only after every earlier Tx descriptor has completed. Complete a reclaim-only batch immediately when there is no driver-visible work in front of it, and prevent another Tx batch from being appended while reclaim entries remain pending. Also cap batch processing by the size of the pool's temporary descriptor array, as Tx rings belonging to sockets sharing a UMEM may have different sizes. This ensures that every invalid Tx descriptor consumed by the ZC batch path is either submitted to the driver as part of a valid packet or returned to userspace without violating CQ completion ordering. Reviewed-by: Jason Xing Signed-off-by: Maciej Fijalkowski Acked-by: Stanislav Fomichev Fixes: cf24f5a5feea ("xsk: add support for AF_XDP multi-buffer on Tx path") Link: https://patch.msgid.link/20260719135609.147823-5-maciej.fijalkowski@intel.com Signed-off-by: Jakub Kicinski Signed-off-by: Sasha Levin

xsk: provide sufficient space in pool->tx_descs

2026-08-09T18:26:28+00:00

[ Upstream commit 08c9a8e794b4694c100dafcb80e069e29ad81b64 ] The temporary Tx descriptor array in an XSK buffer pool is currently sized from the Tx ring of the socket that creates the pool. This is insufficient for shared-UMEM Tx. A later socket may have a larger Tx ring and submit a valid multi-buffer packet containing more descriptors than the first socket's ring, while still remaining within the device's xdp_zc_max_segs limit. A packet-framed batch parser bounded by the temporary array cannot reach the end-of-packet descriptor in that case. It leaves the packet on the Tx ring and encounters the same packet on every subsequent attempt, stalling Tx processing for that socket. Size the temporary descriptor array to the larger of the first Tx ring and the device's xdp_zc_max_segs capability. This keeps the array large enough to inspect one maximum-sized valid packet. Larger shared Tx rings do not require further resizing, as they can be processed over multiple batches. Following commit will actually address the data path side. Fixes: d5581966040f ("xsk: support ZC Tx multi-buffer in batch API") Reviewed-by: Jason Xing Signed-off-by: Maciej Fijalkowski Acked-by: Stanislav Fomichev Link: https://patch.msgid.link/20260719135609.147823-4-maciej.fijalkowski@intel.com Signed-off-by: Jakub Kicinski Signed-off-by: Sasha Levin

xsk: drain continuation descs after overflow in xsk_build_skb()

2026-08-09T18:26:28+00:00

[ Upstream commit bd44a6dcd4248883de90f5dad53ae80066e27096 ] Fix generic xmit path multi-buffer logic when packets are either too big (count of descriptors exceed MAX_SKB_FRAGS) or an invalid descriptor is included in fragmented packet. Introduce xdp_sock::drain_cont and act upon this flag - when it is set, keep on consuming descriptors from AF_XDP Tx ring and put them directly onto Cq. Previously these descriptors were silently lost and could never be reached again. Fixes: cf24f5a5feea ("xsk: add support for AF_XDP multi-buffer on Tx path") Closes: https://lore.kernel.org/all/20260425041726.85FB3C2BCB2@smtp.kernel.org/ Reviewed-by: Jason Xing Co-developed-by: Maciej Fijalkowski # wrapped cq addr submission onto routine Signed-off-by: Maciej Fijalkowski Signed-off-by: Jason Xing Acked-by: Stanislav Fomichev Link: https://patch.msgid.link/20260719135609.147823-3-maciej.fijalkowski@intel.com Signed-off-by: Jakub Kicinski Signed-off-by: Sasha Levin

xsk: fix buffer leak in xsk_drop_skb() for AF_XDP multi-buffer Tx

2026-08-09T18:26:28+00:00

[ Upstream commit a3c8382ebce4780c6b3ace2c09bc342313ac0186 ] This patch is inspired by the check[1] from sashiko. It says when overflow happens, the address of cq to be published is invalid. Actually the severer thing is the whole process of publishing the address of cq in this particular case is not right: it should truely publish the address and advance the cached_prod in cq as long as it reads descriptors from txq. The following is the full analysis. xsk_drop_skb() is called in three places, which all discard a partially built multi-buffer skb: 1) xsk_build_skb() -EOVERFLOW error path: packet exceeds MAX_SKB_FRAGS 2) __xsk_generic_xmit() post-loop cleanup: an invalid descriptor in the TX ring prevents the partial packet from completing 3) xsk_release(): socket close while xs->skb holds an incomplete packet In all three cases, the TX descriptors for the already-processed frags have been consumed from the TX ring (xskq_cons_release), and CQ slots have been reserved. However, xsk_drop_skb() calls xsk_consume_skb() which cancels the CQ reservations via xsk_cq_cancel_locked(). Since the buffer addresses never appear in the completion queue, userspace permanently loses track of these buffers. Fix this by letting consume_skb() trigger the existing xsk_destruct_skb destructor, which already submits buffer addresses to the CQ via xsk_cq_submit_addr_locked(). Note that cancelling the descriptors back to the TX ring (via xskq_cons_cancel_n) is not a appropriate option because an oversized packet that always exceeds MAX_SKB_FRAGS would be retried indefinitely, which is an obviously deadlock bug in the TX path. Also move the desc->addr assignment in xsk_build_skb() above the overflow check so that the current descriptor's address is recorded before a potential -EOVERFLOW jump to free_err, consistent with the zerocopy path in xsk_build_skb_zerocopy(). [1]: https://lore.kernel.org/all/20260425041726.85FB3C2BCB2@smtp.kernel.org/ Fixes: cf24f5a5feea ("xsk: add support for AF_XDP multi-buffer on Tx path") Acked-by: Maciej Fijalkowski Signed-off-by: Jason Xing Acked-by: Stanislav Fomichev Link: https://patch.msgid.link/20260719135609.147823-2-maciej.fijalkowski@intel.com Signed-off-by: Jakub Kicinski Signed-off-by: Sasha Levin

xsk: cache csum_start/csum_offset to fix TOCTOU in xsk_skb_metadata()

2026-06-04T00:45:42+00:00

The TX metadata area resides in the UMEM buffer which is memory-mapped and concurrently writable by userspace. In xsk_skb_metadata(), csum_start and csum_offset are read from shared memory for bounds validation, then read again for skb assignment. A malicious userspace application can race to overwrite these values between the two reads, bypassing the bounds check and causing out-of-bounds memory access during checksum computation in the transmit path. Fix this by reading csum_start and csum_offset into local variables once, then using the local copies for both validation and assignment. Note that other metadata fields (flags, launch_time) and the cached csum fields may be mutually inconsistent due to concurrent userspace writes, but this is benign: the only security-critical invariant is that each field's validated value is the same one used, which local caching guarantees. Closes: https://lore.kernel.org/all/20260503200927.73EA1C2BCB4@smtp.kernel.org/ Reviewed-by: Maciej Fijalkowski Signed-off-by: Jason Xing Acked-by: Stanislav Fomichev Fixes: 48eb03dd2630 ("xsk: Add TX timestamp and TX checksum offload support") Link: https://patch.msgid.link/20260530042630.80626-1-kerneljasonxing@gmail.com Signed-off-by: Jakub Kicinski

Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf

2026-05-10T01:42:54+00:00

Pull bpf fixes from Alexei Starovoitov: - Fix sk_local_storage diag dump via netlink (Amery Hung) - Fix off-by-one in arena direct-value access (Junyoung Jang) - Reject TCP_NODELAY in bpf-tcp congestion control (KaFai Wan) - Fix type confusion in bpf_*_sock() (Kuniyuki Iwashima) - Reject TX-only AF_XDP sockets (Linpu Yu) - Don't run arg-tracking analysis twice on main subprog (Paul Chaignon) - Fix NULL pointer dereference in bpf_sk_storage_clone and fib lookup (Weiming Shi) * tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: bpf: Fix off-by-one boundary validation in arena direct-value access xskmap: reject TX-only AF_XDP sockets bpf: Don't run arg-tracking analysis twice on main subprog bpf: Free reuseport cBPF prog after RCU grace period. bpf: tcp: Fix type confusion in sol_tcp_sockopt(). bpf: tcp: Fix type confusion in bpf_skc_to_tcp6_sock(). bpf: tcp: Fix type confusion in bpf_skc_to_tcp_sock(). mptcp: bpf: Fix type confusion in bpf_mptcp_sock_from_subflow() selftest: bpf: Add test for bpf_tcp_sock() and RAW socket. bpf: tcp: Fix type confusion in bpf_tcp_sock(). tools/headers: Regenerate stddef.h to fix BPF selftests bpf: Fix sk_local_storage diag dumping uninitialized special fields bpf: Fix NULL pointer dereference in bpf_skb_fib_lookup() sockmap: Fix sk_psock_drop() race vs sock_map_{unhash,close,destroy}(). bpf: Fix NULL pointer dereference in bpf_sk_storage_clone and diag paths selftests/bpf: Verify bpf-tcp-cc rejects TCP_NODELAY selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks bpf: Reject TCP_NODELAY in bpf-tcp-cc bpf: Reject TCP_NODELAY in TCP header option callbacks

xskmap: reject TX-only AF_XDP sockets

2026-05-09T23:17:01+00:00

XSKMAP entries are used as redirect targets for incoming XDP frames. A TX-only AF_XDP socket lacks an Rx ring and cannot handle redirected traffic, but xsk_map_update_elem() currently allows such sockets to be inserted into the map. Redirecting packets to such a socket on the veth generic-XDP path causes a kernel crash in xsk_generic_rcv(). This became possible after xsk_is_setup_for_bpf_map() was removed from the XSKMAP update path, which allowed bound TX-only sockets to be inserted into the map. Reject TX-only sockets during XSKMAP updates to avoid the crash. They remain fully operational for pure Tx purposes outside XSKMAP. Fixes: 968be23ceaca ("xsk: Fix possible segfault at xskmap entry insertion") Reported-by: Juefei Pu Reported-by: Yuan Tan Reported-by: Xin Liu Signed-off-by: Yifan Wu Signed-off-by: Linpu Yu Reviewed-by: Jason Xing Link: https://lore.kernel.org/r/20260508144344.694-1-linpu5433@gmail.com Signed-off-by: Alexei Starovoitov

xsk: fix u64 descriptor address truncation on 32-bit architectures

2026-05-06T02:27:51+00:00

In copy mode TX, xsk_skb_destructor_set_addr() stores the 64-bit descriptor address into skb_shinfo(skb)->destructor_arg (void *) via a uintptr_t cast: skb_shinfo(skb)->destructor_arg = (void *)((uintptr_t)addr | 0x1UL); On 32-bit architectures uintptr_t is 32 bits, so the upper 32 bits of the descriptor address are silently dropped. In XDP_ZEROCOPY unaligned mode the chunk offset is encoded in bits 48-63 of the descriptor address (XSK_UNALIGNED_BUF_OFFSET_SHIFT = 48), meaning the offset is lost entirely. The completion queue then returns a truncated address to userspace, making buffer recycling impossible. Fix this by handling the 32-bit case directly in xsk_skb_destructor_set_addr(): when !CONFIG_64BIT, allocate an xsk_addrs struct (the same path already used for multi-descriptor SKBs) to store the full u64 address. The existing tagged-pointer logic in xsk_skb_destructor_is_addr() stays unchanged: slab pointers returned from kmem_cache_zalloc() are always word-aligned and therefore have bit 0 clear, which correctly identifies them as a struct pointer rather than an inline tagged address on every architecture. Factor the shared kmem_cache_zalloc + destructor_arg assignment into __xsk_addrs_alloc() and add a wrapper xsk_addrs_alloc() that handles the inline-to-list upgrade (is_addr check + get_addr + num_descs = 1). The three former open-coded kmem_cache_zalloc call sites now reduce to a single call each. Propagate the -ENOMEM from xsk_skb_destructor_set_addr() through xsk_skb_init_misc() so the caller can clean up the skb via kfree_skb() before skb->destructor is installed. The overhead is one extra kmem_cache_zalloc per first descriptor on 32-bit only; 64-bit builds are completely unchanged. Closes: https://lore.kernel.org/all/20260419045824.D9E5EC2BCAF@smtp.kernel.org/ Fixes: 0ebc27a4c67d ("xsk: avoid data corruption on cq descriptor number") Signed-off-by: Jason Xing Acked-by: Stanislav Fomichev Reviewed-by: Alexander Lobakin Link: https://patch.msgid.link/20260502200722.53960-9-kerneljasonxing@gmail.com Signed-off-by: Jakub Kicinski

xsk: fix xsk_addrs slab leak on multi-buffer error path

2026-05-06T02:27:51+00:00

When xsk_build_skb() / xsk_build_skb_zerocopy() sees the first continuation descriptor, it promotes destructor_arg from an inlined address to a freshly allocated xsk_addrs (num_descs = 1). The counter is bumped to >= 2 only at the very end of a successful build (by calling xsk_inc_num_desc()). If the build fails in between (e.g. alloc_page() returns NULL with -EAGAIN, or the MAX_SKB_FRAGS overflow hits), we jump to free_err, skip calling xsk_inc_num_desc() to increment num_descs and leave the half-built skb attached to xs->skb for the app to retry. The skb now has 1) destructor_arg = a real xsk_addrs pointer, 2) num_descs = 1 If the app never retries and just close()s the socket, xsk_release() calls xsk_drop_skb() -> xsk_consume_skb(), which decides whether to free xsk_addrs by testing num_descs > 1: if (unlikely(num_descs > 1)) kmem_cache_free(xsk_tx_generic_cache, destructor_arg); Because num_descs is exactly 1 the branch is skipped and the xsk_addrs object is leaked to the xsk_tx_generic_cache slab. Fix it by directly testing if destructor_arg is still addr. Or else it is modified and used to store the newly allocated memory from xsk_tx_generic_cache regardless of increment of num_desc, which we need to handle. Closes: https://lore.kernel.org/all/20260419045824.D9E5EC2BCAF@smtp.kernel.org/ Fixes: 0ebc27a4c67d ("xsk: avoid data corruption on cq descriptor number") Acked-by: Stanislav Fomichev Signed-off-by: Jason Xing Reviewed-by: Alexander Lobakin Link: https://patch.msgid.link/20260502200722.53960-8-kerneljasonxing@gmail.com Signed-off-by: Jakub Kicinski

xsk: avoid skb leak in XDP_TX_METADATA case

2026-05-06T02:27:50+00:00

Fix it by explicitly adding kfree_skb() before returning back to its caller. How to reproduce it in virtio_net: 1. the current skb is the first one (which means no frag and xs->skb is NULL) and users enable metadata feature. 2. xsk_skb_metadata() returns a error code. 3. the caller xsk_build_skb() clears skb by using 'skb = NULL;'. 4. there is no chance to free this skb anymore. Closes: https://lore.kernel.org/all/20260415085204.3F87AC19424@smtp.kernel.org/ Fixes: 30c3055f9c0d ("xsk: wrap generic metadata handling onto separate function") Acked-by: Stanislav Fomichev Signed-off-by: Jason Xing Reviewed-by: Alexander Lobakin Link: https://patch.msgid.link/20260502200722.53960-7-kerneljasonxing@gmail.com Signed-off-by: Jakub Kicinski