kernel/linux.git/net/packet/internal.h, branch v6.19.12

net: af_packet: Use hrtimer to do the retire operation

2025-09-12T01:40:06+00:00

In a system with high real-time requirements, the timeout mechanism of ordinary timers with jiffies granularity is insufficient to meet the demands for real-time performance. Meanwhile, the optimization of CPU usage with af_packet is quite significant. Use hrtimer instead of timer to help compensate for the shortcomings in real-time performance. In HZ=100 or HZ=250 system, the update of TP_STATUS_USER is not real-time enough, with fluctuations reaching over 8ms (on a system with HZ=250). This is unacceptable in some high real-time systems that require timely processing of network packets. By replacing it with hrtimer, if a timeout of 2ms is set, the update of TP_STATUS_USER can be stabilized to within 3 ms. Delete delete_blk_timer field, because hrtimer_cancel will check and wait until the timer callback return and ensure never enter callback again. Simplify the logic related to setting timeout, only update the hrtimer expire time within the hrtimer callback, no longer update the expire time in prb_open_block which is called by tpacket_rcv or timer callback. Reasons why NOT update hrtimer in prb_open_block: 1) It will increase complexity to distinguish the two caller scenario. 2) hrtimer_cancel and hrtimer_start need to be called if you want to update TMO of an already enqueued hrtimer, leading to complex shutdown logic. One side effect of NOT update hrtimer when called by tpacket_rcv is that a newly opened block triggered by tpacket_rcv may be retired earlier than expected. On the other hand, if timeout is updated in prb_open_block, the frequent reception of network packets that leads to prb_open_block being called may cause hrtimer to be removed and enqueued repeatedly. The retire hrtimer expiration is unconditional and periodic. If there are numerous packet sockets on the system, please set an appropriate timeout to avoid frequent enqueueing of hrtimers. Reviewed-by: Willem de Bruijn Reviewed-by: Jason Xing Link: https://lore.kernel.org/all/20250831100822.1238795-1-jackzxcui1989@163.com/ Signed-off-by: Xin Zhao Link: https://patch.msgid.link/20250908104549.204412-3-jackzxcui1989@163.com Signed-off-by: Jakub Kicinski

net: af_packet: remove last_kactive_blk_num field

2025-09-12T01:40:06+00:00

kactive_blk_num (K) is only incremented on block close. In timer callback prb_retire_rx_blk_timer_expired, except delete_blk_timer is true, last_kactive_blk_num (L) is set to match kactive_blk_num (K) in all cases. L is also set to match K in prb_open_block. The only case K not equal to L is when scheduled by tpacket_rcv and K is just incremented on block close but no new block could be opened, so that it does not call prb_open_block in prb_dispatch_next_block. This patch modifies the prb_retire_rx_blk_timer_expired function by simply removing the check for L == K. This patch just provides another checkpoint to thaw the might-be-frozen block in any case. It doesn't have any effect because __packet_lookup_frame_in_block() has the same logic and does it again without this patch when detecting the ring is frozen. The patch only advances checking the status of the ring. Suggested-by: Willem de Bruijn Reviewed-by: Willem de Bruijn Reviewed-by: Jason Xing Link: https://lore.kernel.org/all/20250831100822.1238795-1-jackzxcui1989@163.com/ Signed-off-by: Xin Zhao Link: https://patch.msgid.link/20250908104549.204412-2-jackzxcui1989@163.com Signed-off-by: Jakub Kicinski

af_packet: move notifier's packet_dev_mc out of rcu critical section

2025-05-27T09:36:26+00:00

Syzkaller reports the following issue: BUG: sleeping function called from invalid context at kernel/locking/mutex.c:578 __mutex_lock+0x106/0xe80 kernel/locking/mutex.c:746 team_change_rx_flags+0x38/0x220 drivers/net/team/team_core.c:1781 dev_change_rx_flags net/core/dev.c:9145 [inline] __dev_set_promiscuity+0x3f8/0x590 net/core/dev.c:9189 netif_set_promiscuity+0x50/0xe0 net/core/dev.c:9201 dev_set_promiscuity+0x126/0x260 net/core/dev_api.c:286 packet_dev_mc net/packet/af_packet.c:3698 [inline] packet_dev_mclist_delete net/packet/af_packet.c:3722 [inline] packet_notifier+0x292/0xa60 net/packet/af_packet.c:4247 notifier_call_chain+0x1b3/0x3e0 kernel/notifier.c:85 call_netdevice_notifiers_extack net/core/dev.c:2214 [inline] call_netdevice_notifiers net/core/dev.c:2228 [inline] unregister_netdevice_many_notify+0x15d8/0x2330 net/core/dev.c:11972 rtnl_delete_link net/core/rtnetlink.c:3522 [inline] rtnl_dellink+0x488/0x710 net/core/rtnetlink.c:3564 rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6955 netlink_rcv_skb+0x219/0x490 net/netlink/af_netlink.c:2534 Calling `PACKET_ADD_MEMBERSHIP` on an ops-locked device can trigger the `NETDEV_UNREGISTER` notifier, which may require disabling promiscuous and/or allmulti mode. Both of these operations require acquiring the netdev instance lock. Move the call to `packet_dev_mc` outside of the RCU critical section. The `mclist` modifications (add, del, flush, unregister) are protected by the RTNL, not the RCU. The RCU only protects the `sklist` and its associated `sks`. The delayed operation on the `mclist` entry remains within the RTNL. Reported-by: syzbot+b191b5ccad8d7a986286@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=b191b5ccad8d7a986286 Fixes: ad7c7b2172c3 ("net: hold netdev instance lock during sysfs operations") Signed-off-by: Stanislav Fomichev Reviewed-by: Willem de Bruijn Link: https://patch.msgid.link/20250522031129.3247266-1-stfomichev@gmail.com Signed-off-by: Paolo Abeni

packet: Move reference count in packet_sock to atomic_long_t

2023-12-04T22:45:04+00:00

In some potential instances the reference count on struct packet_sock could be saturated and cause overflows which gets the kernel a bit confused. To prevent this, move to a 64-bit atomic reference count on 64-bit architectures to prevent the possibility of this type to overflow. Because we can not handle saturation, using refcount_t is not possible in this place. Maybe someday in the future if it changes it could be used. Also, instead of using plain atomic64_t, use atomic_long_t instead. 32-bit machines tend to be memory-limited (i.e. anything that increases a reference uses so much memory that you can't actually get to 2**32 references). 32-bit architectures also tend to have serious problems with 64-bit atomics. Hence, atomic_long_t is the more natural solution. Reported-by: "The UK's National Cyber Security Centre (NCSC)" Co-developed-by: Greg Kroah-Hartman Signed-off-by: Greg Kroah-Hartman Signed-off-by: Daniel Borkmann Cc: Linus Torvalds Cc: stable@kernel.org Reviewed-by: Willem de Bruijn Reviewed-by: Eric Dumazet Link: https://lore.kernel.org/r/20231201131021.19999-1-daniel@iogearbox.net Signed-off-by: Jakub Kicinski

net/packet: Annotate struct packet_fanout with __counted_by

2023-10-06T10:36:25+00:00

Prepare for the coming implementation by GCC and Clang of the __counted_by attribute. Flexible array members annotated with __counted_by can have their accesses bounds-checked at run-time via CONFIG_UBSAN_BOUNDS (for array indexing) and CONFIG_FORTIFY_SOURCE (for strcpy/memcpy-family functions). As found with Coccinelle[1], add __counted_by for struct packet_fanout. Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Willem de Bruijn Cc: Anqi Shen Cc: netdev@vger.kernel.org Link: https://github.com/kees/kernel-tools/blob/trunk/coccinelle/examples/counted_by.cocci [1] Signed-off-by: Kees Cook Reviewed-by: Gustavo A. R. Silva Signed-off-by: David S. Miller

net/packet: support mergeable feature of virtio

2023-04-21T11:01:58+00:00

Packet sockets, like tap, can be used as the backend for kernel vhost. In packet sockets, virtio net header size is currently hardcoded to be the size of struct virtio_net_hdr, which is 10 bytes; however, it is not always the case: some virtio features, such as mrg_rxbuf, need virtio net header to be 12-byte long. Mergeable buffers, as a virtio feature, is worthy of supporting: packets that are larger than one-mbuf size will be dropped in vhost worker's handle_rx if mrg_rxbuf feature is not used, but large packets cannot be avoided and increasing mbuf's size is not economical. With this virtio feature enabled by virtio-user, packet sockets with hardcoded 10-byte virtio net header will parse mac head incorrectly in packet_snd by taking the last two bytes of virtio net header as part of mac header. This incorrect mac header parsing will cause packet to be dropped due to invalid ether head checking in later under-layer device packet receiving. By adding extra field vnet_hdr_sz with utilizing holes in struct packet_sock to record currently used virtio net header size and supporting extra sockopt PACKET_VNET_HDR_SZ to set specified vnet_hdr_sz, packet sockets can know the exact length of virtio net header that virtio user gives. In packet_snd, tpacket_snd and packet_recvmsg, instead of using hardcoded virtio net header size, it can get the exact vnet_hdr_sz from corresponding packet_sock, and parse mac header correctly based on this information to avoid the packets being mistakenly dropped. Signed-off-by: Jianfeng Tan Co-developed-by: Anqi Shen Signed-off-by: Anqi Shen Reviewed-by: Willem de Bruijn Signed-off-by: David S. Miller

net/packet: remove po->xmit

2023-03-19T10:57:54+00:00

Use PACKET_SOCK_QDISC_BYPASS atomic bit instead of a pointer. This removes one indirect call in fast path, and READ_ONCE()/WRITE_ONCE() annotations as well. Signed-off-by: Eric Dumazet Suggested-by: Willem de Bruijn Cc: Daniel Borkmann Reviewed-by: Willem de Bruijn Signed-off-by: David S. Miller

af_packet: preserve const qualifier in pkt_sk()

2023-03-18T12:23:33+00:00

We can change pkt_sk() to propagate const qualifier of its argument, thanks to container_of_const() This should avoid some potential errors caused by accidental (const -> not_const) promotion. Signed-off-by: Eric Dumazet Cc: Willem de Bruijn Reviewed-by: Simon Horman Signed-off-by: David S. Miller

net/packet: convert po->pressure to an atomic flag

2023-03-17T08:52:06+00:00

Not only this removes some READ_ONCE()/WRITE_ONCE(), this also removes one integer. Signed-off-by: Eric Dumazet Signed-off-by: David S. Miller

net/packet: convert po->running to an atomic flag

2023-03-17T08:52:06+00:00

Instead of consuming 32 bits for po->running, use one available bit in po->flags. Signed-off-by: Eric Dumazet Signed-off-by: David S. Miller