kernel/linux.git/include/net/rps.h, branch v6.12.80Linux kernel stable tree (mirror)https://git.radix-linux.su/kernel/linux.git/atom?h=v6.12.802024-04-01T10:28:32+00:00net: rps: add rps_input_queue_head_add() helper2024-04-01T10:28:32+00:00Eric Dumazetedumazet@google.com2024-03-29T15:42:24+00:00urn:sha1:c62fdf5b11ef12b89ac2450c25c12bffc3d924c5
process_backlog() can batch increments of sd->input_queue_head,
saving some memory bandwidth.
Also add READ_ONCE()/WRITE_ONCE() annotations around
sd->input_queue_head accesses.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
net: rps: change input_queue_tail_incr_save()2024-04-01T10:28:32+00:00Eric Dumazetedumazet@google.com2024-03-29T15:42:23+00:00urn:sha1:36b83ffcf209a2e6099dae1070df6a2001dfab27
input_queue_tail_incr_save() is incrementing the sd queue_tail
and save it in the flow last_qtail.
Two issues here :
- no lock protects the write on last_qtail, we should use appropriate
annotations.
- We can perform this write after releasing the per-cpu backlog lock,
to decrease this lock hold duration (move away the cache line miss)
Also move input_queue_head_incr() and rps helpers to include/net/rps.h,
while adding rps_ prefix to better reflect their role.
v2: Fixed a build issue (Jakub and kernel build bots)
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
net: move rps_sock_flow_table to net_hotdata2024-03-08T05:12:43+00:00Eric Dumazetedumazet@google.com2024-03-06T16:00:31+00:00urn:sha1:ce7f49ab741591d83e33e56948bac2f12de6e14e
rps_sock_flow_table and rps_cpu_mask are used in fast path.
Move them to net_hotdata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Reviewed-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/r/20240306160031.874438-19-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
net: introduce include/net/rps.h2024-03-08T05:12:43+00:00Eric Dumazetedumazet@google.com2024-03-06T16:00:30+00:00urn:sha1:490a79faf95e705ba0ffd9ebf04a624b379e53c9
Move RPS related structures and helpers from include/linux/netdevice.h
and include/net/sock.h to a new include file.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Reviewed-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/r/20240306160031.874438-18-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>