kernel/linux.git/include/linux/srcu.h, branch v7.3-rc1

Merge tag 'rcu.2026.08.18a' of git://git.kernel.org/pub/scm/linux/kernel/git/rcu/linux

2026-08-24T01:00:22+00:00

Pull RCU updates from Paul McKenney: "Make expedited grace periods expedite normal RCU callbacks Miscellaneous fixes: - Improve diagnostic output with character task states - Mark accesses to inform KCSAN of concurrency design - Move from kmalloc() to kmalloc_obj() - Documentation updates - Improve handling of RCU deferred quiescent states - Clean up unused function arguments and structure fields - Reduce show_rcu_gp_kthreads() stack space Tasks RCU updates: - Clean up after SRCU re-implementation of Tasks Trace RCU - Mark accesses to inform KCSAN of concurrency design - Add ->lazy_timer status to diagnostic output - Remove an unnecessary memory barrier - Fix a data race, courtesy of KCSAN - Documentation updates - Convert cond_resched_tasks_rcu_qs() from macro to static inline function SRCU updates: - Add Rust helpers for SRCU - Avoid losing queued work at cleanup_srcu_struct() time Torture-test updates: - Preparation work for immediate RCU priority deboosting - Test RCU readers from real interrupt handlers (as opposed to softirq) - Simplify code through use of cpumask_next_wrap() - Improve diagnostic output with character task states - Add rcutorture.nwriters parameter to allow lightweight stall testing, and rcutorture.stall_only to make doing so easier - Test an RCU Tasks Trace grace period implying an RCU grace period - Make RCU Tasks Trace torturing track reader batches - Fix a data race, courtesy of KCSAN - Plug a shuffle_tmp_mask memory leak on kthread spawn failure" * tag 'rcu.2026.08.18a' of git://git.kernel.org/pub/scm/linux/kernel/git/rcu/linux: (59 commits) rcu: Add closing parenthesis in comment in rcu_read_unlock_strict() rcutorture: Make {,s}rcu_read_delay() better handle forward-progress testing rcutorture: Announce declining to forward-progress test torture: Don't leak shuffle_tmp_mask when shuffler kthread fails to start rcutorture: Use this_cpu_inc() for rcu_torture_count[] and rcu_torture_batch[] rcutorture: Make RCU Tasks Trace track Reader Batches rcutorture: Test RCU Tasks Trace GP implying RCU GP rcutorture: Add a stall_only module parameter rcutorture: Add nwriters module parameter rcutorture: Use task_state_to_char() for task-state reporting rcutorture: Use cpumask_next_wrap() in rcu_torture_preempt() rcutorture: Test RCU readers from hardware interrupt handlers rcutorture: Check for immediate deboosting at reader end srcu: Queue sdp->work when the delay timer is successfully deleted rcu-tasks: Convert cond_resched_tasks_rcu_qs() to static inline rcu-tasks: Fix some comments for call_rcu_tasks() and call_rcu_tasks_rude() rcu-tasks: Rename tasks_rcu_exit_srcu_stall_timer to tasks_rcu_exit_stall_timer rcu: Mark interrupts-enabled accesses to rdp->cpu_no_qs.s rcu: Reduce stack usage in show_rcu_gp_kthreads() rcu: Mark accesses to ->rcu_urgent_qs and ->rcu_need_heavy_qs ...

srcu: Add lock guard for srcu_fast_updown flavor

2026-08-03T08:09:08+00:00

Add a guard(srcu_fast_updown) definition for scoped SRCU-fast-updown read-side critical sections, following the existing pattern of guard(srcu) and guard(srcu_fast). Signed-off-by: Puranjay Mohan Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Paul E. McKenney Reviewed-by: Oleg Nesterov Link: https://patch.msgid.link/20260706172744.3920417-2-puranjay@kernel.org

srcu: make init_srcu_struct() consistently wrap __init_srcu_struct()

2026-07-15T22:34:25+00:00

Restructure the SRCU initialization functions so it always follows one direction: init_srcu_struct() -> __init_srcu_struct() -> lockdep or generic This uses the same wrapper style as mutex. It avoids the old confusing style where init_srcu_struct() and __init_srcu_struct() called each other in different configs. It also helps Rust side to have simpler helper for SRCU initialization. Signed-off-by: Onur Özkan Reviewed-by: Gary Guo Reviewed-by: Alice Ryhl Reviewed-by: Boqun Feng Signed-off-by: Paul E. McKenney

srcu: Fix kerneldoc header comment typo in srcu_down_read_fast()

2026-05-24T07:39:41+00:00

s/srcu_read_lock_safe()/srcu_read_lock_fast_updown(), there being no such thing as srcu_read_lock_safe(). Signed-off-by: Paul E. McKenney Signed-off-by: Uladzislau Rezki (Sony)

srcu: Fix SRCU read flavor macro comments

2026-03-30T19:48:13+00:00

The SRCU_READ_FLAVOR_FAST and SRCU_READ_FLAVOR_FAST_UPDOWN comments need repair. The former fails to not that SRCU-fast can be used in NMI handlers, and the latter says that it goes with srcu_read_lock_fast() when it really goes with srcu_read_lock_fast_updown(). This commit therefore fixes both comments. Signed-off-by: Paul E. McKenney Signed-off-by: Joel Fernandes

srcu: Support Clang's context analysis

2026-01-05T15:43:30+00:00

Add support for Clang's context analysis for SRCU. Signed-off-by: Marco Elver Signed-off-by: Peter Zijlstra (Intel) Acked-by: Paul E. McKenney Link: https://patch.msgid.link/20251219154418.3592607-16-elver@google.com

srcu: Create an SRCU-fast-updown API

2025-11-27T13:22:31+00:00

This commit creates an SRCU-fast-updown API, including DEFINE_SRCU_FAST_UPDOWN(), DEFINE_STATIC_SRCU_FAST_UPDOWN(), __init_srcu_struct_fast_updown(), init_srcu_struct_fast_updown(), srcu_read_lock_fast_updown(), srcu_read_unlock_fast_updown(), __srcu_read_lock_fast_updown(), and __srcu_read_unlock_fast_updown(). These are initially identical to their SRCU-fast counterparts, but both SRCU-fast and SRCU-fast-updown will be optimized in different directions by later commits. SRCU-fast will lack any sort of srcu_down_read() and srcu_up_read() APIs, which will enable extremely efficient NMI safety. For its part, SRCU-fast-updown will not be NMI safe, which will enable reasonably efficient implementations of srcu_down_read_fast() and srcu_up_read_fast(). This API fork happens to meet two different future use cases. * SRCU-fast will become the reimplementation basis for RCU-TASK-TRACE for consolidation. Since RCU-TASK-TRACE must be NMI safe, SRCU-fast must be as well. * SRCU-fast-updown will be needed for uretprobes code in order to get rid of the read-side memory barriers while still allowing entering the reader at task level while exiting it in a timer handler. This commit also adds rcutorture tests for the new APIs. This (annoyingly) needs to be in the same commit for bisectability. With this commit, the 0x8 value tests SRCU-fast-updown. However, most SRCU-fast testing will be via the RCU Tasks Trace wrappers. [ paulmck: Apply s/0x8/0x4/ missing change per Boqun Feng feedback. ] [ paulmck: Apply Akira Yokosawa feedback. ] Signed-off-by: Paul E. McKenney Cc: Andrii Nakryiko Cc: Alexei Starovoitov Cc: Peter Zijlstra Cc: Signed-off-by: Frederic Weisbecker

srcu: Add SRCU_READ_FLAVOR_FAST_UPDOWN CPP macro

2025-11-07T12:57:38+00:00

This commit adds the SRCU_READ_FLAVOR_FAST_UPDOWN=0x8 macro and adjusts rcutorture to make use of it. In this commit, both SRCU_READ_FLAVOR_FAST=0x4 and the new SRCU_READ_FLAVOR_FAST_UPDOWN test SRCU-fast. When the SRCU-fast-updown is added, the new SRCU_READ_FLAVOR_FAST_UPDOWN macro will test it when passed to the rcutorture.reader_flavor module parameter. The old SRCU_READ_FLAVOR_FAST macro's value changed from 0x8 to 0x4. Signed-off-by: Paul E. McKenney Cc: Mathieu Desnoyers Cc: Steven Rostedt Cc: Sebastian Andrzej Siewior Cc: Signed-off-by: Frederic Weisbecker

srcu: Make SRCU-fast readers enforce use of SRCU-fast definition/init

2025-11-05T22:58:28+00:00

This commit makes CONFIG_PROVE_RCU=y kernels enforce the new rule that srcu_struct structures that are passed to srcu_read_lock_fast() and other SRCU-fast read-side markers be either initialized with init_srcu_struct_fast() on the one hand or defined with DEFINE_SRCU_FAST() or DEFINE_STATIC_SRCU_FAST() on the other. This eliminates the read-side test that was formerly included in srcu_read_lock_fast() and friends, speeding these primitives up by about 25% (admittedly only about half of a nanosecond, but when tracing on fastpaths...) Signed-off-by: Paul E. McKenney Cc: Mathieu Desnoyers Cc: Steven Rostedt Cc: Sebastian Andrzej Siewior Cc: Signed-off-by: Frederic Weisbecker

srcu: Require special srcu_struct define/init for SRCU-fast readers

2025-11-05T22:58:26+00:00

This commit adds CONFIG_PROVE_RCU=y checking to enforce the new rule that srcu_struct structures passed to srcu_read_lock_fast() and other SRCU-fast read-side markers be either initialized with init_srcu_struct_fast() on the one hand or defined using either DEFINE_SRCU_FAST() or DEFINE_STATIC_SRCU_FAST(). This will enable removal of the non-debug read-side checks from srcu_read_lock_fast() and friends, which on my laptop provides a 25% speedup (which admittedly amounts to about half a nanosecond, but when tracing fastpaths...) Signed-off-by: Paul E. McKenney Cc: Mathieu Desnoyers Cc: Steven Rostedt Cc: Sebastian Andrzej Siewior Cc: Signed-off-by: Frederic Weisbecker