diff options
| author | Dmitry Ilvokhin <d@ilvokhin.com> | 2026-06-04 10:15:06 +0300 |
|---|---|---|
| committer | Peter Zijlstra <peterz@infradead.org> | 2026-06-11 14:41:25 +0300 |
| commit | 7cfa62cf9432df67b1c95a59984a03a3bc023f98 (patch) | |
| tree | 7f6f150ec067fdd24a812f7ee88682d69fb0ecf7 /include/linux | |
| parent | 76080d81e511f66eeb094b204e1e217bff674922 (diff) | |
| download | linux-7cfa62cf9432df67b1c95a59984a03a3bc023f98.tar.xz | |
locking/percpu-rwsem: Extract __percpu_up_read()
Move the percpu_up_read() slowpath out of the inline function into a new
__percpu_up_read() to avoid binary size increase from adding a
tracepoint to an inlined function.
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Usama Arif <usama.arif@linux.dev>
Link: https://patch.msgid.link/3dd2a1b9ab4f469e1892766cb63f41d6b0f53d29.1780506267.git.d@ilvokhin.com
Diffstat (limited to 'include/linux')
| -rw-r--r-- | include/linux/percpu-rwsem.h | 15 |
1 files changed, 3 insertions, 12 deletions
diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index c8cb010d655e..39d5bf8e6562 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -107,6 +107,8 @@ static inline bool percpu_down_read_trylock(struct percpu_rw_semaphore *sem) return ret; } +extern void __percpu_up_read(struct percpu_rw_semaphore *sem); + static inline void percpu_up_read(struct percpu_rw_semaphore *sem) { rwsem_release(&sem->dep_map, _RET_IP_); @@ -118,18 +120,7 @@ static inline void percpu_up_read(struct percpu_rw_semaphore *sem) if (likely(rcu_sync_is_idle(&sem->rss))) { this_cpu_dec(*sem->read_count); } else { - /* - * slowpath; reader will only ever wake a single blocked - * writer. - */ - smp_mb(); /* B matches C */ - /* - * In other words, if they see our decrement (presumably to - * aggregate zero, as that is the only time it matters) they - * will also see our critical section. - */ - this_cpu_dec(*sem->read_count); - rcuwait_wake_up(&sem->writer); + __percpu_up_read(sem); } preempt_enable(); } |