diff options
| author | Heiko Carstens <hca@linux.ibm.com> | 2026-01-13 22:43:58 +0300 |
|---|---|---|
| committer | Heiko Carstens <hca@linux.ibm.com> | 2026-01-17 17:52:49 +0300 |
| commit | 23ba7d31633da6b0706b4154e4eb74cdfab710ef (patch) | |
| tree | 9e43697551545e4bdad6a207b9bf02f38bdf88ba | |
| parent | 6cce3609a1e0dedeef9b4bfdc87d0d4692f691d7 (diff) | |
| download | linux-23ba7d31633da6b0706b4154e4eb74cdfab710ef.tar.xz | |
s390/preempt: Optimize __preemp_count_add()/__preempt_count_sub()
Provide an inline assembly using alternatives to avoid the need of a
base register due to relocatable lowcore when adding or subtracting
small constants from preempt_count.
Main user is preempt_disable(), which subtracts one from
preempt_count. With this the generated code changes from
10012c: a7 b9 00 00 lghi %r11,0
100130: eb 01 b3 a8 00 6a asi 936(%r11),1
to something like this:
10012c: eb 01 03 a8 00 6a asi 936,1
Kernel image size is reduced by 13kb (bloat-o-meter -t, defconfig, gcc15).
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
| -rw-r--r-- | arch/s390/include/asm/preempt.h | 12 |
1 files changed, 11 insertions, 1 deletions
diff --git a/arch/s390/include/asm/preempt.h b/arch/s390/include/asm/preempt.h index 6d6a28bee4e7..ef6ea3163c27 100644 --- a/arch/s390/include/asm/preempt.h +++ b/arch/s390/include/asm/preempt.h @@ -84,7 +84,17 @@ static __always_inline void __preempt_count_add(int val) */ if (!IS_ENABLED(CONFIG_PROFILE_ALL_BRANCHES)) { if (__builtin_constant_p(val) && (val >= -128) && (val <= 127)) { - __atomic_add_const(val, &get_lowcore()->preempt_count); + unsigned long lc_preempt; + + lc_preempt = offsetof(struct lowcore, preempt_count); + asm_inline( + ALTERNATIVE("asi %[offzero](%%r0),%[val]\n", + "asi %[offalt](%%r0),%[val]\n", + ALT_FEATURE(MFEATURE_LOWCORE)) + : "+m" (((struct lowcore *)0)->preempt_count) + : [offzero] "i" (lc_preempt), [val] "i" (val), + [offalt] "i" (lc_preempt + LOWCORE_ALT_ADDRESS) + : "cc"); return; } } |