kernel/linux.git/arch/powerpc/include/asm/atomic.h, branch v6.19.11

powerpc/atomic: Use YZ constraints for DS-form instructions

2024-09-17T12:17:57+00:00

The 'ld' and 'std' instructions require a 4-byte aligned displacement because they are DS-form instructions. But the "m" asm constraint doesn't enforce that. That can lead to build errors if the compiler chooses a non-aligned displacement, as seen with GCC 14: /tmp/ccuSzwiR.s: Assembler messages: /tmp/ccuSzwiR.s:2579: Error: operand out of domain (39 is not a multiple of 4) make[5]: *** [scripts/Makefile.build:229: net/core/page_pool.o] Error 1 Dumping the generated assembler shows: ld 8,39(8) # MEM[(const struct atomic64_t *)_29].counter, t Use the YZ constraints to tell the compiler either to generate a DS-form displacement, or use an X-form instruction, either of which prevents the build error. See commit 2d43cc701b96 ("powerpc/uaccess: Fix build errors seen with GCC 13/14") for more details on the constraint letters. Fixes: 9f0cbea0d8cc ("[POWERPC] Implement atomic{, 64}_{read, write}() without volatile") Cc: stable@vger.kernel.org # v2.6.24+ Reported-by: Stephen Rothwell Closes: https://lore.kernel.org/all/20240913125302.0a06b4c7@canb.auug.org.au Tested-by: Mina Almasry Reviewed-by: Segher Boessenkool Signed-off-by: Michael Ellerman Link: https://msgid.link/20240916120510.2017749-1-mpe@ellerman.id.au

locking/atomic: make atomic*_{cmp,}xchg optional

2023-06-05T07:57:14+00:00

Most architectures define the atomic/atomic64 xchg and cmpxchg operations in terms of arch_xchg and arch_cmpxchg respectfully. Add fallbacks for these cases and remove the trivial cases from arch code. On some architectures the existing definitions are kept as these are used to build other arch_atomic*() operations. Signed-off-by: Mark Rutland Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Kees Cook Link: https://lore.kernel.org/r/20230605070124.3741859-5-mark.rutland@arm.com

powerpc/64: Add support to build with prefixed instructions

2023-04-20T02:54:22+00:00

Add an option to build kernel and module with prefixed instructions if the CPU and toolchain support it. This is not related to kernel support for userspace execution of prefixed instructions. Building with prefixed instructions breaks some extended inline asm memory addressing, for example it will provide immediates that exceed the range of simple load/store displacement. Whether this is a toolchain or a kernel asm problem remains to be seen. For now, these are replaced with simpler and less efficient direct register addressing when compiling with prefixed. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://msgid.link/20230408021752.862660-4-npiggin@gmail.com

powerpc/atomics: Remove unused function

2023-04-04T12:11:03+00:00

Remove arch_atomic_try_cmpxchg_lock function as it is no longer used since commit 9f61521c7a28 ("powerpc/qspinlock: powerpc qspinlock implementation") Signed-off-by: Nysal Jan K.A Reviewed-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://msgid.link/20230224103940.1328725-1-nysal@linux.ibm.com

powerpc: Make eh value more explicit when using lwarx

2022-08-10T05:32:20+00:00

Just like the first patch of this series, define a local 'eh' in order to make the code clearer. And IS_ENABLED() returns either 1 or 0 so no need to do IS_ENABLED(CONFIG_PPC64) ? 1 : 0. Signed-off-by: Christophe Leroy [mpe: Use symbolic names, use 'n' constraint per Segher] Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/629befaa2d05e2922346e58a383886510d6af55a.1659430931.git.christophe.leroy@csgroup.eu

powerpc/atomics: Remove atomic_inc()/atomic_dec() and friends

2021-11-30T00:45:57+00:00

Now that atomic_add() and atomic_sub() handle immediate operands, atomic_inc() and atomic_dec() have no added value compared to the generic fallback which calls atomic_add(1) and atomic_sub(1). Also remove atomic_inc_not_zero() which fallsback to atomic_add_unless() which itself fallsback to atomic_fetch_add_unless() which now handles immediate operands. Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/0bc64a2f18726055093dbb2e479cefc60a409cfd.1632236981.git.christophe.leroy@csgroup.eu

powerpc/atomics: Use immediate operand when possible

2021-11-30T00:45:57+00:00

Today we get the following code generation for atomic operations: c001bb2c: 39 20 00 01 li r9,1 c001bb30: 7d 40 18 28 lwarx r10,0,r3 c001bb34: 7d 09 50 50 subf r8,r9,r10 c001bb38: 7d 00 19 2d stwcx. r8,0,r3 c001c7a8: 39 40 00 01 li r10,1 c001c7ac: 7d 00 18 28 lwarx r8,0,r3 c001c7b0: 7c ea 42 14 add r7,r10,r8 c001c7b4: 7c e0 19 2d stwcx. r7,0,r3 By allowing GCC to choose between immediate or regular operation, we get: c001bb2c: 7d 20 18 28 lwarx r9,0,r3 c001bb30: 39 49 ff ff addi r10,r9,-1 c001bb34: 7d 40 19 2d stwcx. r10,0,r3 -- c001c7a4: 7d 40 18 28 lwarx r10,0,r3 c001c7a8: 39 0a 00 01 addi r8,r10,1 c001c7ac: 7d 00 19 2d stwcx. r8,0,r3 For "and", the dot form has to be used because "andi" doesn't exist. For logical operations we use unsigned 16 bits immediate. For arithmetic operations we use signed 16 bits immediate. On pmac32_defconfig, it reduces the text by approx another 8 kbytes. Signed-off-by: Christophe Leroy Acked-by: Segher Boessenkool Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/2ec558d44db8045752fe9dbd29c9ba84bab6030b.1632236981.git.christophe.leroy@csgroup.eu

powerpc/asm: Remove UPD_CONSTR after GCC 4.9 removal

2021-10-08T13:15:59+00:00

UPD_CONSTR was previously a preprocessor define for an old GCC 4.9 inline asm bug with m<> constraints. Fixes: 6563139d90ad ("powerpc: remove GCC version check for UPD_CONSTR") Suggested-by: Nathan Chancellor Suggested-by: Christophe Leroy Suggested-by: Michael Ellerman Signed-off-by: Nick Desaulniers Reviewed-by: Nathan Chancellor Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20210914161712.2463458-1-ndesaulniers@google.com

powerpc: Use lwarx/ldarx directly instead of PPC_LWARX/LDARX macros

2021-08-25T03:35:49+00:00

Force the eh flag at 0 on PPC32. Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/1fc81f07cabebb875b963e295408cc3dd38c8d85.1614674882.git.christophe.leroy@csgroup.eu

locking/atomic: powerpc: move to ARCH_ATOMIC

2021-05-26T11:20:52+00:00

We'd like all architectures to convert to ARCH_ATOMIC, as once all architectures are converted it will be possible to make significant cleanups to the atomics headers, and this will make it much easier to generically enable atomic functionality (e.g. debug logic in the instrumented wrappers). As a step towards that, this patch migrates powerpc to ARCH_ATOMIC. The arch code provides arch_{atomic,atomic64,xchg,cmpxchg}*(), and common code wraps these with optional instrumentation to provide the regular functions. While atomic_try_cmpxchg_lock() is not part of the common atomic API, it is given an `arch_` prefix for consistency. Signed-off-by: Mark Rutland Cc: Benjamin Herrenschmidt Cc: Boqun Feng Cc: Michael Ellerman Cc: Paul Mackerras Cc: Peter Zijlstra Cc: Will Deacon Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20210525140232.53872-28-mark.rutland@arm.com