kernel/linux.git/arch/x86/include/asm/percpu.h, branch v6.12.80

x86/locking: Use ALT_OUTPUT_SP() for percpu_{,try_}cmpxchg{64,128}_op()

2025-05-29T09:02:41+00:00

[ Upstream commit 4087e16b033140cf2ce509ec23503bddec818a16 ] percpu_{,try_}cmpxchg{64,128}() macros use CALL instruction inside asm statement in one of their alternatives. Use ALT_OUTPUT_SP() macro to add required dependence on %esp register. ALT_OUTPUT_SP() implements the above dependence by adding ASM_CALL_CONSTRAINT to its arguments. This constraint should be used for any inline asm which has a CALL instruction, otherwise the compiler may schedule the asm before the frame pointer gets set up by the containing function, causing objtool to print a "call without frame pointer save/setup" warning. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Linus Torvalds Link: https://lore.kernel.org/r/20250214150929.5780-1-ubizjak@gmail.com Signed-off-by: Sasha Levin

x86/percpu: Clean up vertical alignment details

2024-05-20T08:56:25+00:00

- Fix/unify misc vertical alignment inconsistencies - Make CPP macros look a bit more like C code by adding an empty line after local variable declaration blocks, and before final rvalue statements. No change in code. Signed-off-by: Ingo Molnar Cc: Uros Bizjak Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: linux-kernel@vger.kernel.org

x86/percpu: Clean up a bit

2024-05-20T08:56:23+00:00

- Fix misc typos - There's 4 variants of the same spelling right now: 'per-CPU', 'per CPU', 'percpu' and 'per-cpu' Standardize on 'per-CPU' only. - s/makes gcc load /makes the compiler load - Instead of: #ifdef CONFIG_XXXX #define YYYY FOO #else #define YYYY BAR #endif Use the slightly more readable form of: #ifdef CONFIG_XXXX # define YYYY FOO #else # define YYYY BAR #endif - Standardize & expand '#else' and '#endif' comments - Fix comment style - Capitalize x86 instruction names in comments No change in code. Signed-off-by: Ingo Molnar Cc: Uros Bizjak Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: linux-kernel@vger.kernel.org

x86/percpu: Move some percpu accessors around to reduce ifdeffery

2024-05-20T08:25:31+00:00

Move some percpu accessors around, mainly to reduce ifdeffery and improve readabilty by following dependencies between accessors. No functional change intended. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240520080951.121049-2-ubizjak@gmail.com

x86/percpu: Rename percpu_stable_op() to __raw_cpu_read_stable()

2024-05-20T08:17:10+00:00

Rename percpu_stable_op() to __raw_cpu_read_stable() to be in line with other read/write percpu accessors. No functional change intended. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Uros Bizjak Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240520080951.121049-1-ubizjak@gmail.com

x86/percpu: Fix operand constraint modifier in __raw_cpu_write()

2024-05-18T09:18:42+00:00

__raw_cpu_write() with !USE_X86_SEG_SUPPORT config uses read/write operand constraint modifier "+" for its memory location. This signals the compiler that the location is both read and written by the asm. This is not true, because MOV insn only writes to the output. Correct the modifier to "=" to inform the compiler that the memory location is only written to. This also prevents the compiler from value tracking the undefined value from the uninitialized memory. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240430091833.196482-5-ubizjak@gmail.com

x86/percpu: Introduce the __raw_cpu_read_const() macro

2024-05-18T09:18:42+00:00

Introduce the __raw_cpu_read_const() macro to further reduce ifdeffery and differences between configs w/ and w/o USE_X86_SEG_SUPPORT. No functional change intended. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240430091833.196482-4-ubizjak@gmail.com

x86/percpu: Unify percpu read-write accessors

2024-05-18T09:18:41+00:00

Redefine percpu_from_op() and percpu_to_op() as __raw_cpu_read() and __raw_cpu_write(). Unify __raw_cpu_{read,write}() macros between configs w/ and w/o USE_X86_SEG_SUPPORT in order to unify {raw,this}_cpu{read_write}_N() accessors between configs. No functional change intended. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240430091833.196482-3-ubizjak@gmail.com

x86/percpu: Move some percpu macros around for readability

2024-05-18T09:18:41+00:00

Move some percpu macros around to make a follow-up patch more readable. No functional change intended. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240430091833.196482-2-ubizjak@gmail.com

x86/percpu: Introduce the pcpu_binary_op() macro

2024-05-18T09:18:40+00:00

Introduce the pcpu_binary_op() macro, a copy of the percpu_to_op() macro. Update percpu binary operators to use the new macro, since percpu_to_op() will be re-purposed as a raw percpu write accessor in a follow-up patch. No functional change intended. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Andy Lutomirski Cc: Josh Poimboeuf Cc: Linus Torvalds Link: https://lore.kernel.org/r/20240430091833.196482-1-ubizjak@gmail.com