diff options
| author | Steve Capper <steve.capper@linaro.org> | 2014-11-19 19:53:43 +0300 |
|---|---|---|
| committer | Will Deacon <will.deacon@arm.com> | 2014-11-20 15:05:18 +0300 |
| commit | f97fc810798c261b2790c2a1660461a508a479e0 (patch) | |
| tree | d0778121159f47b0337c368e5f2f1be35082d1fb /arch/arm64/include/asm/cmpxchg.h | |
| parent | 15670ef1eac9817cf48da12c885aabcdd88e9add (diff) | |
| download | linux-f97fc810798c261b2790c2a1660461a508a479e0.tar.xz | |
arm64: percpu: Implement this_cpu operations
The generic this_cpu operations disable interrupts to ensure that the
requested operation is protected from pre-emption. For arm64, this is
overkill and can hurt throughput and latency.
This patch provides arm64 specific implementations for the this_cpu
operations. Rather than disable interrupts, we use the exclusive
monitor or atomic operations as appropriate.
The following operations are implemented: add, add_return, and, or,
read, write, xchg. We also wire up a cmpxchg implementation from
cmpxchg.h.
Testing was performed using the percpu_test module and hackbench on a
Juno board running 3.18-rc4.
Signed-off-by: Steve Capper <steve.capper@linaro.org>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Diffstat (limited to 'arch/arm64/include/asm/cmpxchg.h')
| -rw-r--r-- | arch/arm64/include/asm/cmpxchg.h | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/arch/arm64/include/asm/cmpxchg.h b/arch/arm64/include/asm/cmpxchg.h index 89e397befad5..cb9593079f29 100644 --- a/arch/arm64/include/asm/cmpxchg.h +++ b/arch/arm64/include/asm/cmpxchg.h @@ -246,8 +246,10 @@ static inline unsigned long __cmpxchg_mb(volatile void *ptr, unsigned long old, __ret; \ }) -#define this_cpu_cmpxchg_8(ptr, o, n) \ - cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n) +#define this_cpu_cmpxchg_1(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n) +#define this_cpu_cmpxchg_2(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n) +#define this_cpu_cmpxchg_4(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n) +#define this_cpu_cmpxchg_8(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n) #define this_cpu_cmpxchg_double_8(ptr1, ptr2, o1, o2, n1, n2) \ cmpxchg_double_local(raw_cpu_ptr(&(ptr1)), raw_cpu_ptr(&(ptr2)), \ |