kernel/linux.git/arch/mips/include/asm/spinlock.h, branch v6.19.12

locking/arch: Move qrwlock.h include after qspinlock.h

2021-02-11T12:59:54+00:00

include/asm-generic/qrwlock.h was trying to get arch_spin_is_locked via asm-generic/qspinlock.h. However, this does not work because architectures might be using queued rwlocks but not queued spinlocks (csky), or because they might be defining their own queued_* macros before including asm/qspinlock.h. To fix this, ensure that asm/spinlock.h always includes qrwlock.h after defining arch_spin_is_locked (either directly for csky, or via asm/qspinlock.h for other architectures). The only inclusion elsewhere is in kernel/locking/qrwlock.c. That one is really unnecessary because the file is only compiled in SMP configurations (config QUEUED_RWLOCKS depends on SMP) and in that case linux/spinlock.h already includes asm/qrwlock.h if needed, via asm/spinlock.h. Reported-by: Guenter Roeck Signed-off-by: Waiman Long Fixes: 26128cb6c7e6 ("locking/rwlocks: Add contention detection for rwlocks") Tested-by: Guenter Roeck Reviewed-by: Ben Gardon [Add arch/sparc and kernel/locking parts per discussion with Waiman. - Paolo] Signed-off-by: Paolo Bonzini

mips/mmiowb: Add unconditional mmiowb() to arch_spin_unlock()

2019-04-08T11:00:28+00:00

The mmiowb() macro is horribly difficult to use and drivers will continue to work most of the time if they omit a call when it is required. Rather than rely on driver authors getting this right, push mmiowb() into arch_spin_unlock() for mips. If this is deemed to be a performance issue, a subsequent optimisation could make use of ARCH_HAS_MMIOWB to elide the barrier in cases where no I/O writes were performed inside the critical section. Acked-by: Paul Burton Acked-by: Linus Torvalds Signed-off-by: Will Deacon

locking/arch: Remove dummy arch_{read,spin,write}_lock_flags() implementations

2017-10-10T09:50:19+00:00

The arch_{read,spin,write}_lock_flags() macros are simply mapped to the non-flags versions by the majority of architectures, so do this in core code and remove the dummy implementations. Also remove the implementation in spinlock_up.h, since all callers of do_raw_spin_lock_flags() call local_irq_save(flags) anyway. Signed-off-by: Will Deacon Signed-off-by: Peter Zijlstra (Intel) Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: paulmck@linux.vnet.ibm.com Link: http://lkml.kernel.org/r/1507055129-12300-4-git-send-email-will.deacon@arm.com Signed-off-by: Ingo Molnar

locking/arch: Remove dummy arch_{read,spin,write}_relax() implementations

2017-10-10T09:50:18+00:00

arch_{read,spin,write}_relax() are defined as cpu_relax() by the core code, so architectures that can't do better (i.e. most of them) don't need to bother with the dummy definitions. Signed-off-by: Will Deacon Signed-off-by: Peter Zijlstra (Intel) Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: paulmck@linux.vnet.ibm.com Link: http://lkml.kernel.org/r/1507055129-12300-3-git-send-email-will.deacon@arm.com Signed-off-by: Ingo Molnar

MIPS: Use queued spinlocks (qspinlock)

2017-06-29T00:42:26+00:00

This patch switches MIPS to make use of generically implemented queued spinlocks, rather than the ticket spinlocks used previously. This allows us to drop a whole load of inline assembly, share more generic code, and is also a performance win. Results from running the AIM7 short workload on a MIPS Creator Ci40 (ie. 2 core 2 thread interAptiv CPU clocked at 546MHz) with v4.12-rc4 pistachio_defconfig, with ftrace disabled due to a current bug, and both with & without use of queued rwlocks & spinlocks: Forks | v4.12-rc4 | +qlocks | Change -------|-----------|----------|-------- 10 | 52630.32 | 53316.31 | +1.01% 20 | 51777.80 | 52623.15 | +1.02% 30 | 51645.92 | 52517.26 | +1.02% 40 | 51634.88 | 52419.89 | +1.02% 50 | 51506.75 | 52307.81 | +1.02% 60 | 51500.74 | 52322.72 | +1.02% 70 | 51434.81 | 52288.60 | +1.02% 80 | 51423.22 | 52434.85 | +1.02% 90 | 51428.65 | 52410.10 | +1.02% The kernels used for these tests also had my "MIPS: Hardcode cpu_has_* where known at compile time due to ISA" patch applied, which allows the kernel_uses_llsc checks in cmpxchg() & xchg() to be optimised away at compile time. Signed-off-by: Paul Burton Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16358/ Signed-off-by: Ralf Baechle

MIPS: Use queued read/write locks (qrwlock)

2017-06-29T00:42:25+00:00

This patch switches MIPS to make use of generically implemented queued read/write locks, rather than the custom implementation used previously. This allows us to drop a whole load of inline assembly, share more generic code, and is also a performance win. Results from running the AIM7 short workload on a MIPS Creator Ci40 (ie. 2 core 2 thread interAptiv CPU clocked at 546MHz) with v4.12-rc4 pistachio_defconfig, with ftrace disabled due to a current bug, and both with & without use of queued rwlocks & spinlocks: Forks | v4.12-rc4 | +qlocks | Change -------|-----------|----------|-------- 10 | 52630.32 | 53316.31 | +1.01% 20 | 51777.80 | 52623.15 | +1.02% 30 | 51645.92 | 52517.26 | +1.02% 40 | 51634.88 | 52419.89 | +1.02% 50 | 51506.75 | 52307.81 | +1.02% 60 | 51500.74 | 52322.72 | +1.02% 70 | 51434.81 | 52288.60 | +1.02% 80 | 51423.22 | 52434.85 | +1.02% 90 | 51428.65 | 52410.10 | +1.02% The kernels used for these tests also had my "MIPS: Hardcode cpu_has_* where known at compile time due to ISA" patch applied, which allows the kernel_uses_llsc checks in cmpxchg() & xchg() to be optimised away at compile time. Signed-off-by: Paul Burton Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/16357/ Signed-off-by: Ralf Baechle

MIPS: End spinlocks with .insn

2017-03-08T10:16:26+00:00

When building for microMIPS we need to ensure that the assembler always knows that there is code at the target of a branch or jump. Recent toolchains will fail to link a microMIPS kernel when this isn't the case due to what it thinks is a branch to non-microMIPS code. mips-mti-linux-gnu-ld kernel/built-in.o: .spinlock.text+0x2fc: Unsupported branch between ISA modes. mips-mti-linux-gnu-ld final link failed: Bad value This is due to inline assembly labels in spinlock.h not being followed by an instruction mnemonic, either due to a .subsection pseudo-op or the end of the inline asm block. Fix this with a .insn direction after such labels. Signed-off-by: Paul Burton Signed-off-by: James Hogan Reviewed-by: Maciej W. Rozycki Cc: Ralf Baechle Cc: Peter Zijlstra Cc: Ingo Molnar Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: Patchwork: https://patchwork.linux-mips.org/patch/15325/ Signed-off-by: James Hogan

locking/spinlock, arch: Update and fix spin_unlock_wait() implementations

2016-06-14T09:55:15+00:00

This patch updates/fixes all spin_unlock_wait() implementations. The update is in semantics; where it previously was only a control dependency, we now upgrade to a full load-acquire to match the store-release from the spin_unlock() we waited on. This ensures that when spin_unlock_wait() returns, we're guaranteed to observe the full critical section we waited on. This fixes a number of spin_unlock_wait() users that (not unreasonably) rely on this. I also fixed a number of ticket lock versions to only wait on the current lock holder, instead of for a full unlock, as this is sufficient. Furthermore; again for ticket locks; I added an smp_rmb() in between the initial ticket load and the spin loop testing the current value because I could not convince myself the address dependency is sufficient, esp. if the loads are of different sizes. I'm more than happy to remove this smp_rmb() again if people are certain the address dependency does indeed work as expected. Note: PPC32 will be fixed independently Signed-off-by: Peter Zijlstra (Intel) Cc: Andrew Morton Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: chris@zankel.net Cc: cmetcalf@mellanox.com Cc: davem@davemloft.net Cc: dhowells@redhat.com Cc: james.hogan@imgtec.com Cc: jejb@parisc-linux.org Cc: linux@armlinux.org.uk Cc: mpe@ellerman.id.au Cc: ralf@linux-mips.org Cc: realmz6@gmail.com Cc: rkuo@codeaurora.org Cc: rth@twiddle.net Cc: schwidefsky@de.ibm.com Cc: tony.luck@intel.com Cc: vgupta@synopsys.com Cc: ysato@users.sourceforge.jp Cc: linux-kernel@vger.kernel.org Signed-off-by: Ingo Molnar

MIPS: Select CONFIG_ARCH_USE_CMPXCHG_LOCKREF for MIPS64

2015-09-03T10:08:01+00:00

On MIPS64 we have spinlocks that are 32b in size and an efficient cmpxchg64 implementation, so we qualify to make use of cmpxchg backed lockrefs. Select the ARCH_USE_CMPXCHG_LOCKREF Kconfig symbol and provide a trivial implementation of arch_spin_value_unlocked to satisfy the lockref code. Using Linus' simple testcase from http://article.gmane.org/gmane.linux.file-systems/77466 on a dual core system with an in-development MIPS64 CPU running on FPGA I see around an 8% gain: Pre-patch: Total loops: 252698 Total loops: 251482 Total loops: 250806 Total loops: 252885 Total loops: 251666 Post-patch: Total loops: 273728 Total loops: 269932 Total loops: 269341 Total loops: 275004 Total loops: 270208 [ralf@linux-mips.org: Fixed conflict.] Signed-off-by: Paul Burton Cc: linux-mips@linux-mips.org Cc: Steven J. Hill Cc: linux-kernel@vger.kernel.org Cc: Maciej W. Rozycki Cc: Markos Chandras Patchwork: https://patchwork.linux-mips.org/patch/10810/ Signed-off-by: Ralf Baechle

MIPS: spinlock: Adjust arch_spin_lock back-off time

2015-06-24T12:57:46+00:00

Make it similar to the trylock and R10000_LLSC_WAR cases. Signed-off-by: Markos Chandras Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/9789/ Signed-off-by: Ralf Baechle