kernel/linux.git/arch/arm64/lib/delay.c, branch v6.18.22

arm64: Fix sampling the "stable" virtual counter in preemptible section

2026-03-04T12:21:36+00:00

[ Upstream commit e5cb94ba5f96d691d8885175d4696d6ae6bc5ec9 ] Ben reports that when running with CONFIG_DEBUG_PREEMPT, using __arch_counter_get_cntvct_stable() results in well deserves warnings, as we access a per-CPU variable without preemption disabled. Fix the issue by disabling preemption on reading the counter. We can probably do a lot better by not disabling preemption on systems that do not require horrible workarounds to return a valid counter value, but this plugs the issue for the time being. Fixes: 29cc0f3aa7c6 ("arm64: Force the use of CNTVCT_EL0 in __delay()") Reported-by: Ben Horgan Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/aZw3EGs4rbQvbAzV@e134344.arm.com Tested-by: Ben Horgan Tested-by: André Draszik Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

arm64: Force the use of CNTVCT_EL0 in __delay()

2026-03-04T12:21:34+00:00

[ Upstream commit 29cc0f3aa7c64d3b3cb9d94c0a0984ba6717bf72 ] Quentin forwards a report from Hyesoo Yu, describing an interesting problem with the use of WFxT in __delay() when a vcpu is loaded and that KVM is *not* in VHE mode (either nVHE or hVHE). In this case, CNTVOFF_EL2 is set to a non-zero value to reflect the state of the guest virtual counter. At the same time, __delay() is using get_cycles() to read the counter value, which is indirected to reading CNTPCT_EL0. The core of the issue is that WFxT is using the *virtual* counter, while the kernel is using the physical counter, and that the offset introduces a really bad discrepancy between the two. Fix this by forcing the use of CNTVCT_EL0, making __delay() consistent irrespective of the value of CNTVOFF_EL2. Reported-by: Hyesoo Yu Reported-by: Quentin Perret Reviewed-by: Quentin Perret Fixes: 7d26b0516a0d ("arm64: Use WFxT for __delay() when possible") Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/ktosachvft2cgqd5qkukn275ugmhy6xrhxur4zqpdxlfr3qh5h@o3zrfnsq63od Cc: stable@vger.kernel.org Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

arm64: Avoid cpus_have_const_cap() for ARM64_HAS_WFXT

2023-10-16T13:17:05+00:00

In __delay() we use cpus_have_const_cap() to check for ARM64_HAS_WFXT, but this is not necessary and alternative_has_cap() would be preferable. For historical reasons, cpus_have_const_cap() is more complicated than it needs to be. Before cpucaps are finalized, it will perform a bitmap test of the system_cpucaps bitmap, and once cpucaps are finalized it will use an alternative branch. This used to be necessary to handle some race conditions in the window between cpucap detection and the subsequent patching of alternatives and static branches, where different branches could be out-of-sync with one another (or w.r.t. alternative sequences). Now that we use alternative branches instead of static branches, these are all patched atomically w.r.t. one another, and there are only a handful of cases that need special care in the window between cpucap detection and alternative patching. Due to the above, it would be nice to remove cpus_have_const_cap(), and migrate callers over to alternative_has_cap_*(), cpus_have_final_cap(), or cpus_have_cap() depending on when their requirements. This will remove redundant instructions and improve code generation, and will make it easier to determine how each callsite will behave before, during, and after alternative patching. The cpus_have_const_cap() check in __delay() is an optimization to use WFIT and WFET in preference to busy-polling the counter and/or using regular WFE and relying upon the architected timer event stream. It is not necessary to apply this optimization in the window between detecting the ARM64_HAS_WFXT cpucap and patching alternatives. This patch replaces the use of cpus_have_const_cap() with alternative_has_cap_unlikely(), which will avoid generating code to test the system_cpucaps bitmap and should be better for all subsequent calls at runtime. Signed-off-by: Mark Rutland Cc: Marc Zyngier Cc: Suzuki K Poulose Cc: Will Deacon Signed-off-by: Catalin Marinas

arm64: Use WFxT for __delay() when possible

2022-04-20T12:24:44+00:00

Marginally optimise __delay() by using a WFIT/WFET sequence. It probably is a win if no interrupt fires during the delay. Signed-off-by: Marc Zyngier Acked-by: Catalin Marinas Link: https://lore.kernel.org/r/20220419182755.601427-11-maz@kernel.org

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234

2019-06-19T15:09:07+00:00

Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not see http www gnu org licenses extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 503 file(s). Signed-off-by: Thomas Gleixner Reviewed-by: Alexios Zavras Reviewed-by: Allison Randal Reviewed-by: Enrico Weigelt Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190602204653.811534538@linutronix.de Signed-off-by: Greg Kroah-Hartman

arm64: use WFE for long delays

2017-10-13T17:56:15+00:00

The current delay implementation uses the yield instruction, which is a hint that it is beneficial to schedule another thread. As this is a hint, it may be implemented as a NOP, causing all delays to be busy loops. This is the case for many existing CPUs. Taking advantage of the generic timer sending periodic events to all cores, we can use WFE during delays to reduce power consumption. This is beneficial only for delays longer than the period of the timer event stream. If timer event stream is not enabled, delays will behave as yield/busy loops. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Signed-off-by: Will Deacon

arm64: Miscellaneous library functions

2012-09-17T12:42:18+00:00

This patch adds udelay, memory and bit operations together with the ksyms exports. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Acked-by: Tony Lindgren Acked-by: Nicolas Pitre Acked-by: Olof Johansson Acked-by: Santosh Shilimkar