kernel/linux.git/include/linux/hrtimer.h, branch v6.12.80

hrtimers: Make hrtimer_update_function() less expensive

2026-01-08T09:15:03+00:00

commit 2ea97b76d6712bfb0408e5b81ffd7bc4551d3153 upstream. The sanity checks in hrtimer_update_function() are expensive for high frequency usage like in the io/uring code due to locking. Hide the sanity checks behind CONFIG_PROVE_LOCKING, which has a decent chance to be enabled on a regular basis for testing. Fixes: 8f02e3563bb5 ("hrtimers: Introduce hrtimer_update_function()") Reported-by: Jens Axboe Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/all/87ikpllali.ffs@tglx Signed-off-by: Greg Kroah-Hartman

hrtimers: Introduce hrtimer_update_function()

2026-01-08T09:14:59+00:00

[ Upstream commit 8f02e3563bb5824eb01c94f2c75f1dcee2d05625 ] Some users of hrtimer need to change the callback function after the initial setup. They write to hrtimer::function directly. That's not safe under all circumstances as the write is lockless and a concurrent timer expiry might end up using the wrong function pointer. Introduce hrtimer_update_function(), which also performs runtime checks whether it is safe to modify the callback. This allows to make hrtimer::function private once all users are converted. Signed-off-by: Nam Cao Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/all/20a937b0ae09ad54b5b6d86eabead7c570f1b72e.1730386209.git.namcao@linutronix.de Stable-dep-of: 267ee93c417e ("serial: xilinx_uartps: fix rs485 delay_rts_after_send") Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

hrtimers: Handle CPU state correctly on hotplug

2025-01-23T16:23:03+00:00

commit 2f8dea1692eef2b7ba6a256246ed82c365fdc686 upstream. Consider a scenario where a CPU transitions from CPUHP_ONLINE to halfway through a CPU hotunplug down to CPUHP_HRTIMERS_PREPARE, and then back to CPUHP_ONLINE: Since hrtimers_prepare_cpu() does not run, cpu_base.hres_active remains set to 1 throughout. However, during a CPU unplug operation, the tick and the clockevents are shut down at CPUHP_AP_TICK_DYING. On return to the online state, for instance CFS incorrectly assumes that the hrtick is already active, and the chance of the clockevent device to transition to oneshot mode is also lost forever for the CPU, unless it goes back to a lower state than CPUHP_HRTIMERS_PREPARE once. This round-trip reveals another issue; cpu_base.online is not set to 1 after the transition, which appears as a WARN_ON_ONCE in enqueue_hrtimer(). Aside of that, the bulk of the per CPU state is not reset either, which means there are dangling pointers in the worst case. Address this by adding a corresponding startup() callback, which resets the stale per CPU state and sets the online flag. [ tglx: Make the new callback unconditionally available, remove the online modification in the prepare() callback and clear the remaining state in the starting callback instead of the prepare callback ] Fixes: 5c0930ccaad5 ("hrtimers: Push pending hrtimers away from outgoing CPU earlier") Signed-off-by: Koichiro Den Signed-off-by: Thomas Gleixner Cc: stable@vger.kernel.org Link: https://lore.kernel.org/all/20241220134421.3809834-1-koichiro.den@canonical.com Signed-off-by: Greg Kroah-Hartman

hrtimers: Update formatting of documentation

2024-02-19T08:37:59+00:00

Documentation of functions lacks the annotations which are used by kernel-doc and *.rst to make appearance in rendered documents more user-friendly. Use those annotations to improve user-friendliness. While at it prevent duplication of comments and use a reference instead. Signed-off-by: Anna-Maria Behnsen Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/20240123164702.55612-3-anna-maria@linutronix.de

hrtimers: Move hrtimer base related definitions into hrtimer_defs.h

2024-02-19T08:37:59+00:00

hrtimer base related struct definitions are part of hrtimers.h as it is required there. With this, also the struct documentation which is for core code internal use, is exposed into the general api. To prevent this, move all core internal definitions and the related includes into hrtimer_defs.h. Signed-off-by: Anna-Maria Behnsen Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/20240123164702.55612-2-anna-maria@linutronix.de

hrtimers: Split out hrtimer_types.h

2023-12-21T00:26:30+00:00

We need to reduce the scope of what's included in sched.h: task_struct includes a hrtimer, so split out the core types into their own header. Signed-off-by: Kent Overstreet Cc: Thomas Gleixner

timekeeping: Kill percpu.h dependency

2023-12-21T00:26:30+00:00

Slimming down recursive header includes. Signed-off-by: Kent Overstreet Cc: Thomas Gleixner

hrtimers: Push pending hrtimers away from outgoing CPU earlier

2023-11-11T17:06:42+00:00

2b8272ff4a70 ("cpu/hotplug: Prevent self deadlock on CPU hot-unplug") solved the straight forward CPU hotplug deadlock vs. the scheduler bandwidth timer. Yu discovered a more involved variant where a task which has a bandwidth timer started on the outgoing CPU holds a lock and then gets throttled. If the lock required by one of the CPU hotplug callbacks the hotplug operation deadlocks because the unthrottling timer event is not handled on the dying CPU and can only be recovered once the control CPU reaches the hotplug state which pulls the pending hrtimers from the dead CPU. Solve this by pushing the hrtimers away from the dying CPU in the dying callbacks. Nothing can queue a hrtimer on the dying CPU at that point because all other CPUs spin in stop_machine() with interrupts disabled and once the operation is finished the CPU is marked offline. Reported-by: Yu Liao Signed-off-by: Thomas Gleixner Tested-by: Liu Tie Link: https://lore.kernel.org/r/87a5rphara.ffs@tglx

timekeeping: Distangle resume and clock-was-set events

2021-08-10T15:57:23+00:00

Resuming timekeeping is a clock-was-set event and uses the clock-was-set notification mechanism. This is in the way of making the clock-was-set update for hrtimers selective so unnecessary IPIs are avoided when a CPU base does not have timers queued which are affected by the clock setting. Distangle it by invoking hrtimer_resume() on each unfreezing CPU and invoke the new timerfd_resume() function from timekeeping_resume() which is the only place where this is needed. Rename hrtimer_resume() to hrtimer_resume_local() to reflect the change. With this the clock_was_set*() functions are not longer required to IPI all CPUs unconditionally and can get some smarts to avoid them. Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/20210713135158.488853478@linutronix.de

timerfd: Provide timerfd_resume()

2021-08-10T15:57:22+00:00