kernel/linux.git/include/linux/cpuhotplug.h, branch v5.15.209

hrtimers: Push pending hrtimers away from outgoing CPU earlier

2023-12-13T17:36:31+00:00

[ Upstream commit 5c0930ccaad5a74d74e8b18b648c5eb21ed2fe94 ] 2b8272ff4a70 ("cpu/hotplug: Prevent self deadlock on CPU hot-unplug") solved the straight forward CPU hotplug deadlock vs. the scheduler bandwidth timer. Yu discovered a more involved variant where a task which has a bandwidth timer started on the outgoing CPU holds a lock and then gets throttled. If the lock required by one of the CPU hotplug callbacks the hotplug operation deadlocks because the unthrottling timer event is not handled on the dying CPU and can only be recovered once the control CPU reaches the hotplug state which pulls the pending hrtimers from the dead CPU. Solve this by pushing the hrtimers away from the dying CPU in the dying callbacks. Nothing can queue a hrtimer on the dying CPU at that point because all other CPUs spin in stop_machine() with interrupts disabled and once the operation is finished the CPU is marked offline. Reported-by: Yu Liao Signed-off-by: Thomas Gleixner Tested-by: Liu Tie Link: https://lore.kernel.org/r/87a5rphara.ffs@tglx Signed-off-by: Sasha Levin

arm64/arm: xen: enlighten: Fix KPTI checks

2023-11-20T10:08:20+00:00

[ Upstream commit 20f3b8eafe0ba5d3c69d5011a9b07739e9645132 ] When KPTI is in use, we cannot register a runstate region as XEN requires that this is always a valid VA, which we cannot guarantee. Due to this, xen_starting_cpu() must avoid registering each CPU's runstate region, and xen_guest_init() must avoid setting up features that depend upon it. We tried to ensure that in commit: f88af7229f6f22ce (" xen/arm: do not setup the runstate info page if kpti is enabled") ... where we added checks for xen_kernel_unmapped_at_usr(), which wraps arm64_kernel_unmapped_at_el0() on arm64 and is always false on 32-bit arm. Unfortunately, as xen_guest_init() is an early_initcall, this happens before secondary CPUs are booted and arm64 has finalized the ARM64_UNMAP_KERNEL_AT_EL0 cpucap which backs arm64_kernel_unmapped_at_el0(), and so this can subsequently be set as secondary CPUs are onlined. On a big.LITTLE system where the boot CPU does not require KPTI but some secondary CPUs do, this will result in xen_guest_init() intializing features that depend on the runstate region, and xen_starting_cpu() registering the runstate region on some CPUs before KPTI is subsequent enabled, resulting the the problems the aforementioned commit tried to avoid. Handle this more robsutly by deferring the initialization of the runstate region until secondary CPUs have been initialized and the ARM64_UNMAP_KERNEL_AT_EL0 cpucap has been finalized. The per-cpu work is moved into a new hotplug starting function which is registered later when we're certain that KPTI will not be used. Fixes: f88af7229f6f ("xen/arm: do not setup the runstate info page if kpti is enabled") Signed-off-by: Mark Rutland Cc: Bertrand Marquis Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Suzuki K Poulose Cc: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Sasha Levin

firmware: arm_sdei: Fix sleep from invalid context BUG

2023-05-24T16:36:44+00:00

[ Upstream commit d2c48b2387eb89e0bf2a2e06e30987cf410acad4 ] Running a preempt-rt (v6.2-rc3-rt1) based kernel on an Ampere Altra triggers: BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 in_atomic(): 0, irqs_disabled(): 128, non_block: 0, pid: 24, name: cpuhp/0 preempt_count: 0, expected: 0 RCU nest depth: 0, expected: 0 3 locks held by cpuhp/0/24: #0: ffffda30217c70d0 (cpu_hotplug_lock){++++}-{0:0}, at: cpuhp_thread_fun+0x5c/0x248 #1: ffffda30217c7120 (cpuhp_state-up){+.+.}-{0:0}, at: cpuhp_thread_fun+0x5c/0x248 #2: ffffda3021c711f0 (sdei_list_lock){....}-{3:3}, at: sdei_cpuhp_up+0x3c/0x130 irq event stamp: 36 hardirqs last enabled at (35): [] finish_task_switch+0xb4/0x2b0 hardirqs last disabled at (36): [] cpuhp_thread_fun+0x21c/0x248 softirqs last enabled at (0): [] copy_process+0x63c/0x1ac0 softirqs last disabled at (0): [<0000000000000000>] 0x0 CPU: 0 PID: 24 Comm: cpuhp/0 Not tainted 5.19.0-rc3-rt5-[...] Hardware name: WIWYNN Mt.Jade Server [...] Call trace: dump_backtrace+0x114/0x120 show_stack+0x20/0x70 dump_stack_lvl+0x9c/0xd8 dump_stack+0x18/0x34 __might_resched+0x188/0x228 rt_spin_lock+0x70/0x120 sdei_cpuhp_up+0x3c/0x130 cpuhp_invoke_callback+0x250/0xf08 cpuhp_thread_fun+0x120/0x248 smpboot_thread_fn+0x280/0x320 kthread+0x130/0x140 ret_from_fork+0x10/0x20 sdei_cpuhp_up() is called in the STARTING hotplug section, which runs with interrupts disabled. Use a CPUHP_AP_ONLINE_DYN entry instead to execute the cpuhp cb later, with preemption enabled. SDEI originally got its own cpuhp slot to allow interacting with perf. It got superseded by pNMI and this early slot is not relevant anymore. [1] Some SDEI calls (e.g. SDEI_1_0_FN_SDEI_PE_MASK) take actions on the calling CPU. It is checked that preemption is disabled for them. _ONLINE cpuhp cb are executed in the 'per CPU hotplug thread'. Preemption is enabled in those threads, but their cpumask is limited to 1 CPU. Move 'WARN_ON_ONCE(preemptible())' statements so that SDEI cpuhp cb don't trigger them. Also add a check for the SDEI_1_0_FN_SDEI_PRIVATE_RESET SDEI call which acts on the calling CPU. [1]: https://lore.kernel.org/all/5813b8c5-ae3e-87fd-fccc-94c9cd08816d@arm.com/ Suggested-by: James Morse Signed-off-by: Pierre Gondois Reviewed-by: James Morse Link: https://lore.kernel.org/r/20230216084920.144064-1-pierre.gondois@arm.com Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

random: clear fast pool, crng, and batches in cpuhp bring up

2022-05-30T07:29:09+00:00

commit 3191dd5a1179ef0fad5a050a1702ae98b6251e8f upstream. For the irq randomness fast pool, rather than having to use expensive atomics, which were visibly the most expensive thing in the entire irq handler, simply take care of the extreme edge case of resetting count to zero in the cpuhp online handler, just after workqueues have been reenabled. This simplifies the code a bit and lets us use vanilla variables rather than atomics, and performance should be improved. As well, very early on when the CPU comes up, while interrupts are still disabled, we clear out the per-cpu crng and its batches, so that it always starts with fresh randomness. Cc: Thomas Gleixner Cc: Peter Zijlstra Cc: Theodore Ts'o Cc: Sultan Alsawaf Cc: Dominik Brodowski Acked-by: Sebastian Andrzej Siewior Signed-off-by: Jason A. Donenfeld Signed-off-by: Greg Kroah-Hartman

mm/migrate: fix CPUHP state to update node demotion order

2021-10-19T06:22:03+00:00

The node demotion order needs to be updated during CPU hotplug. Because whether a NUMA node has CPU may influence the demotion order. The update function should be called during CPU online/offline after the node_states[N_CPU] has been updated. That is done in CPUHP_AP_ONLINE_DYN during CPU online and in CPUHP_MM_VMSTAT_DEAD during CPU offline. But in commit 884a6e5d1f93 ("mm/migrate: update node demotion order on hotplug events"), the function to update node demotion order is called in CPUHP_AP_ONLINE_DYN during CPU online/offline. This doesn't satisfy the order requirement. For example, there are 4 CPUs (P0, P1, P2, P3) in 2 sockets (P0, P1 in S0 and P2, P3 in S1), the demotion order is - S0 -> NUMA_NO_NODE - S1 -> NUMA_NO_NODE After P2 and P3 is offlined, because S1 has no CPU now, the demotion order should have been changed to - S0 -> S1 - S1 -> NO_NODE but it isn't changed, because the order updating callback for CPU hotplug doesn't see the new nodemask. After that, if P1 is offlined, the demotion order is changed to the expected order as above. So in this patch, we added CPUHP_AP_MM_DEMOTION_ONLINE and CPUHP_MM_DEMOTION_DEAD to be called after CPUHP_AP_ONLINE_DYN and CPUHP_MM_VMSTAT_DEAD during CPU online and offline, and register the update function on them. Link: https://lkml.kernel.org/r/20210929060351.7293-1-ying.huang@intel.com Fixes: 884a6e5d1f93 ("mm/migrate: update node demotion order on hotplug events") Signed-off-by: "Huang, Ying" Cc: Thomas Gleixner Cc: Dave Hansen Cc: Yang Shi Cc: Zi Yan Cc: Michal Hocko Cc: Wei Xu Cc: Oscar Salvador Cc: David Rientjes Cc: Dan Williams Cc: David Hildenbrand Cc: Greg Thelen Cc: Keith Busch Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

Documentation: core-api/cpuhotplug: Rewrite the API section

2021-09-10T22:41:21+00:00

Dave stumbled over the incomplete and confusing documentation of the CPU hotplug API. Rewrite it, add the missing function documentations and correct the existing ones. Reported-by: Dave Chinner Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/20210909123212.489059409@linutronix.de

Merge tag 'xfs-5.15-merge-6' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

2021-09-02T15:26:03+00:00

Pull xfs updates from Darrick Wong: "There's a lot in this cycle. Starting with bug fixes: To avoid livelocks between the logging code and the quota code, we've disabled the ability of quotaoff to turn off quota accounting. (Admins can still disable quota enforcement, but truly turning off accounting requires a remount.) We've tried to do this in a careful enough way that there shouldn't be any user visible effects aside from quotaoff no longer randomly hanging the system. We've also fixed some bugs in runtime log behavior that could trip up log recovery if (otherwise unrelated) transactions manage to start and commit concurrently; some bugs in the GETFSMAP ioctl where we would incorrectly restrict the range of records output if the two xfs devices are of different sizes; a bug that resulted in fallocate funshare failing unnecessarily; and broken behavior in the xfs inode cache when DONTCACHE is in play. As for new features: we now batch inode inactivations in percpu background threads, which sharply decreases frontend thread wait time when performing file deletions and should improve overall directory tree deletion times. This eliminates both the problem where closing an unlinked file (especially on a frozen fs) can stall for a long time, and should also ease complaints about direct reclaim bogging down on unlinked file cleanup. Starting with this release, we've enabled pipelining of the XFS log. On workloads with high rates of metadata updates to different shards of the filesystem, multiple threads can be used to format committed log updates into log checkpoints. Lastly, with this release, two new features have graduated to supported status: inode btree counters (for faster mounts), and support for dates beyond Y2038. Expect these to be enabled by default in a future release of xfsprogs. Summary: - Fix a potential log livelock on busy filesystems when there's so much work going on that we can't finish a quotaoff before filling up the log by removing the ability to disable quota accounting. - Introduce the ability to use per-CPU data structures in XFS so that we can do a better job of maintaining CPU locality for certain operations. - Defer inode inactivation work to per-CPU lists, which will help us batch that processing. Deletions of large sparse files will *appear* to run faster, but all that means is that we've moved the work to the backend. - Drop the EXPERIMENTAL warnings from the y2038+ support and the inode btree counters, since it's been nearly a year and no complaints have come in. - Remove more of our bespoke kmem* variants in favor of using the standard Linux calls. - Prepare for the addition of log incompat features in upcoming cycles by actually adding code to support this. - Small cleanups of the xattr code in preparation for landing support for full logging of extended attribute updates in a future cycle. - Replace the various log shutdown state and flag code all over xfs with a single atomic bit flag. - Fix a serious log recovery bug where log item replay can be skipped based on the start lsn of a transaction even though the transaction commit lsn is the key data point for that by enforcing start lsns to appear in the log in the same order as commit lsns. - Enable pipelining in the code that pushes log items to disk. - Drop ->writepage. - Fix some bugs in GETFSMAP where the last fsmap record reported for a device could extend beyond the end of the device, and a separate bug where query keys for one device could be applied to another. - Don't let GETFSMAP query functions edit their input parameters. - Small cleanups to the scrub code's handling of perag structures. - Small cleanups to the incore inode tree walk code. - Constify btree function parameters that aren't changed, so that there will never again be confusion about range query functions changing their input parameters. - Standardize the format and names of tracepoint data attributes. - Clean up all the mount state and feature flags to use wrapped bitset functions instead of inconsistently open-coded flag checks. - Fix some confusion between xfs_buf hash table key variable vs. block number. - Fix a mis-interaction with iomap where we reported shared delalloc cow fork extents to iomap, which would cause the iomap unshare operation to return IO errors unnecessarily. - Fix DONTCACHE behavior" * tag 'xfs-5.15-merge-6' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (103 commits) xfs: fix I_DONTCACHE xfs: only set IOMAP_F_SHARED when providing a srcmap to a write xfs: fix perag structure refcounting error when scrub fails xfs: rename buffer cache index variable b_bn xfs: convert bp->b_bn references to xfs_buf_daddr() xfs: introduce xfs_buf_daddr() xfs: kill xfs_sb_version_has_v3inode() xfs: introduce xfs_sb_is_v5 helper xfs: remove unused xfs_sb_version_has wrappers xfs: convert xfs_sb_version_has checks to use mount features xfs: convert scrub to use mount-based feature checks xfs: open code sb verifier feature checks xfs: convert xfs_fs_geometry to use mount feature checks xfs: replace XFS_FORCED_SHUTDOWN with xfs_is_shutdown xfs: convert remaining mount flags to state flags xfs: convert mount flags to features xfs: consolidate mount option features in m_features xfs: replace xfs_sb_version checks with feature flag checks xfs: reflect sb features in xfs_mount xfs: rework attr2 feature and mount options ...

Merge tag 'io_uring-bio-cache.5-2021-08-30' of git://git.kernel.dk/linux-block

2021-08-31T02:30:30+00:00

Pull support for struct bio recycling from Jens Axboe: "This adds bio recycling support for polled IO, allowing quick reuse of a bio for high IOPS scenarios via a percpu bio_set list. It's good for almost a 10% improvement in performance, bumping our per-core IO limit from ~3.2M IOPS to ~3.5M IOPS" * tag 'io_uring-bio-cache.5-2021-08-30' of git://git.kernel.dk/linux-block: bio: improve kerneldoc documentation for bio_alloc_kiocb() block: provide bio_clear_hipri() helper block: use the percpu bio cache in __blkdev_direct_IO io_uring: enable use of bio alloc cache block: clear BIO_PERCPU_CACHE flag if polling isn't supported bio: add allocation cache abstraction fs: add kiocb alloc cache flag bio: optimize initialization of a bio

bio: add allocation cache abstraction

2021-08-23T19:44:43+00:00

Add a per-cpu bio_set cache for bio allocations, enabling us to quickly recycle them instead of going through the slab allocator. This cache isn't IRQ safe, and hence is only really suitable for polled IO. Very simple - keeps a count of bio's in the cache, and maintains a max of 512 with a slack of 64. If we get above max + slack, we drop slack number of bio's. Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe

cpu/hotplug: Fix comment typo

2021-08-10T16:02:37+00:00

/s/reatdown/teardown/ Signed-off-by: John Garry Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/1621585689-177398-1-git-send-email-john.garry@huawei.com