kernel/linux.git/include/linux/workqueue.h, branch v7.3-rc1

NFS/localio: issue IO inline when not in a memory-reclaim context

2026-08-17T16:02:07+00:00

Every LOCALIO read and write is currently bounced through the dedicated !WQ_MEM_RECLAIM nfslocaliod_workqueue. That bounce is only actually required when the submitting context is a memory-reclaim context: LOCALIO issues IO directly into a stacked local filesystem (e.g. XFS) which may in turn flush its own !WQ_MEM_RECLAIM workqueue. Doing that from a WQ_MEM_RECLAIM worker (most importantly writeback's wb_workfn on bdi_wq) or an explicit PF_MEMALLOC reclaim task trips check_flush_dependency() and risks a forward-progress deadlock, which is why commit b9f5dd57f4a5 ("nfs/localio: use dedicated workqueues for filesystem read and write") introduced the intermediate workqueue. Outside of reclaim context -- ordinary application/task submission such as O_DIRECT or fsync-driven writeback -- the workqueue hop buys nothing and merely adds a context switch and scheduling latency per IO while discarding the NFS client's inherent application-context parallelism. Add current_is_workqueue_mem_reclaim(), which reports whether %current is a WQ_MEM_RECLAIM worker using the same predicate check_flush_dependency() warns on. Use it, together with the PF_MEMALLOC check, in the new nfs_local_defer_io() helper to decide per-IO whether nfs_local_do_read() and nfs_local_do_write() must defer to nfslocaliod_workqueue or may issue the IO inline. Buffered writeback continues to bounce (wb_workfn is a WQ_MEM_RECLAIM worker); O_DIRECT and app-context submission now run inline. Running nfs_local_call_write() inline is safe: it already saves and restores current->flags around the PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO it sets and scopes the file opener's creds. The async O_DIRECT completion path is likewise unaffected: when the underlying filesystem returns -EIOCBQUEUED, the kiocb ki_complete callback (nfs_local_read_aio_complete / nfs_local_write_aio_complete) can run in bottom-half context and so must still defer the pgio completion (nfs_local_pgio_release -> rpc_call_done) to nfsiod_workqueue via nfs_local_pgio_aio_complete(). That completion hop is independent of how the IO was submitted, and this change leaves it as-is; only the submission side stops unconditionally hopping through nfslocaliod_workqueue. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Mike Snitzer Signed-off-by: Trond Myklebust

workqueue: Add warnings and fallback if system_{unbound}_wq is used

2026-05-29T17:55:33+00:00

Currently many users transitioned already to the new introduced workqueue (system_percpu_wq, system_dfl_wq), but there are new users who still use the older system_wq and system_unbound_wq. This change try to push this transition forward, by warning whether the old workqueues are used. Link: https://lore.kernel.org/all/20250221112003.1dSuoGyc@linutronix.de/ Suggested-by: Tejun Heo Signed-off-by: Marco Crivellari Signed-off-by: Tejun Heo

workqueue: fix devm_alloc_workqueue() va_list misuse

2026-04-28T16:13:40+00:00

devm_alloc_workqueue() built a va_list and passed it as a single positional argument to the variadic alloc_workqueue() macro: va_start(args, max_active); wq = alloc_workqueue(fmt, flags, max_active, args); va_end(args); C does not allow forwarding a va_list through a ... parameter. alloc_workqueue() expands to alloc_workqueue_noprof(), which runs its own va_start() over its ... params, so the inner vsnprintf(wq->name, sizeof(wq->name), fmt, args) in __alloc_workqueue() received the outer va_list object as the first variadic slot rather than the caller's actual format arguments. Add a new static helper alloc_workqueue_va() that wraps __alloc_workqueue() and runs wq_init_lockdep() on success, and fold both alloc_workqueue_noprof() and devm_alloc_workqueue_noprof() onto it as suggested by Tejun. The wq_init_lockdep() step is required on the devm path too, otherwise __flush_workqueue()'s on-stack COMPLETION_INITIALIZER_ONSTACK_MAP would NULL-deref wq->lockdep_map. No caller changes are required. devm_alloc_ordered_workqueue() is a macro forwarding to devm_alloc_workqueue() and inherits the fix. Two in-tree callers actively trigger the broken path on every probe: drivers/power/supply/mt6370-charger.c:889 drivers/power/supply/max77705_charger.c:649 both of which use devm_alloc_ordered_workqueue(dev, "%s", 0, dev_name(dev)). A standalone reproducer module is available at[1]. Link: https://github.com/leitao/debug/blob/main/workqueue/valist/wq_va_test.c [1] Fixes: 1dfc9d60a69e ("workqueue: devres: Add device-managed allocate workqueue") Signed-off-by: Breno Leitao Signed-off-by: Tejun Heo

workqueue: add WQ_AFFN_CACHE_SHARD affinity scope

2026-04-01T20:24:18+00:00

On systems where many CPUs share one LLC, unbound workqueues using WQ_AFFN_CACHE collapse to a single worker pool, causing heavy spinlock contention on pool->lock. For example, Chuck Lever measured 39% of cycles lost to native_queued_spin_lock_slowpath on a 12-core shared-L3 NFS-over-RDMA system. The existing affinity hierarchy (cpu, smt, cache, numa, system) offers no intermediate option between per-LLC and per-SMT-core granularity. Add WQ_AFFN_CACHE_SHARD, which subdivides each LLC into groups of at most wq_cache_shard_size cores (default 8, tunable via boot parameter). Shards are always split on core (SMT group) boundaries so that Hyper-Threading siblings are never placed in different pods. Cores are distributed across shards as evenly as possible -- for example, 36 cores in a single LLC with max shard size 8 produces 5 shards of 8+7+7+7+7 cores. The implementation follows the same comparator pattern as other affinity scopes: precompute_cache_shard_ids() pre-fills the cpu_shard_id[] array from the already-initialized WQ_AFFN_CACHE and WQ_AFFN_SMT topology, and cpus_share_cache_shard() is passed to init_pod_type(). Benchmark on NVIDIA Grace (72 CPUs, single LLC, 50k items/thread), show cache_shard delivers ~5x the throughput and ~6.5x lower p50 latency compared to cache scope on this 72-core single-LLC system. Suggested-by: Tejun Heo Signed-off-by: Breno Leitao Signed-off-by: Tejun Heo

workqueue: fix typo in WQ_AFFN_SMT comment

2026-04-01T20:24:18+00:00

Fix "poer" -> "per" in the WQ_AFFN_SMT enum comment. Signed-off-by: Breno Leitao Signed-off-by: Tejun Heo

Merge branch 'for-7.1-devm-alloc-wq' into for-7.1

2026-03-10T17:03:57+00:00

workqueue: devres: Add device-managed allocate workqueue

2026-03-10T17:03:39+00:00

Add a Resource-managed version of alloc_workqueue() to fix common problem of drivers mixing devm() calls with destroy_workqueue. Such naive and discouraged driver approach leads to difficult to debug bugs when the driver: 1. Allocates workqueue in standard way and destroys it in driver remove() callback, 2. Sets work struct with devm_work_autocancel(), 3. Registers interrupt handler with devm_request_threaded_irq(). Which leads to following unbind/removal path: 1. destroy_workqueue() via driver remove(), Any interrupt coming now would still execute the interrupt handler, which queues work on destroyed workqueue. 2. devm_irq_release(), 3. devm_work_drop() -> cancel_work_sync() on destroyed workqueue. devm_alloc_workqueue() has two benefits: 1. Solves above problem of mix-and-match devres and non-devres code in driver, 2. Simplify any sane drivers which were correctly using alloc_workqueue() + devm_add_action_or_reset(). Signed-off-by: Krzysztof Kozlowski Acked-by: Tejun Heo Reviewed-by: Andy Shevchenko Signed-off-by: Tejun Heo

workqueue: Add system_dfl_long_wq for long unbound works

2026-03-09T16:45:08+00:00

Currently there are users of queue_delayed_work() who specify system_long_wq, the per-cpu workqueue. This workqueue should be used for long per-cpu works, but queue_delayed_work() queue the work using: queue_delayed_work_on(WORK_CPU_UNBOUND, ...); This would end up calling __queue_delayed_work() that does: if (housekeeping_enabled(HK_TYPE_TIMER)) { // [....] } else { if (likely(cpu == WORK_CPU_UNBOUND)) add_timer_global(timer); else add_timer_on(timer, cpu); } So when cpu == WORK_CPU_UNBOUND the timer is global and is not using a specific CPU. Later, when __queue_work() is called: if (req_cpu == WORK_CPU_UNBOUND) { if (wq->flags & WQ_UNBOUND) cpu = wq_select_unbound_cpu(raw_smp_processor_id()); else cpu = raw_smp_processor_id(); } Because the wq is not unbound, it takes the CPU where the timer fired and enqueue the work on that CPU. The consequence of all of this is that the work can run anywhere, depending on where the timer fired. Introduce system_dfl_long_wq in order to change, in a future step, users that are still calling: queue_delayed_work(system_long_wq, ...); with the new system_dfl_long_wq instead, so that the work may benefit from scheduler task placement. Signed-off-by: Marco Crivellari Signed-off-by: Tejun Heo

workqueue: Update documentation as per system_percpu_wq naming

2026-02-27T17:40:44+00:00

Update documentation to use "per-CPU workqueue" instead of "global workqueue" to match the system_wq to system_percpu_wq rename. The workqueue behavior remains unchanged; this just aligns terminology with the clearer naming. Fixes: a2be943b46b4 ("workqueue: replace use of system_wq with system_percpu_wq") Signed-off-by: Mallesh Koujalagi Signed-off-by: Tejun Heo

cpuset: Propagate cpuset isolation update to workqueue through housekeeping

2026-02-03T14:23:34+00:00

Until now, cpuset would propagate isolated partition changes to workqueues so that unbound workers get properly reaffined. Since housekeeping now centralizes, synchronize and propagates isolation cpumask changes, perform the work from that subsystem for consolidation and consistency purposes. For simplification purpose, the target function is adapted to take the new housekeeping mask instead of the isolated mask. Suggested-by: Tejun Heo Signed-off-by: Frederic Weisbecker Reviewed-by: Waiman Long Acked-by: Tejun Heo Cc: "Michal Koutný" Cc: Ingo Molnar Cc: Johannes Weiner Cc: Lai Jiangshan Cc: Marco Crivellari Cc: Michal Hocko Cc: Peter Zijlstra Cc: Tejun Heo Cc: Thomas Gleixner Cc: Vlastimil Babka Cc: Waiman Long Cc: cgroups@vger.kernel.org