kernel/linux.git/io_uring/futex.c, branch v7.1.12

io_uring: defer eventfd signaling when queued from a wakeup handler

2026-08-27T12:34:35+00:00

commit cd305ee3633a45fcf5f3a5d83f99f3cb77d87b6e upstream. io_req_local_work_add() signals the CQ ring eventfd inline when it is the one to push the first entry onto ->work_list. For DEFER_TASKRUN rings that add is frequently done from a waitqueue wakeup handler, where an arbitrary waitqueue lock is held. eventfd_signal_mask() only refuses to recurse when current->in_eventfd is set, but that bit is set by eventfd_signal_mask() itself. If the wake chain starts somewhere else, signal goes out inline and can feed back into epoll. Add IOU_F_TWQ_IN_WAKE, set it on the task_work add done from the three waitqueue callbacks, and use it to force io_eventfd_signal() down the existing call_rcu_hurry() deferral instead of signaling inline. Fixes: 21a091b970cd ("io_uring: signal registered eventfd to process deferred task work") Cc: stable@vger.kernel.org Link: https://lore.kernel.org/all/20260813133843.2933127-1-4ncienth@gmail.com/ Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring/futex: only mark private futex waits as inflight

2026-08-27T12:34:35+00:00

commit 4d327bbd1cd2485783553b6b2b401aa475f90dd0 upstream. Inflight tracking of futex wait requests exists to ensure that do_exit() -> io_uring_files_cancel() cancels them before the mm goes away, as a private futex wait depends on the mm private futex hash staying alive for the duration of the request. Shared futexes have no such dependency. A FLAGS_SHARED request always resolves to either an inode based key or an mm-shared key, both of which fail futex_key_is_private() and hence always hash into the global futex hash, whose lifetime isn't tied to the mm. Only mark vectored futex waits as inflight if the futex is private. Cc: stable@vger.kernel.org Fixes: 079afb081c42 ("io_uring/futex: mark wait requests as inflight") Link: https://lore.kernel.org/io-uring/CANGjgdn=R_qyUdE=j9za+vkmqcxacbP-84OHXF4nZ4ho9qRyVg@mail.gmail.com/ Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring/futex: don't mark futex wake requests as inflight

2026-08-27T12:34:35+00:00

commit 73e7019097473fc9f83a334ef2c6ab3343709fef upstream. Commit 079afb081c42 ("io_uring/futex: mark wait requests as inflight") added inflight tracking to ensure that do_exit() -> io_uring_files_cancel() finds and cancels pending futex waits before the mm goes away, as a private futex wait depends on the mm private futex hash staying alive for the duration of the request. However, as io_futex_prep() is shared between FUTEX_WAIT and FUTEX_WAKE, wake requests got marked as inflight as well. A futex wake executes fully inline at issue time and never depends on the mm staying alive after completion, hence there's no need to track it. Kill it. Cc: stable@vger.kernel.org Fixes: 079afb081c42 ("io_uring/futex: mark wait requests as inflight") Reported-by: Chengfeng Lin Link: https://lore.kernel.org/io-uring/CANGjgdn=R_qyUdE=j9za+vkmqcxacbP-84OHXF4nZ4ho9qRyVg@mail.gmail.com/ Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

io_uring/futex: ensure partial wakes are appropriately dequeued

2026-04-21T18:19:06+00:00

If a FUTEX_WAITV vectored operation is only partially woken, we should call __futex_wake_mark() on the queue to account for that. If not, then a later wakeup will wake the same entry, rather than the next one in line. Fixes: 8f350194d5cfd ("io_uring: add support for vectored futex waits") Reviewed-by: Gabriel Krisman Bertazi Signed-off-by: Jens Axboe

treewide: Replace kmalloc with kmalloc_obj for non-scalar types

2026-02-21T09:02:28+00:00

This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook

io_uring/futex: use GFP_KERNEL_ACCOUNT for futex data allocation

2026-01-25T17:07:09+00:00

Be a bit nicer and ensure it plays nice with memcg accounting. Signed-off-by: Jens Axboe

io_uring/futex: move futexv owned status to struct io_futexv_data

2025-11-05T19:55:07+00:00

Free up a bit of space in the shared futex opcode private data, by moving the futexv specific futexv_owned out of there and into the struct specific to vectored futexes. Signed-off-by: Jens Axboe

io_uring/futex: move futexv async data handling to struct io_futexv_data

2025-11-05T19:54:31+00:00

Rather than alloc an array of struct futex_vector for the futexv wait handling, wrap it in a struct io_futexv_data struct, similar to what the non-vectored futex wait handling does. No functional changes in this patch. Signed-off-by: Jens Axboe

io_uring: add wrapper type for io_req_tw_func_t arg

2025-11-03T15:31:26+00:00

In preparation for uring_cmd implementations to implement functions with the io_req_tw_func_t signature, introduce a wrapper struct io_tw_req to hide the struct io_kiocb * argument. The intention is for only the io_uring core to access the inner struct io_kiocb *. uring_cmd implementations should instead call a helper from io_uring/cmd.h to convert struct io_tw_req to struct io_uring_cmd *. Signed-off-by: Caleb Sander Mateos Signed-off-by: Jens Axboe

io_uring: add async data clear/free helpers

2025-08-27T17:24:25+00:00

Futex recently had an issue where it mishandled how ->async_data and REQ_F_ASYNC_DATA is handled. To avoid future issues like that, add a set of helpers that either clear or clear-and-free the async data assigned to a struct io_kiocb. Convert existing manual handling of that to use the helpers. No intended functional changes in this patch. Reviewed-by: Caleb Sander Mateos Signed-off-by: Jens Axboe