kernel/linux.git/io_uring/rw.c, branch v7.3-rc1

Merge tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2026-08-20T20:47:56+00:00

Pull io_uring update from Jens Axboe: "On top of the usual cleanups and fixes, the bigger items in here are: - zcrx work, most of it centered around adding dynamic area provisioning, plus a bunch of prep and cleanups leading up to it: scale refilling with large pages, coalesce same-niov RQEs on refill, separate the RQ head/tail cache lines and cache the RQ tail, and rework the area creation locking. - Fix the futex inflight accounting so that only private futex waits are marked inflight, and don't mark wake requests as inflight at all. - Drop the custom iov copy in the buffer select prep and msg header copy paths, using the generic helpers instead. - Fix a folio size overflow in io_vec_fill_bvec(), and account the pages a compound region really uses in the memmap path. - Fix an iovec leak in uring_cmd when the async cmd isn't recycled, skip the blocking task work for io_uring_cmd_issue_blocking(), and don't skip completion for a synchronous multishot cmd - Defer eventfd signaling when queued from a wakeup handler - Fix io-wq worker accounting when canceling creation callbacks - Annotate remote tasks for kcoverage" * tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (30 commits) io_uring: Add missing include for ITER_SOURCE and ITER_DEST io_uring/uring_cmd: don't skip completion for a synchronous multishot cmd io_uring/memmap: account the pages a compound region really uses io_uring/zcrx: add dynamic area provisioning io_uring/zcrx: lock area creation with pp_lock io_uring/zcrx: keep array of areas io_uring/zcrx: move freelist lock to struct zcrx io_uring/zcrx: unmap under netdev lock io_uring/zcrx: split dmabuf unmap and release io_uring/zcrx: don't pass ifq_reg to area creation io_uring/zcrx: add helper for deriving area token io_uring/zcrx: don't reload skb_shinfo io_urint/zcrx: narrow var scope in io_zcrx_recv_skb() io_uring/zcrx: constify area_reg on import io_uring/zcrx: coalesce same-niov RQEs on refill io_uring/zcrx: cache RQ tail io_uring/zcrx: add RQ iterator io_uring/zcrx: move RQ head/tail to separate cache lines io_uring/zcrx: scale refilling with large pages io_uring/io-wq: fix worker accounting when canceling creation callbacks ...

io_uring/rw: Drop custom iov copy in io_iov_buffer_select_prep

2026-08-15T23:25:33+00:00

Similar to commit f4eaf8eda89e ("io_uring/rsrc: Drop io_copy_iov in favor of iovec API"), avoid custom copy and just rely on the iovec api. Signed-off-by: Gabriel Krisman Bertazi Link: https://patch.msgid.link/20260813004022.3514537-2-krisman@suse.de Signed-off-by: Jens Axboe

io_uring/rw: fix missing ERESTARTSYS conversion in read paths

2026-07-22T13:36:25+00:00

Both read and write may receive internal restart error codes from the filesystem layer and should be converted to -EINTR. However, when multishot read support was added, the error code normalization was lost for both io_read() and io_read_mshot(). Extract the conversion into io_fixup_restart_res() and apply it in all three locations: io_rw_done(), io_read(), and io_read_mshot(). Fixes: a08d195b586a ("io_uring/rw: split io_read() into a helper") Cc: stable@vger.kernel.org Signed-off-by: Yitang Yang Link: https://patch.msgid.link/20260722124551.130563-1-yi1tang.yang@gmail.com Signed-off-by: Jens Axboe

io_uring/rw: preserve partial result for iopoll

2026-06-16T12:20:01+00:00

A partial read will store the completed byte count in io->bytes_done. The regular completion path applies io_fixup_rw_res() so that, when the following operation reaches EOF, the number of bytes already read is returned. The iopoll completion path does not apply this fixup to the return value and can return zero instead. Use the fixup result when updating the CQE, and the raw result for the reissue check. Cc: stable@vger.kernel.org Fixes: 4d9cb92ca41d ("io_uring/rw: fix short rw error handling") Signed-off-by: Michael Wigham Link: https://patch.msgid.link/20260613225240.34032-1-michael@wigham.net Signed-off-by: Jens Axboe

io_uring/rw: drop unused attr_type_mask from io_prep_rw_pi()

2026-05-13T12:35:57+00:00

io_prep_rw_pi() never used the attr_type_mask argument. Callers already validate sqe->attr_type_mask before invoking the helper (only IORING_RW_ATTR_FLAG_PI is supported today). Remove the dead parameter to avoid implying further interpretation happens here. Signed-off-by: Yang Xiuwei Link: https://patch.msgid.link/20260513094303.866533-1-yangxiuwei@kylinos.cn Signed-off-by: Jens Axboe

io_uring/rw: add defensive hardening for negative kbuf lengths

2026-04-21T18:19:03+00:00

No real bug here, just being a bit defensive in ensuring that whatever gets passed into io_put_kbuf() is always >= 0 and not some random error value. Reviewed-by: Gabriel Krisman Bertazi Signed-off-by: Jens Axboe

io_uring/rw: clean up __io_read() obsolete comment and early returns

2026-04-02T12:55:50+00:00

After commit a9165b83c193 ("io_uring/rw: always setup io_async_rw for read/write requests") which moved the iovec allocation into the prep path and stores it in req->async_data where it now gets freed as part of the request lifecycle, this comment is now outdated. Remove it and clean up the goto as well. Signed-off-by: Joanne Koong Link: https://patch.msgid.link/20260401173511.4052303-1-joannelkoong@gmail.com Signed-off-by: Jens Axboe

io_uring/rw: use cached file rather than req->file

2026-03-17T20:35:00+00:00

In io_rw_init_file(), req->file is cached in file, yet the former is still being used when checking for O_DIRECT. As this is post setting the kiocb flags, the compiler has to reload req->file. Just use the locally cached file instead. Signed-off-by: Jens Axboe

io_uring: add REQ_F_IOPOLL

2026-03-16T22:14:14+00:00

A subsequent commit will allow uring_cmds to files that don't implement ->uring_cmd_iopoll() to be issued to IORING_SETUP_IOPOLL io_urings. This means the ctx's IORING_SETUP_IOPOLL flag isn't sufficient to determine whether a given request needs to be iopolled. Introduce a request flag REQ_F_IOPOLL set in ->issue() if a request needs to be iopolled to completion. Set the flag in io_rw_init_file() and io_uring_cmd() for requests issued to IORING_SETUP_IOPOLL ctxs. Use the request flag instead of IORING_SETUP_IOPOLL in places dealing with a specific request. A future possibility would be to add an option to enable/disable iopoll in the io_uring SQE instead of determining it from IORING_SETUP_IOPOLL. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta Link: https://patch.msgid.link/20260302172914.2488599-2-csander@purestorage.com Signed-off-by: Jens Axboe

io_uring: add IORING_OP_URING_CMD128 to opcode checks

2026-02-19T14:25:39+00:00

io_should_commit(), io_uring_classic_poll(), and io_do_iopoll() compare struct io_kiocb's opcode against IORING_OP_URING_CMD to implement special treatment for uring_cmds. The recently added opcode IORING_OP_URING_CMD128 is meant to be equivalent to IORING_OP_URING_CMD, so treat it the same way in these functions. Fixes: 1cba30bf9fdd ("io_uring: add support for IORING_SETUP_SQE_MIXED") Signed-off-by: Caleb Sander Mateos Reviewed-by: Anuj Gupta Reviewed-by: Kanchan Joshi Signed-off-by: Jens Axboe