kernel/linux.git/include/uapi/linux/io_uring.h, branch v6.5.13

nvme: improved uring polling

2023-06-28T22:09:41+00:00

Drivers can poll requests directly, so use that. We just need to ensure the driver's request was allocated from a polled hctx, so a special driver flag is added to struct io_uring_cmd. The allows unshared and multipath namespaces to use the same polling callback, and multipath is guaranteed to get the same queue as the command was submitted on. Previously multipath polling might check a different path and poll the wrong info. The other bonus is we don't need a bio payload in order to poll, allowing commands like 'flush' and 'write zeroes' to be submitted on the same high priority queue as read and write commands. Finally, using the request based polling skips the unnecessary bio overhead. Signed-off-by: Keith Busch Reviewed-by: Sagi Grimberg Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20230612190343.2087040-3-kbusch@meta.com Signed-off-by: Jens Axboe

io_uring: Add io_uring_setup flag to pre-register ring fd and never install it

2023-05-16T14:06:00+00:00

With IORING_REGISTER_USE_REGISTERED_RING, an application can register the ring fd and use it via registered index rather than installed fd. This allows using a registered ring for everything *except* the initial mmap. With IORING_SETUP_NO_MMAP, io_uring_setup uses buffers allocated by the user, rather than requiring a subsequent mmap. The combination of the two allows a user to operate *entirely* via a registered ring fd, making it unnecessary to ever install the fd in the first place. So, add a flag IORING_SETUP_REGISTERED_FD_ONLY to make io_uring_setup register the fd and return a registered index, without installing the fd. This allows an application to avoid touching the fd table at all, and allows a library to never even momentarily install a file descriptor. This splits out an io_ring_add_registered_file helper from io_ring_add_registered_fd, for use by io_uring_setup. Signed-off-by: Josh Triplett Link: https://lore.kernel.org/r/bc8f431bada371c183b95a83399628b605e978a3.1682699803.git.josh@joshtriplett.org Signed-off-by: Jens Axboe

io_uring: support for user allocated memory for rings/sqes

2023-05-16T14:04:55+00:00

Currently io_uring applications must call mmap(2) twice to map the rings themselves, and the sqes array. This works fine, but it does not support using huge pages to back the rings/sqes. Provide a way for the application to pass in pre-allocated memory for the rings/sqes, which can then suitably be allocated from shmfs or via mmap to get huge page support. Particularly for larger rings, this reduces the TLBs needed. If an application wishes to take advantage of that, it must pre-allocate the memory needed for the sq/cq ring, and the sqes. The former must be passed in via the io_uring_params->cq_off.user_data field, while the latter is passed in via the io_uring_params->sq_off.user_data field. Then it must set IORING_SETUP_NO_MMAP in the io_uring_params->flags field, and io_uring will then map the existing memory into the kernel for shared use. The application must not call mmap(2) to map rings as it otherwise would have, that will now fail with -EINVAL if this setup flag was used. The pages used for the rings and sqes must be contigious. The intent here is clearly that huge pages should be used, otherwise the normal setup procedure works fine as-is. The application may use one huge page for both the rings and sqes. Outside of those initialization changes, everything works like it did before. Signed-off-by: Jens Axboe

io_uring: add support for multishot timeouts

2023-04-19T01:38:36+00:00

A multishot timeout submission will repeatedly generate completions with the IORING_CQE_F_MORE cflag set. Depending on the value of the `off' field in the submission, these timeouts can either repeat indefinitely until cancelled (`off' = 0) or for a fixed number of times (`off' > 0). Only noseq timeouts (i.e. not dependent on the number of I/O completions) are supported. An indefinite timer will be cancelled if the CQ ever overflows. Signed-off-by: David Wei Link: https://lore.kernel.org/r/20230418225817.1905027-1-davidhwei@meta.com Signed-off-by: Jens Axboe

io_uring: kill unused notif declarations

2023-04-03T13:16:15+00:00

There are two leftover structures from the notification registration mechanism that has never been released, kill them. Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/f05f65aebaf8b1b5bf28519a8fdb350e3e7c9ad0.1679924536.git.asml.silence@gmail.com Signed-off-by: Jens Axboe

io_uring: add support for user mapped provided buffer ring

2023-04-03T13:14:21+00:00

The ring mapped provided buffer rings rely on the application allocating the memory for the ring, and then the kernel will map it. This generally works fine, but runs into issues on some architectures where we need to be able to ensure that the kernel and application virtual address for the ring play nicely together. This at least impacts architectures that set SHM_COLOUR, but potentially also anyone setting SHMLBA. To use this variant of ring provided buffers, the application need not allocate any memory for the ring. Instead the kernel will do so, and the allocation must subsequently call mmap(2) on the ring with the offset set to: IORING_OFF_PBUF_RING | (bgid << IORING_OFF_PBUF_SHIFT) to get a virtual address for the buffer ring. Normally the application would allocate a suitable piece of memory (and correctly aligned) and simply pass that in via io_uring_buf_reg.ring_addr and the kernel would map it. Outside of the setup differences, the kernel allocate + user mapped provided buffer ring works exactly the same. Acked-by: Helge Deller Signed-off-by: Jens Axboe

io_uring/kbuf: rename struct io_uring_buf_reg 'pad' to'flags'

2023-04-03T13:14:21+00:00

In preparation for allowing flags to be set for registration, rename the padding and use it for that. Acked-by: Helge Deller Signed-off-by: Jens Axboe

Merge tag 'hardening-v6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux

2023-02-21T19:07:23+00:00

Pull hardening updates from Kees Cook: "Beyond some specific LoadPin, UBSAN, and fortify features, there are other fixes scattered around in various subsystems where maintainers were okay with me carrying them in my tree or were non-responsive but the patches were reviewed by others: - Replace 0-length and 1-element arrays with flexible arrays in various subsystems (Paulo Miguel Almeida, Stephen Rothwell, Kees Cook) - randstruct: Disable Clang 15 support (Eric Biggers) - GCC plugins: Drop -std=gnu++11 flag (Sam James) - strpbrk(): Refactor to use strchr() (Andy Shevchenko) - LoadPin LSM: Allow root filesystem switching when non-enforcing - fortify: Use dynamic object size hints when available - ext4: Fix CFI function prototype mismatch - Nouveau: Fix DP buffer size arguments - hisilicon: Wipe entire crypto DMA pool on error - coda: Fully allocate sig_inputArgs - UBSAN: Improve arm64 trap code reporting - copy_struct_from_user(): Add minimum bounds check on kernel buffer size" * tag 'hardening-v6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: randstruct: disable Clang 15 support uaccess: Add minimum bounds check on kernel buffer size arm64: Support Clang UBSAN trap codes for better reporting coda: Avoid partial allocation of sig_inputArgs gcc-plugins: drop -std=gnu++11 to fix GCC 13 build lib/string: Use strchr() in strpbrk() crypto: hisilicon: Wipe entire pool on error net/i40e: Replace 0-length array with flexible array io_uring: Replace 0-length array with flexible array ext4: Fix function prototype mismatch for ext4_feat_ktype i915/gvt: Replace one-element array with flexible-array member drm/nouveau/disp: Fix nvif_outp_acquire_dp() argument size LoadPin: Allow filesystem switch when not enforcing LoadPin: Move pin reporting cleanly out of locking LoadPin: Refactor sysctl initialization LoadPin: Refactor read-only check into a helper ARM: ixp4xx: Replace 0-length arrays with flexible arrays fortify: Use __builtin_dynamic_object_size() when available rxrpc: replace zero-lenth array with DECLARE_FLEX_ARRAY() helper

io_uring: Support calling io_uring_register with a registered ring fd

2023-02-16T13:09:30+00:00

Add a new flag IORING_REGISTER_USE_REGISTERED_RING (set via the high bit of the opcode) to treat the fd as a registered index rather than a file descriptor. This makes it possible for a library to open an io_uring, register the ring fd, close the ring fd, and subsequently use the ring entirely via registered index. Signed-off-by: Josh Triplett Link: https://lore.kernel.org/r/f2396369e638284586b069dbddffb8c992afba95.1676419314.git.josh@joshtriplett.org [axboe: remove extra high bit clear] Signed-off-by: Jens Axboe

io_uring/msg_ring: Pass custom flags to the cqe

2023-01-29T22:17:40+00:00

This patch adds a new flag (IORING_MSG_RING_FLAGS_PASS) in the message ring operations (IORING_OP_MSG_RING). This new flag enables the sender to specify custom flags, which will be copied over to cqe->flags in the receiving ring. These custom flags should be specified using the sqe->file_index field. This mechanism provides additional flexibility when sending messages between rings. Signed-off-by: Breno Leitao Link: https://lore.kernel.org/r/20230103160507.617416-1-leitao@debian.org Signed-off-by: Jens Axboe