kernel/linux.git/io_uring/tctx.h, branch v6.19.11Linux kernel stable tree (mirror)https://git.radix-linux.su/kernel/linux.git/atom?h=v6.19.112022-10-07T18:25:30+00:00io_uring: simplify __io_uring_add_tctx_node2022-10-07T18:25:30+00:00Dylan Yudakendylany@fb.com2022-09-26T17:09:26+00:00urn:sha1:97c96e9fa36616d7890a6f3438172fc501927f01
Remove submitter parameter from __io_uring_add_tctx_node.
It was only called from one place, and we can do that logic in that one
place.
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Fixes: 97bbdc06a444 ("io_uring: add IORING_SETUP_SINGLE_ISSUER")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: export io_put_task()2022-07-25T00:41:06+00:00Pavel Begunkovasml.silence@gmail.com2022-07-12T20:52:37+00:00urn:sha1:e70cb60893ca64b7df06864aa16c1cf6d6c671db
Make io_put_task() available to non-core parts of io_uring, we'll need
it for notification infrastructure.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/3686807d4c03b72e389947b0e8692d4d44334ef0.1657643355.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: lockless task list2022-07-25T00:39:15+00:00Dylan Yudakendylany@fb.com2022-06-22T13:40:23+00:00urn:sha1:f88262e60bb9cb5740891672ce9f405e7f9393e5
With networking use cases we see contention on the spinlock used to
protect the task_list when multiple threads try and add completions at once.
Instead we can use a lockless list, and assume that the first caller to
add to the list is responsible for kicking off task work.
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/20220622134028.2013417-4-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: remove priority tw list optimisation2022-07-25T00:39:15+00:00Dylan Yudakendylany@fb.com2022-06-22T13:40:21+00:00urn:sha1:ed5ccb3beeba0cadb0fcf353ae192021dfecf252
This optimisation has some built in assumptions that make it easy to
introduce bugs. It also does not have clear wins that make it worth keeping.
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/20220622134028.2013417-2-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: optimize io_uring_task layout2022-07-25T00:39:15+00:00Pavel Begunkovasml.silence@gmail.com2022-06-20T14:27:35+00:00urn:sha1:4a0fef62788b69df09267c8e3f3f11d4bb9d50e7
task_work bits of io_uring_task are split into two cache lines causing
extra cache bouncing, place them into a separate cache line. Also move
the most used submission path fields closer together, so there are hot.
Cc: stable@vger.kernel.org # 5.15+
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: add IORING_SETUP_SINGLE_ISSUER2022-07-25T00:39:13+00:00Pavel Begunkovasml.silence@gmail.com2022-06-16T09:22:08+00:00urn:sha1:97bbdc06a4446bc69d8ba71d722abae542a6b70c
Add a new IORING_SETUP_SINGLE_ISSUER flag and the userspace visible part
of it, i.e. put limitations of submitters. Also, don't allow it together
with IOPOLL as we're not going to put it to good use.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/4bcc41ee467fdf04c8aab8baf6ce3ba21858c3d4.1655371007.git.asml.silence@gmail.com
Reviewed-by: Hao Xu <howeyxu@tencent.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: inline ->registered_rings2022-07-25T00:39:13+00:00Pavel Begunkovasml.silence@gmail.com2022-06-15T16:33:53+00:00urn:sha1:6a02e4be8187588ac476136496af9e3feeeb9a75
There can be only 16 registered rings, no need to allocate an array for
them separately but store it in tctx.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/495f0b953c87994dd9e13de2134019054fa5830d.1655310733.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring: move io_uring_task (tctx) helpers into its own file2022-07-25T00:39:12+00:00Jens Axboeaxboe@kernel.dk2022-05-25T17:01:04+00:00urn:sha1:c9f06aa7de153cdbe424e7e38be39f2272cf78bc
Signed-off-by: Jens Axboe <axboe@kernel.dk>