diff options
| author | Jens Axboe <axboe@kernel.dk> | 2020-10-16 01:24:45 +0300 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2020-10-17 18:25:47 +0300 |
| commit | d8a6df10aac9f2e4d5f30aff3129d552d2984ce7 (patch) | |
| tree | c254a63bd2029e29853e1f28568bfc06e4e41b2d /include/linux/io_uring.h | |
| parent | 500a373d731ac506612db12631ec21295c1ff360 (diff) | |
| download | linux-d8a6df10aac9f2e4d5f30aff3129d552d2984ce7.tar.xz | |
io_uring: use percpu counters to track inflight requests
Even though we place the req_issued and req_complete in separate
cachelines, there's considerable overhead in doing the atomics
particularly on the completion side.
Get rid of having the two counters, and just use a percpu_counter for
this. That's what it was made for, after all. This considerably
reduces the overhead in __io_free_req().
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'include/linux/io_uring.h')
| -rw-r--r-- | include/linux/io_uring.h | 7 |
1 files changed, 2 insertions, 5 deletions
diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 607d14f61132..28939820b6b0 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -23,13 +23,10 @@ struct io_uring_task { struct xarray xa; struct wait_queue_head wait; struct file *last; - atomic_long_t req_issue; + struct percpu_counter inflight; struct io_identity __identity; struct io_identity *identity; - - /* completion side */ - bool in_idle ____cacheline_aligned_in_smp; - atomic_long_t req_complete; + bool in_idle; }; #if defined(CONFIG_IO_URING) |