diff options
| author | Kashyap Desai <kashyap.desai@broadcom.com> | 2020-08-19 18:20:28 +0300 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2020-09-04 00:20:47 +0300 |
| commit | b445547ec1bbd3e7bf4b1c142550942f70527d95 (patch) | |
| tree | 452adb8f18969cd32966206e046632f1d7ff9b44 /include | |
| parent | f1b49fdc1c64db110aa1315831e5fe0f8599fa56 (diff) | |
| download | linux-b445547ec1bbd3e7bf4b1c142550942f70527d95.tar.xz | |
blk-mq, elevator: Count requests per hctx to improve performance
High CPU utilization on "native_queued_spin_lock_slowpath" due to lock
contention is possible for mq-deadline and bfq IO schedulers
when nr_hw_queues is more than one.
It is because kblockd work queue can submit IO from all online CPUs
(through blk_mq_run_hw_queues()) even though only one hctx has pending
commands.
The elevator callback .has_work for mq-deadline and bfq scheduler considers
pending work if there are any IOs on request queue but it does not account
hctx context.
Add a per-hctx 'elevator_queued' count to the hctx to avoid triggering
the elevator even though there are no requests queued.
[jpg: Relocated atomic_dec() in dd_dispatch_request(), update commit message per Kashyap]
Signed-off-by: Kashyap Desai <kashyap.desai@broadcom.com>
Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: John Garry <john.garry@huawei.com>
Tested-by: Douglas Gilbert <dgilbert@interlog.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'include')
| -rw-r--r-- | include/linux/blk-mq.h | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 8279c807e1f3..b23eeca4d677 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -139,6 +139,10 @@ struct blk_mq_hw_ctx { * shared across request queues. */ atomic_t nr_active; + /** + * @elevator_queued: Number of queued requests on hctx. + */ + atomic_t elevator_queued; /** @cpuhp_online: List to store request if CPU is going to die */ struct hlist_node cpuhp_online; |