kernel/linux.git/drivers/md/dm-delay.c, branch v6.12.80

dm-delay: remove timer_lock

2024-05-09T13:10:58+00:00

Instead of manually checking the timer details in queue_timeout(), call timer_reduce() to start the timer or reduce the expiration time. This avoids needing a lock. Signed-off-by: Benjamin Marzinski Signed-off-by: Mike Snitzer

dm-delay: change locking to avoid contention

2024-05-09T13:10:58+00:00

The delayed_bios list is protected by one mutex shared by all dm-delay devices. This mutex must be held whenever a bio is added or expired bios are removed from the list. Since a large number of expired bios could be on the list, flush_delayed_bios() can schedule while holding the mutex. This means a flush_delayed_bios() call on any dm-delay device can slow down delay_map() calls on any other dm-delay device. To keep dm-delay devices from slowing each other down and keep processing delay bios from slowing adding delayed bios, the global mutex has been removed, and each dm-delay device now has two locks. delayed_bios_lock is a spinlock that must be held whenever the delayed_bios list is accessed. process_bios_lock is a mutex that must be held whenever a process has temporarily pulled bios off the delayed_bios list to check which ones should be processed. It must be held until all the bios that won't be processed are returned to the list. This is what flush_delayed_bios() now does. The mutex is necessary to guarantee that delay_presuspend() sees the entire list of delayed bios when it calls flush_delayed_bios(). Signed-off-by: Benjamin Marzinski Signed-off-by: Mike Snitzer

dm-delay: fix max_delay calculations

2024-05-09T13:10:58+00:00

delay_ctr() pointlessly compared max_delay in cases where multiple delay classes were initialized identically. Also, when write delays were configured different than read delays, delay_ctr() never compared their value against max_delay. Fix these issues. Fixes: 70bbeb29fab0 ("dm delay: for short delays, use kthread instead of timers and wq") Signed-off-by: Benjamin Marzinski Signed-off-by: Mike Snitzer

dm-delay: fix hung task introduced by kthread mode

2024-05-09T13:10:58+00:00

If the worker thread is not woken due to a bio, then it is not woken at all. This causes the hung task check to trigger. This occurs, for instance, when no bios are submitted. Also when a delay of 0 is configured, delay_bio() returns without waking the worker. Prevent the hung task check from triggering by creating the thread with kthread_run() instead of using kthread_create() directly. Fixes: 70bbeb29fab0 ("dm delay: for short delays, use kthread instead of timers and wq") Signed-off-by: Joel Colledge Reviewed-by: Benjamin Marzinski Signed-off-by: Mike Snitzer

dm-delay: fix workqueue delay_timer race

2024-05-09T13:10:58+00:00

delay_timer could be pending when delay_dtr() is called. It needs to be shut down before kdelayd_wq is destroyed, so it won't try queueing more work to kdelayd_wq while that's getting destroyed. Also the del_timer_sync() call in delay_presuspend() doesn't protect against the timer getting immediately rearmed by the queued call to flush_delayed_bios(), but there's no real harm if that does happen. timer_delete() is less work, and is basically just as likely to stop a pointless call to flush_delayed_bios(). Fixes: 26b9f228703f ("dm: delay target") Signed-off-by: Benjamin Marzinski Signed-off-by: Mike Snitzer

dm-delay: avoid duplicate logic

2023-11-17T19:41:14+00:00

This is small refactoring of dm-delay - we avoid duplicate logic in flush_delayed_bios and flush_delayed_bios_fast and join these two functions into one. We also add cond_resched() to flush_delayed_bios because the list may have unbounded number of entries. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer

dm-delay: fix bugs introduced by kthread mode

2023-11-17T19:41:14+00:00

This commit fixes the following bugs introduced by commit 70bbeb29fab0 ("dm delay: for short delays, use kthread instead of timers and wq"): * the function flush_worker_fn has no exit path - on unload, this function will just loop and consume 100% CPU without any progress * the wake-up mechanism in flush_worker_fn is racy - a wake up will be missed if the process adds entries to the delayed_bios list just before set_current_state(TASK_INTERRUPTIBLE) * flush_delayed_bios_fast submits a bio while holding a global mutex; this may deadlock if we have multiple stacked dm-delay devices and the underlying device attempts to acquire the mutex too * if the target constructor fails, it will call delay_dtr. delay_dtr would attempt to free dc->timer_lock without it being initialized by the constructor. * if the target constructor's kthread allocation fails, delay_dtr would crash trying to dereference dc->worker because it is non-NULL due to ERR_PTR. Fixes: 70bbeb29fab0 ("dm delay: for short delays, use kthread instead of timers and wq") Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer

dm-delay: fix a race between delay_presuspend and delay_bio

2023-11-17T19:38:46+00:00

In delay_presuspend, we set the atomic variable may_delay and then stop the timer and flush pending bios. The intention here is to prevent the delay target from re-arming the timer again. However, this test is racy. Suppose that one thread goes to delay_bio, sees that dc->may_delay is one and proceeds; now, another thread executes delay_presuspend, it sets dc->may_delay to zero, deletes the timer and flushes pending bios. Then, the first thread continues and adds the bio to delayed->list despite the fact that dc->may_delay is false. Fix this bug by changing may_delay's type from atomic_t to bool and only access it while holding the delayed_bios_lock mutex. Note that we don't have to grab the mutex in delay_resume because there are no bios in flight at this point. Signed-off-by: Mikulas Patocka Cc: stable@vger.kernel.org Signed-off-by: Mike Snitzer

dm delay: for short delays, use kthread instead of timers and wq

2023-10-31T15:06:21+00:00

DM delay's current design of using timers and wq to realize the delays is insufficient for delays below ~50ms. This commit enhances the design to use a kthread to flush the expired delays, trading some CPU time (in some cases) for better delay accuracy and delays closer to what the user requested for smaller delays. The new design is chosen as long as all the delays are below 50ms. Since bios can't be completed in interrupt context using a kthread is probably the most reasonable way to approach this. Testing with echo "0 2097152 zero" | dmsetup create dm-zeros for i in $(seq 0 20); do echo "0 2097152 delay /dev/mapper/dm-zeros 0 $i" | dmsetup create dm-delay-${i}ms; done Some performance numbers for comparison, on beaglebone black (single core) CONFIG_HZ_1000=y: fio --name=1msread --rw=randread --bs=4k --runtime=60 --time_based \ --filename=/dev/mapper/dm-delay-1ms Theoretical maximum: 1000 IOPS Previous: 250 IOPS Kthread: 500 IOPS fio --name=10msread --rw=randread --bs=4k --runtime=60 --time_based \ --filename=/dev/mapper/dm-delay-10ms Theoretical maximum: 100 IOPS Previous: 45 IOPS Kthread: 50 IOPS fio --name=1mswrite --rw=randwrite --direct=1 --bs=4k --runtime=60 \ --time_based --filename=/dev/mapper/dm-delay-1ms Theoretical maximum: 1000 IOPS Previous: 498 IOPS Kthread: 1000 IOPS fio --name=10mswrite --rw=randwrite --direct=1 --bs=4k --runtime=60 \ --time_based --filename=/dev/mapper/dm-delay-10ms Theoretical maximum: 100 IOPS Previous: 90 IOPS Kthread: 100 IOPS (This one is just to prove the new design isn't impacting throughput, not really about delays): fio --name=10mswriteasync --rw=randwrite --direct=1 --bs=4k \ --runtime=60 --time_based --filename=/dev/mapper/dm-delay-10ms \ --numjobs=32 --iodepth=64 --ioengine=libaio --group_reporting Previous: 13.3k IOPS Kthread: 13.3k IOPS Signed-off-by: Christian Loehle [Harshit: kthread_create error handling fix in delay_ctr] Signed-off-by: Harshit Mogalapalli Signed-off-by: Mike Snitzer

dm: add helper macro for simple DM target module init and exit

2023-04-11T16:09:08+00:00

Eliminate duplicate boilerplate code for simple modules that contain a single DM target driver without any additional setup code. Add a new module_dm() macro, which replaces the module_init() and module_exit() with template functions that call dm_register_target() and dm_unregister_target() respectively. Signed-off-by: Yangtao Li Signed-off-by: Mike Snitzer