BMC/Intel-BMC/linux.git/fs/btrfs/transaction.c, branch dev

btrfs: Fix lockdep warning of btrfs_run_delayed_iputs()

2015-07-23T01:20:51+00:00

Liu Bo reported a lockdep warning of delayed_iput_sem in xfstests generic/241: [ 2061.345955] ============================================= [ 2061.346027] [ INFO: possible recursive locking detected ] [ 2061.346027] 4.1.0+ #268 Tainted: G W [ 2061.346027] --------------------------------------------- [ 2061.346027] btrfs-cleaner/3045 is trying to acquire lock: [ 2061.346027] (&fs_info->delayed_iput_sem){++++..}, at: [] btrfs_run_delayed_iputs+0x6b/0x100 [ 2061.346027] but task is already holding lock: [ 2061.346027] (&fs_info->delayed_iput_sem){++++..}, at: [] btrfs_run_delayed_iputs+0x6b/0x100 [ 2061.346027] other info that might help us debug this: [ 2061.346027] Possible unsafe locking scenario: [ 2061.346027] CPU0 [ 2061.346027] ---- [ 2061.346027] lock(&fs_info->delayed_iput_sem); [ 2061.346027] lock(&fs_info->delayed_iput_sem); [ 2061.346027] *** DEADLOCK *** It is rarely happened, about 1/400 in my test env. The reason is recursion of btrfs_run_delayed_iputs(): cleaner_kthread -> btrfs_run_delayed_iputs() *1 -> get delayed_iput_sem lock *2 -> iput() -> ... -> btrfs_commit_transaction() -> btrfs_run_delayed_iputs() *1 -> get delayed_iput_sem lock (dead lock) *2 *1: recursion of btrfs_run_delayed_iputs() *2: warning of lockdep about delayed_iput_sem When fs is in high stress, new iputs may added into fs_info->delayed_iputs list when btrfs_run_delayed_iputs() is running, which cause second btrfs_run_delayed_iputs() run into down_read(&fs_info->delayed_iput_sem) again, and cause above lockdep warning. Actually, it will not cause real problem because both locks are read lock, but to avoid lockdep warning, we can do a fix. Fix: Don't do btrfs_run_delayed_iputs() in btrfs_commit_transaction() for cleaner_kthread thread to break above recursion path. cleaner_kthread is calling btrfs_run_delayed_iputs() explicitly in code, and don't need to call btrfs_run_delayed_iputs() again in btrfs_commit_transaction(), it also give us a bonus to avoid stack overflow. Test: No above lockdep warning after patch in 1200 generic/241 tests. Reported-by: Liu Bo Signed-off-by: Zhao Lei Reviewed-by: Liu Bo Signed-off-by: Chris Mason

Btrfs: fix list transaction->pending_ordered corruption

2015-07-11T21:35:05+00:00

When we call btrfs_commit_transaction(), we splice the list "ordered" of our transaction handle into the transaction's "pending_ordered" list, but we don't re-initialize the "ordered" list of our transaction handle, this means it still points to the same elements it used to before the splice. Then we check if the current transaction's state is >= TRANS_STATE_COMMIT_START and if it is we end up calling btrfs_end_transaction() which simply splices again the "ordered" list of our handle into the transaction's "pending_ordered" list, leaving multiple pointers to the same ordered extents which results in list corruption when we are iterating, removing and freeing ordered extents at btrfs_wait_pending_ordered(), resulting in access to dangling pointers / use-after-free issues. Similarly, btrfs_end_transaction() can end up in some cases calling btrfs_commit_transaction(), and both did a list splice of the transaction handle's "ordered" list into the transaction's "pending_ordered" without re-initializing the handle's "ordered" list, resulting in exactly the same problem. This produces the following warning on a kernel with linked list debugging enabled: [109749.265416] ------------[ cut here ]------------ [109749.266410] WARNING: CPU: 7 PID: 324 at lib/list_debug.c:59 __list_del_entry+0x5a/0x98() [109749.267969] list_del corruption. prev->next should be ffff8800ba087e20, but was fffffff8c1f7c35d (...) [109749.287505] Call Trace: [109749.288135] [] dump_stack+0x4f/0x7b [109749.298080] [] ? console_unlock+0x356/0x3a2 [109749.331605] [] warn_slowpath_common+0xa1/0xbb [109749.334849] [] ? __list_del_entry+0x5a/0x98 [109749.337093] [] warn_slowpath_fmt+0x46/0x48 [109749.337847] [] __list_del_entry+0x5a/0x98 [109749.338678] [] btrfs_wait_pending_ordered+0x46/0xdb [btrfs] [109749.340145] [] ? __btrfs_run_delayed_items+0x149/0x163 [btrfs] [109749.348313] [] btrfs_commit_transaction+0x36b/0xa10 [btrfs] [109749.349745] [] ? trace_hardirqs_on+0xd/0xf [109749.350819] [] btrfs_sync_file+0x36f/0x3fc [btrfs] [109749.351976] [] vfs_fsync_range+0x8f/0x9e [109749.360341] [] vfs_fsync+0x1c/0x1e [109749.368828] [] do_fsync+0x34/0x4e [109749.369790] [] SyS_fsync+0x10/0x14 [109749.370925] [] system_call_fastpath+0x12/0x6f [109749.382274] ---[ end trace 48e0d07f7c03d95a ]--- On a non-debug kernel this leads to invalid memory accesses, causing a crash. Fix this by using list_splice_init() instead of list_splice() in btrfs_commit_transaction() and btrfs_end_transaction(). Cc: stable@vger.kernel.org Fixes: 50d9aa99bd35 ("Btrfs: make sure logged extents complete in the current transaction V3" Signed-off-by: Filipe Manana Reviewed-by: David Sterba

btrfs: qgroup: Make snapshot accounting work with new extent-oriented

2015-06-10T16:26:29+00:00

qgroup. Make snapshot accounting work with new extent-oriented mechanism by skipping given root in new/old_roots in create_pending_snapshot(). Signed-off-by: Qu Wenruo Signed-off-by: Chris Mason

btrfs: qgroup: Add the ability to skip given qgroup for old/new_roots.

2015-06-10T16:26:23+00:00

This is used by later qgroup fix patches for snapshot. As current snapshot accounting is done by btrfs_qgroup_inherit(), but new extent oriented quota mechanism will account extent from btrfs_copy_root() and other snapshot things, causing wrong result. So add this ability to handle snapshot accounting. Signed-off-by: Qu Wenruo Signed-off-by: Chris Mason

btrfs: qgroup: Switch to new extent-oriented qgroup mechanism.

2015-06-10T16:25:59+00:00

Switch from old ref_node based qgroup to extent based qgroup mechanism for normal operations. The new mechanism should hugely reduce the overhead of btrfs quota system, and further more, the codes and logic should be more clean and easier to maintain. Signed-off-by: Qu Wenruo Signed-off-by: Chris Mason

btrfs: qgroup: Record possible quota-related extent for qgroup.

2015-06-10T16:25:32+00:00

Add hook in add_delayed_ref_head() to record quota-related extent record into delayed_ref_root->dirty_extent_record rb-tree for later qgroup accounting. Signed-off-by: Qu Wenruo Signed-off-by: Chris Mason

Btrfs: fix -ENOSPC when finishing block group creation

2015-06-03T11:03:04+00:00

While creating a block group, we often end up getting ENOSPC while updating the chunk tree, which leads to a transaction abortion that produces a trace like the following: [30670.116368] WARNING: CPU: 4 PID: 20735 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x52/0x106 [btrfs]() [30670.117777] BTRFS: Transaction aborted (error -28) (...) [30670.163567] Call Trace: [30670.163906] [] dump_stack+0x4f/0x7b [30670.164522] [] ? console_unlock+0x361/0x3ad [30670.165171] [] warn_slowpath_common+0xa1/0xbb [30670.166323] [] ? __btrfs_abort_transaction+0x52/0x106 [btrfs] [30670.167213] [] warn_slowpath_fmt+0x46/0x48 [30670.167862] [] __btrfs_abort_transaction+0x52/0x106 [btrfs] [30670.169116] [] btrfs_create_pending_block_groups+0x101/0x130 [btrfs] [30670.170593] [] __btrfs_end_transaction+0x84/0x366 [btrfs] [30670.171960] [] btrfs_end_transaction+0x10/0x12 [btrfs] [30670.174649] [] btrfs_check_data_free_space+0x11f/0x27c [btrfs] [30670.176092] [] btrfs_fallocate+0x7c8/0xb96 [btrfs] [30670.177218] [] ? __this_cpu_preempt_check+0x13/0x15 [30670.178622] [] vfs_fallocate+0x14c/0x1de [30670.179642] [] ? __fget_light+0x2d/0x4f [30670.180692] [] SyS_fallocate+0x47/0x62 [30670.186737] [] system_call_fastpath+0x12/0x17 [30670.187792] ---[ end trace 0373e6b491c4a8cc ]--- This is because we don't do proper space reservation for the chunk block reserve when we have multiple tasks allocating chunks in parallel. So block group creation has 2 phases, and the first phase essentially checks if there is enough space in the system space_info, allocating a new system chunk if there isn't, while the second phase updates the device, extent and chunk trees. However, because the updates to the chunk tree happen in the second phase, if we have N tasks, each with its own transaction handle, allocating new chunks in parallel and if there is only enough space in the system space_info to allocate M chunks, where M < N, none of the tasks ends up allocating a new system chunk in the first phase and N - M tasks will get -ENOSPC when attempting to update the chunk tree in phase 2 if they need to COW any nodes/leafs from the chunk tree. Fix this by doing proper reservation in the chunk block reserve. The issue could be reproduced by running fstests generic/038 in a loop, which eventually triggered the problem. Signed-off-by: Filipe Manana Signed-off-by: Chris Mason

Btrfs: allow block group cache writeout outside critical section in commit

2015-04-10T21:07:22+00:00

We loop through all of the dirty block groups during commit and write the free space cache. In order to make sure the cache is currect, we do this while no other writers are allowed in the commit. If a large number of block groups are dirty, this can introduce long stalls during the final stages of the commit, which can block new procs trying to change the filesystem. This commit changes the block group cache writeout to take appropriate locks and allow it to run earlier in the commit. We'll still have to redo some of the block groups, but it means we can get most of the work out of the way without blocking the entire FS. Signed-off-by: Chris Mason

Btrfs: reserve space for block groups

2015-04-10T21:06:48+00:00

This changes our delayed refs calculations to include the space needed to write back dirty block groups. Signed-off-by: Chris Mason

Btrfs: account for crcs in delayed ref processing

2015-04-10T21:04:47+00:00

As we delete large extents, we end up doing huge amounts of COW in order to delete the corresponding crcs. This adds accounting so that we keep track of that space and flushing of delayed refs so that we don't build up too much delayed crc work. This helps limit the delayed work that must be done at commit time and tries to avoid ENOSPC aborts because the crcs eat all the global reserves. Signed-off-by: Chris Mason