kernel/linux.git/fs/btrfs/struct-funcs.c, branch v6.1.168

btrfs: remove redundant check in up check_setget_bounds

2022-07-25T15:45:33+00:00

There are two separate checks in the bounds checker, the first one being a special case of the second. As this function is performance critical due to checking access to any eb member, reducing the size can slightly improve performance. On a release build on x86_64 the helper is completely inlined so the function call overhead is also gone. There was a report of 5% performance drop on metadata heavy workload, that disappeared after disabling asserts. The most significant part of that is the bounds checker. https://lore.kernel.org/linux-btrfs/20200724164147.39925-1-josef@toxicpanda.com/ After the analysis, the optimized code removes the worst overhead which is the function call and the performance was restored. https://lore.kernel.org/linux-btrfs/20200730110943.GE3703@twin.jikos.cz/ 1. baseline, asserts on, setget check on run time: 46s run time with perf: 48s 2. asserts on, comment out setget check run time: 44s run time with perf: 47s So this is confirms the 5% difference 3. asserts on, optimized seget check run time: 44s run time with perf: 47s The optimizations are reducing the number of ifs to 1 and inlining the hot path. Low-level stuff, gets the performance back. Patch below. 4. asserts off, no setget check run time: 44s run time with perf: 45s This verifies that asserts other than the setget check have negligible impact on performance and it's not harmful to keep them on. Analysis where the performance is lost: * check_setget_bounds is short function, but it's still a function call, changing the flow of instructions and given how many times it's called the overhead adds up * there are two conditions, one to check if the range is completely outside (member_offset > eb->len) or partially inside (member_offset + size > eb->len) Reviewed-by: Johannes Thumshirn Signed-off-by: David Sterba

btrfs: add special case to setget helpers for 64k pages

2021-08-23T11:18:58+00:00

On 64K pages the size of the extent_buffer::pages array is 1 and compilation with -Warray-bounds warns due to kaddr = page_address(eb->pages[idx + 1]); when reading byte range crossing page boundary. This does never actually overflow the array because on 64K because all the data fit in one page and bounds are checked by check_setget_bounds. To fix the reported overflows and warnings add a compile-time condition that will allow compiler to eliminate the dead code that reads from the idx + 1 page. Link: https://lore.kernel.org/lkml/20210623083901.1d49d19d@canb.auug.org.au/ CC: Gustavo A. R. Silva Signed-off-by: David Sterba

btrfs: handle sectorsize < PAGE_SIZE case for extent buffer accessors

2020-12-09T18:16:10+00:00

To support sectorsize < PAGE_SIZE case, we need to take extra care of extent buffer accessors. Since sectorsize is smaller than PAGE_SIZE, one page can contain multiple tree blocks, we must use eb->start to determine the real offset to read/write for extent buffer accessors. This patch introduces two helpers to do this: - get_eb_page_index() This is to calculate the index to access extent_buffer::pages. It's just a simple wrapper around "start >> PAGE_SHIFT". For sectorsize == PAGE_SIZE case, nothing is changed. For sectorsize < PAGE_SIZE case, we always get index as 0, and the existing page shift also works. - get_eb_offset_in_page() This is to calculate the offset to access extent_buffer::pages. This needs to take extent_buffer::start into consideration. For sectorsize == PAGE_SIZE case, extent_buffer::start is always aligned to PAGE_SIZE, thus adding extent_buffer::start to offset_in_page() won't change the result. For sectorsize < PAGE_SIZE case, adding extent_buffer::start gives us the correct offset to access. This patch will touch the following parts to cover all extent buffer accessors: - BTRFS_SETGET_HEADER_FUNCS() - read_extent_buffer() - read_extent_buffer_to_user() - memcmp_extent_buffer() - write_extent_buffer_chunk_tree_uuid() - write_extent_buffer_fsid() - write_extent_buffer() - memzero_extent_buffer() - copy_extent_buffer_full() - copy_extent_buffer() - memcpy_extent_buffer() - memmove_extent_buffer() - btrfs_get_token_##bits() - btrfs_get_##bits() - btrfs_set_token_##bits() - btrfs_set_##bits() - generic_bin_search() Signed-off-by: Goldwyn Rodrigues Signed-off-by: Qu Wenruo Reviewed-by: David Sterba Signed-off-by: David Sterba

btrfs: use unaligned helpers for stack and header set/get helpers

2020-10-07T10:13:23+00:00

In the definitions generated by BTRFS_SETGET_HEADER_FUNCS there's direct pointer assignment but we should use the helpers for unaligned access for clarity. It hasn't been a problem so far because of the natural alignment. Similarly for BTRFS_SETGET_STACK_FUNCS, that usually get a structure from stack that has an aligned start but some members may not be aligned due to packing. This as well hasn't caused problems so far. Move the put/get_unaligned_le8 stubs to ctree.h so we can use them. Signed-off-by: David Sterba

btrfs: update documentation of set/get helpers

2020-05-25T09:25:35+00:00

Signed-off-by: David Sterba

btrfs: optimize split page write in btrfs_set_token_##bits

2020-05-25T09:25:34+00:00

The fallback path calls helper write_extent_buffer to do write of the data spanning two extent buffer pages. As the size is known, we can do the write directly in two steps. This removes one function call and compiler can optimize memcpy as the sizes are known at compile time. The cached token address is set to the second page. Reviewed-by: Johannes Thumshirn Signed-off-by: David Sterba

btrfs: optimize split page write in btrfs_set_##bits

2020-05-25T09:25:34+00:00

The helper write_extent_buffer is called to do write of the data spanning two extent buffer pages. As the size is known, we can do the write directly in two steps. This removes one function call and compiler can optimize memcpy as the sizes are known at compile time. Reviewed-by: Johannes Thumshirn Signed-off-by: David Sterba

btrfs: optimize split page read in btrfs_get_token_##bits

2020-05-25T09:25:34+00:00

The fallback path calls helper read_extent_buffer to do read of the data spanning two extent buffer pages. As the size is known, we can do the read directly in two steps. This removes one function call and compiler can optimize memcpy as the sizes are known at compile time. The cached token address is set to the second page. Reviewed-by: Johannes Thumshirn Signed-off-by: David Sterba

btrfs: optimize split page read in btrfs_get_##bits

2020-05-25T09:25:34+00:00

The helper read_extent_buffer is called to do read of the data spanning two extent buffer pages. As the size is known, we can do the read directly in two steps. This removes one function call and compiler can optimize memcpy as the sizes are known at compile time. Reviewed-by: Johannes Thumshirn Signed-off-by: David Sterba

btrfs: constify extent_buffer in the API functions

2020-05-25T09:25:34+00:00

There are many helpers around extent buffers, found in extent_io.h and ctree.h. Most of them can be converted to take constified eb as there are no changes to the extent buffer structure itself but rather the pages. Reviewed-by: Johannes Thumshirn Signed-off-by: David Sterba