kernel/linux.git/lib/zstd, branch v6.6.132

zstd: Increase DYNAMIC_BMI2 GCC version cutoff from 4.8 to 11.0 to work around compiler segfault

2025-04-25T08:45:11+00:00

[ Upstream commit 1400c87e6cac47eb243f260352c854474d9a9073 ] Due to pending percpu improvements in -next, GCC9 and GCC10 are crashing during the build with: lib/zstd/compress/huf_compress.c:1033:1: internal compiler error: Segmentation fault 1033 | { | ^ Please submit a full bug report, with preprocessed source if appropriate. See for instructions. The DYNAMIC_BMI2 feature is a known-challenging feature of the ZSTD library, with an existing GCC quirk turning it off for GCC versions below 4.8. Increase the DYNAMIC_BMI2 version cutoff to GCC 11.0 - GCC 10.5 is the last version known to crash. Reported-by: Michael Kelley Debugged-by: Ard Biesheuvel Signed-off-by: Ingo Molnar Cc: https://lore.kernel.org/r/SN6PR02MB415723FBCD79365E8D72CA5FD4D82@SN6PR02MB4157.namprd02.prod.outlook.com Cc: Linus Torvalds Signed-off-by: Sasha Levin

zstd: Fix array-index-out-of-bounds UBSAN warning

2023-12-13T17:44:58+00:00

[ Upstream commit 77618db346455129424fadbbaec596a09feaf3bb ] Zstd used an array of length 1 to mean a flexible array for C89 compatibility. Switch to a C99 flexible array to fix the UBSAN warning. Tested locally by booting the kernel and writing to and reading from a BtrFS filesystem with zstd compression enabled. I was unable to reproduce the issue before the fix, however it is a trivial change. Link: https://lkml.kernel.org/r/20231012213428.1390905-1-nickrterrell@gmail.com Reported-by: syzbot+1f2eb3e8cd123ffce499@syzkaller.appspotmail.com Reported-by: Eric Biggers Reported-by: Kees Cook Signed-off-by: Nick Terrell Reviewed-by: Kees Cook Signed-off-by: Sasha Levin

add intptr_t

2023-06-10T00:44:13+00:00

Add signed intptr_t given that a) it is standard type and b) uintptr_t is in tree. Link: https://lkml.kernel.org/r/ed66b9e4-1fb7-45be-9bb9-d4bc291c691f@p183 Signed-off-by: Alexey Dobriyan Signed-off-by: Andrew Morton

zstd: Fix definition of assert()

2023-03-06T23:54:54+00:00

assert(x) should emit a warning if x is false. WARN_ON(x) emits a warning if x is true. Thus, assert(x) should be defined as WARN_ON(!x) rather than WARN_ON(x). Signed-off-by: Jonathan Neuschäfer Signed-off-by: Nick Terrell

lib: zstd: Backport fix for in-place decompression

2023-03-06T23:51:44+00:00

Backport the relevant part of upstream commit 5b266196 [0]. This fixes in-place decompression for x86-64 kernel decompression. It uses a bound of 131072 + (uncompressed_size >> 8), which can be violated after upstream commit 6a7ede3d [1], as zstd can use part of the output buffer as temporary storage, and without this patch needs a bound of ~262144. The fix is for zstd to detect that the input and output buffers overlap, so that zstd knows it can't use the overlapping portion of the output buffer as tempoary storage. If the margin is not large enough, this will ensure that zstd will fail the decompression, rather than overwriting part of the input data, and causing corruption. This fix has been landed upstream and is in release v1.5.4. That commit also adds unit and fuzz tests to verify that the margin we use is respected, and correct. That means that the fix is well tested upstream. I have not been able to reproduce the potential bug in x86-64 kernel decompression locally, nor have I recieved reports of failures to decompress the kernel. It is possible that compression saves enough space to make it very hard for the issue to appear. I've boot tested the zstd compressed kernel on x86-64 and i386 with this patch, which uses in-place decompression, and sanity tested zstd compression in btrfs / squashfs to make sure that we don't see any issues, but other uses of zstd shouldn't be affected, because they don't use in-place decompression. Thanks to Vasily Gorbik for debugging a related issue on s390, which was triggered by the same commit, but was a bug in how __decompress() was called [2]. And to Sasha Levin for the CC alerting me of the issue. [0] https://github.com/facebook/zstd/commit/5b266196a41e6a15e21bd4f0eeab43b938db1d90 [1] https://github.com/facebook/zstd/commit/6a7ede3dfccbf3e0a5928b4224a039c260dcff72 [2] https://lore.kernel.org/r/patch-1.thread-41c676.git-41c676c2d153.your-ad-here.call-01675030179-ext-9637@work.hours CC: Vasily Gorbik CC: Heiko Carstens CC: Sasha Levin CC: Yann Collet Signed-off-by: Nick Terrell

lib: zstd: Fix -Wstringop-overflow warning

2023-03-06T23:51:44+00:00

Fix the following -Wstringop-overflow warning when building with GCC 11+: lib/zstd/decompress/huf_decompress.c: In function ‘HUF_readDTableX2_wksp’: lib/zstd/decompress/huf_decompress.c:700:5: warning: ‘HUF_fillDTableX2.constprop’ accessing 624 bytes in a region of size 52 [-Wstringop-overflow=] 700 | HUF_fillDTableX2(dt, maxTableLog, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 701 | wksp->sortedSymbol, sizeOfSort, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 702 | wksp->rankStart0, wksp->rankVal, maxW, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 703 | tableLog+1, | ~~~~~~~~~~~ 704 | wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32)); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ lib/zstd/decompress/huf_decompress.c:700:5: note: referencing argument 6 of type ‘U32 (*)[13]’ {aka ‘unsigned int (*)[13]’} lib/zstd/decompress/huf_decompress.c:571:13: note: in a call to function ‘HUF_fillDTableX2.constprop’ 571 | static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog, | ^~~~~~~~~~~~~~~~ by using pointer notation instead of array notation. This is one of the last remaining warnings to be fixed before globally enabling -Wstringop-overflow. Co-developed-by: Gustavo A. R. Silva Signed-off-by: Gustavo A. R. Silva Cc: Nick Terrell Signed-off-by: Kees Cook Signed-off-by: Nick Terrell

zstd: import usptream v1.5.2

2022-10-24T19:12:32+00:00

Updates the kernel's zstd library to v1.5.2, the latest zstd release. The upstream tag it is updated to is `v1.5.2-kernel`, which contains several cherry-picked commits on top of the v1.5.2 release which are required for the kernel update. I will create this tag once the PR is ready to merge, until then reference the temporary upstream branch `v1.5.2-kernel-cherrypicks`. I plan to submit this patch as part of the v6.2 merge window. I've done basic build testing & testing on x86-64, i386, and aarch64. I'm merging these patches into my `zstd-next` branch, which is pulled into `linux-next` for further testing. I've benchmarked BtrFS with zstd compression on a x86-64 machine, and saw these results. Decompression speed is a small win across the board. The lower compression levels 1-4 see both compression speed and compression ratio wins. The higher compression levels see a small compression speed loss and about neutral ratio. I expect the lower compression levels to be used much more heavily than the high compression levels, so this should be a net win. Level CTime DTime Ratio 1 -2.95% -1.1% -0.7% 3 -3.5% -1.2% -0.5% 5 +3.7% -1.0% +0.0% 7 +3.2% -0.9% +0.0% 9 -4.3% -0.8% +0.1% Signed-off-by: Nick Terrell

zstd: Move zstd-common module exports to zstd_common_module.c

2022-10-24T19:12:26+00:00

The zstd codebase is imported from the upstream zstd repo, and is over-written on every update. Upstream keeps the kernel specific code separate from the main library. So the module definition is moved into the zstd_common_module.c file. This matches the pattern followed by the zstd-compress and zstd-decompress files. I've done build and boot testing on x86-64, i386, and aarch64. I've verified that zstd built both as modules and built-in build and boot. Signed-off-by: Nick Terrell

lib: zstd: Fix comment typo

2022-10-24T19:11:52+00:00

The double `when' is duplicated in line 999, remove one. Signed-off-by: Xin Gao Signed-off-by: Nick Terrell

lib: zstd: fix repeated words in comments

2022-10-24T19:11:40+00:00

Delete the redundant word 'the'. Signed-off-by: Jilin Yuan Signed-off-by: Nick Terrell