kernel/linux.git/lib/lz4, branch v6.6.132

lib: make LZ4_decompress_safe_forceExtDict() static

2022-07-18T00:31:39+00:00

LZ4_decompress_safe_forceExtDict() is only used in lib/lz4/lz4_decompress.c, make it static to fix the build warning about "no previous prototype" [1]. [1] https://lore.kernel.org/lkml/202206260948.akgsho1q-lkp@intel.com/ Link: https://lkml.kernel.org/r/1656298965-8698-1-git-send-email-yangtiezhu@loongson.cn Signed-off-by: Tiezhu Yang Reported-by: kernel test robot Signed-off-by: Andrew Morton

lz4: fix LZ4_decompress_safe_partial read out of bound

2022-04-09T00:20:36+00:00

When partialDecoding, it is EOF if we've either filled the output buffer or can't proceed with reading an offset for following match. In some extreme corner cases when compressed data is suitably corrupted, UAF will occur. As reported by KASAN [1], LZ4_decompress_safe_partial may lead to read out of bound problem during decoding. lz4 upstream has fixed it [2] and this issue has been disscussed here [3] before. current decompression routine was ported from lz4 v1.8.3, bumping lib/lz4 to v1.9.+ is certainly a huge work to be done later, so, we'd better fix it first. [1] https://lore.kernel.org/all/000000000000830d1205cf7f0477@google.com/ [2] https://github.com/lz4/lz4/commit/c5d6f8a8be3927c0bec91bcc58667a6cfad244ad# [3] https://lore.kernel.org/all/CC666AE8-4CA4-4951-B6FB-A2EFDE3AC03B@fb.com/ Link: https://lkml.kernel.org/r/20211111105048.2006070-1-guoxuenan@huawei.com Reported-by: syzbot+63d688f1d899c588fb71@syzkaller.appspotmail.com Signed-off-by: Guo Xuenan Reviewed-by: Nick Terrell Acked-by: Gao Xiang Cc: Yann Collet Cc: Chengyang Fan Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

include/linux/unaligned: replace kernel.h with the necessary inclusions

2022-01-20T06:52:53+00:00

When kernel.h is used in the headers it adds a lot into dependency hell, especially when there are circular dependencies are involved. Replace kernel.h inclusion with the list of what is really being used. The rest of the changes are induced by the above and may not be split. Link: https://lkml.kernel.org/r/20211209123823.20425-1-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Acked-by: Arend van Spriel [brcmfmac] Acked-by: Kalle Valo Cc: Arend van Spriel Cc: Franky Lin Cc: Hante Meuleman Cc: Chi-hsien Lin Cc: Wright Feng Cc: Chung-hsien Hsu Cc: Kalle Valo Cc: David S. Miller Cc: Jakub Kicinski Cc: Heikki Krogerus Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

lz4_decompress: declare LZ4_decompress_safe_withPrefix64k static

2021-07-01T18:06:05+00:00

Declare LZ4_decompress_safe_withPrefix64k as static to fix sparse warning: > warning: symbol 'LZ4_decompress_safe_withPrefix64k' was not declared. > Should it be static? Link: https://lkml.kernel.org/r/20210511154345.610569-1-thisisrast7@gmail.com Signed-off-by: Rajat Asthana Reviewed-by: Nick Terrell Cc: Gao Xiang Cc: Joe Perches Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

lib/lz4: explicitly support in-place decompression

2020-12-16T06:46:16+00:00

LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead. Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well. It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa706fbec ("x86, mem: Optimize memcpy by avoiding memory false dependece") will be enabled then. Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature... arch/x86/lib/memcpy_64.S: ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \ - "jmp memcpy_erms", X86_FEATURE_ERMS + "jmp memcpy_orig", X86_FEATURE_ERMS We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel. [1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b [2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921 [3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518 Link: https://lkml.kernel.org/r/20201122030749.2698994-1-hsiangkao@redhat.com Signed-off-by: Gao Xiang Reviewed-by: Nick Terrell Cc: Yann Collet Cc: Miao Xie Cc: Chao Yu Cc: Li Guifu Cc: Guo Xuenan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

lz4: fix kernel decompression speed

2020-08-15T02:56:56+00:00

This patch replaces all memcpy() calls with LZ4_memcpy() which calls __builtin_memcpy() so the compiler can inline it. LZ4 relies heavily on memcpy() with a constant size being inlined. In x86 and i386 pre-boot environments memcpy() cannot be inlined because memcpy() doesn't get defined as __builtin_memcpy(). An equivalent patch has been applied upstream so that the next import won't lose this change [1]. I've measured the kernel decompression speed using QEMU before and after this patch for the x86_64 and i386 architectures. The speed-up is about 10x as shown below. Code Arch Kernel Size Time Speed v5.8 x86_64 11504832 B 148 ms 79 MB/s patch x86_64 11503872 B 13 ms 885 MB/s v5.8 i386 9621216 B 91 ms 106 MB/s patch i386 9620224 B 10 ms 962 MB/s I also measured the time to decompress the initramfs on x86_64, i386, and arm. All three show the same decompression speed before and after, as expected. [1] https://github.com/lz4/lz4/pull/890 Signed-off-by: Nick Terrell Signed-off-by: Andrew Morton Cc: Yann Collet Cc: Gao Xiang Cc: Sven Schmidt <4sschmid@informatik.uni-hamburg.de> Cc: Greg Kroah-Hartman Cc: Ingo Molnar Cc: Arvind Sankar Link: http://lkml.kernel.org/r/20200803194022.2966806-1-nickrterrell@gmail.com Signed-off-by: Linus Torvalds

lib/lz4/lz4_decompress.c: document deliberate use of `&'

2020-06-11T02:14:17+00:00

This operation was intentional, but tools such as smatch will warn that it might not have been. Signed-off-by: Andrew Morton Cc: Yann Collet Cc: Vasily Averin Cc: Gao Xiang Link: http://lkml.kernel.org/r/3bf931c6ea0cae3e23f3485801986859851b4f04.camel@perches.com Signed-off-by: Linus Torvalds

lz4: do not export static symbol

2019-09-20T16:06:26+00:00

Kbuild now complains (rightly) about it. Signed-off-by: Linus Torvalds

treewide: Add SPDX license identifier - Makefile/Kconfig

2019-05-21T08:50:46+00:00

Add SPDX license identifiers to all Make/Kconfig files which: - Have no license information of any form These files fall under the project license, GPL v2 only. The resulting SPDX license identifier is: GPL-2.0-only Signed-off-by: Thomas Gleixner Signed-off-by: Greg Kroah-Hartman

lib/lz4: update LZ4 decompressor module

2018-10-31T15:54:14+00:00

Update the LZ4 compression module based on LZ4 v1.8.3 in order for the erofs file system to use the newest LZ4_decompress_safe_partial() which can now decode exactly the nb of bytes requested [1] to take place of the open hacked code in the erofs file system itself. Currently, apart from the erofs file system, no other users use LZ4_decompress_safe_partial, so no worry about the interface. In addition, LZ4 v1.8.x boosts up decompression speed compared to the current code which is based on LZ4 v1.7.3, mainly due to shortcut optimization for the specific common LZ4-sequences [2]. lzbench testdata (tested in kirin710, 8 cores, 4 big cores at 2189Mhz, 2GB DDR RAM at 1622Mhz, with enwik8 testdata [3]): Compressor name Compress. Decompress. Compr. size Ratio Filename memcpy 5004 MB/s 4924 MB/s 100000000 100.00 enwik8 lz4hc 1.7.3 -9 12 MB/s 653 MB/s 42203253 42.20 enwik8 lz4hc 1.8.0 -9 12 MB/s 908 MB/s 42203096 42.20 enwik8 lz4hc 1.8.3 -9 11 MB/s 965 MB/s 42203094 42.20 enwik8 [1] https://github.com/lz4/lz4/issues/566 https://github.com/lz4/lz4/commit/08d347b5b217b011ff7487130b79480d8cfdaeb8 [2] v1.8.1 perf: slightly faster compression and decompression speed https://github.com/lz4/lz4/commit/a31b7058cb97e4393da55e78a77a1c6f0c9ae038 v1.8.2 perf: slightly faster HC compression and decompression speed https://github.com/lz4/lz4/commit/45f8603aae389d34c689d3ff7427b314071ccd2c https://github.com/lz4/lz4/commit/1a191b3f8d26b50a7c1d41590b529ec308d768cd [3] http://mattmahoney.net/dc/textdata.html http://mattmahoney.net/dc/enwik8.zip Link: http://lkml.kernel.org/r/1537181207-21932-1-git-send-email-gaoxiang25@huawei.com Signed-off-by: Gao Xiang Tested-by: Guo Xuenan Cc: Colin Ian King Cc: Yann Collet Cc: Greg Kroah-Hartman Cc: Fang Wei Cc: Chao Yu Cc: Miao Xie Cc: Sven Schmidt <4sschmid@informatik.uni-hamburg.de> Cc: Kyungsik Lee Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds