From 3a2486cc1da5cf637fe5da4540929d67c4540022 Mon Sep 17 00:00:00 2001 From: Gary Guo Date: Tue, 3 Feb 2026 11:34:10 +0000 Subject: kbuild: rust: provide an option to inline C helpers into Rust A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to allow C helpers (which were created to allow Rust to call into inline/macro C functions without having to re-implement the logic in Rust) to be inlined into Rust crates without performing global LTO. If the option is enabled, the following is performed: * For helpers, instead of compiling them to an object file to be linked into vmlinux, they're compiled to LLVM IR bitcode. Two versions are generated: one for built-in code (`helpers.bc`) and one for modules (`helpers_module.bc`, with -DMODULE defined). This ensures that C macros/inlines that behave differently for modules (e.g. static calls) function correctly when inlined. * When a Rust crate or object is compiled, instead of generating an object file, LLVM bitcode is generated. * llvm-link is invoked with --internalize to combine the helper bitcode with the crate bitcode. This step is similar to LTO, but this is much faster since it only needs to inline the helpers. * clang is invoked to turn the combined bitcode into a final object file. * Since clang may produce LLVM bitcode when LTO is enabled, and objtool requires ELF input, $(cmd_ld_single) is invoked to ensure the object is converted to ELF before objtool runs. The --internalize flag tells llvm-link to treat all symbols in helpers.bc using `internal` linkage [1]. This matches the behavior of `clang` on `static inline` functions, and avoids exporting the symbol from the object file. To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust: Disallow BTF generation with Rust + LTO") for details. We have an intended triple mismatch of `aarch64-unknown-none` vs `aarch64-unknown-linux-gnu`, so we pass --suppress-warnings to llvm-link to suppress it. I considered adding some sort of check that KBUILD_MODNAME is not present in helpers_module.bc, but this is actually not so easy to carry out because .bc files store strings in a weird binary format, so you cannot just grep it for a string to check whether it ended up using KBUILD_MODNAME anywhere. [ Andreas writes: For the rnull driver, enabling helper inlining with this patch gives an average speedup of 2% over the set of 120 workloads that we publish on [2]. Link: https://rust-for-linux.com/null-block-driver [2] This series also uncovered a pre-existing UB instance thanks to an `objtool` warning which I noticed while testing the series (details in the mailing list). - Miguel ] Link: https://github.com/llvm/llvm-project/pull/170397 [1] Co-developed-by: Boqun Feng Signed-off-by: Boqun Feng Co-developed-by: Matthew Maurer Signed-off-by: Matthew Maurer Signed-off-by: Gary Guo Co-developed-by: Alice Ryhl Signed-off-by: Alice Ryhl Reviewed-by: Nathan Chancellor Tested-by: Nathan Chancellor Tested-by: Andreas Hindborg Reviewed-by: Andreas Hindborg Link: https://patch.msgid.link/20260203-inline-helpers-v2-3-beb8547a03c9@google.com [ Some changes, apart from the rebase: - Added "(EXPERIMENTAL)" to Kconfig as the commit mentions. - Added `depends on ARM64 || X86_64` and `!UML` for now, since this is experimental, other architectures may require other changes (e.g. the issues I mentioned in the mailing list for ARM and UML) and they are not really tested so far. So let arch maintainers pick this up if they think it is worth it. - Gated the `cmd_ld_single` step also into the new mode, which also means that any possible future `objcopy` step is done after the translation, as expected. - Added `.gitignore` for `.bc` with exception for existing script. - Added `part-of-*` for helpers bitcode files as discussed, and dropped `$(if $(filter %_module.bc,$@),-DMODULE)` since `-DMODULE` is already there (would be duplicated otherwise). - Moved `LLVM_LINK` to keep binutils list alphabetized. - Fixed typo in title. - Dropped second `cmd_ld_single` commit message paragraph. - Miguel ] Signed-off-by: Miguel Ojeda --- lib/Kconfig.debug | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) (limited to 'lib/Kconfig.debug') diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 93f356d2b3d9..30d069676309 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -3577,6 +3577,23 @@ config RUST_KERNEL_DOCTESTS If unsure, say N. +config RUST_INLINE_HELPERS + bool "Inline C helpers into Rust code (EXPERIMENTAL)" + depends on RUST && RUSTC_CLANG_LLVM_COMPATIBLE + depends on EXPERT + depends on ARM64 || X86_64 + depends on !UML + help + Inlines C helpers into Rust code using Link Time Optimization. + + If this option is enabled, C helper functions declared in + rust/helpers/ are inlined into Rust code, which is helpful for + performance of Rust code. This requires a matching LLVM version for + Clang and rustc. + + If you are sure that you're using Clang and rustc with matching LLVM + versions, say Y. Otherwise say N. + endmenu # "Rust" endmenu # Kernel hacking -- cgit v1.2.3