kernel/linux.git/arch/x86, branch v6.12.93

x86/ftrace: Relocate %rip-relative percpu refs in dynamic trampolines

2026-06-09T10:26:05+00:00

[ Upstream commit a17dc12bfed8868e6a86f3b45c16065a70641acb ] With CONFIG_CALL_DEPTH_TRACKING enabled on an x86 retbleed-affected platform (eg: Skylake), with retbleed=stuff, registering a dynamic ftrace trampoline crashes on the first call into the traced function: BUG: unable to handle page fault for address: ffff88817ae18880 #PF: supervisor write access in kernel mode #PF: error_code(0x0002) - not-present page PGD 4b53067 P4D 4b53067 PUD 0 Oops: Oops: 0002 [#1] SMP PTI CPU: 3 UID: 0 PID: 187 Comm: usleep Not tainted 7.0.10 #243 PREEMPT(full) Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.17.0-2-2 04/01/2014 Code: 24 78 00 00 00 00 48 89 ea 48 89 54 24 20 48 8b b4 24 b8 00 00 00 48 8b bc 24 b0 00 00 00 48 89 bc 24 80 00 00 00 48 83 ef 05 <65> 48 c1 3d 1f a8 b6 02 05 48 8b 15 f6 00 00 00 4c 89 3c 24 4c 89 Call Trace: ? find_held_lock ? exc_page_fault ? lock_release ? __x64_sys_clock_nanosleep ? lockdep_hardirqs_on_prepare ? trace_hardirqs_on __x64_sys_clock_nanosleep do_syscall_64 ? exc_page_fault ? call_depth_return_thunk entry_SYSCALL_64_after_hwframe ... Kernel panic - not syncing: Fatal exception This small reproducer allows to easily trigger the crash: # echo 'p __x64_sys_clock_nanosleep' > /sys/kernel/tracing/kprobe_events # echo 1 > /sys/kernel/tracing/events/kprobes/p___x64_sys_clock_nanosleep_0/enable # usleep 1 Monitoring the crash under GDB points to the exact instruction in charge of incrementing the call depth: sarq $5, %gs:__x86_call_depth(%rip) This instruction matches the one inserted by the ftrace_regs_caller from ftrace_64.S. This emitted code was likely working fine until the introduction of 59bec00ace28 ("x86/percpu: Introduce %rip-relative addressing to PER_CPU_VAR()"): it has made the call depth accounting addressing relative to $rip, instead of being based on an absolute address. As this code exact location depends on where the trampoline lives in memory, the corresponding displacement needs to be adjusted at runtime to actually correctly find the per-cpu __x86_call_depth value, otherwise the targeted address is wrong, leading to the page fault seen above. Fix the %rip-relative displacement of the copied CALL_DEPTH_ACCOUNT instruction (from ftrace_regs_caller) by calling text_poke_apply_relocation(), as it is done for example by the x86 BPF JIT compiler through x86_call_depth_emit_accounting(). This corrects both CALL_DEPTH_ACCOUNT slots, in ftrace_caller and ftrace_regs_caller. [ bp: Massage. ] Fixes: 59bec00ace28 ("x86/percpu: Introduce %rip-relative addressing to PER_CPU_VAR()") Signed-off-by: Alexis Lothoré (eBPF Foundation) Signed-off-by: Borislav Petkov (AMD) Acked-by: Peter Zijlstra (Intel) Acked-by: Steven Rostedt Cc: Link: https://patch.msgid.link/20260527-fix_call_depth_in_trampoline-v1-1-1c1abc8ae310@bootlin.com Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

x86/alternatives: Rename 'apply_relocation()' to 'text_poke_apply_relocation()'

2026-06-09T10:26:05+00:00

[ Upstream commit 023f42dd59203be8ad2fc0574af32d3b4ad041ec ] Join the text_poke_*() API namespace. Signed-off-by: Ingo Molnar Cc: Juergen Gross Cc: "H . Peter Anvin" Cc: Linus Torvalds Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20250411054105.2341982-52-mingo@kernel.org Stable-dep-of: a17dc12bfed8 ("x86/ftrace: Relocate %rip-relative percpu refs in dynamic trampolines") Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

ring-buffer: Flush and stop persistent ring buffer on panic

2026-06-09T10:26:03+00:00

[ Upstream commit a494d3c8d5392bcdff83c2a593df0c160ff9f322 ] On real hardware, panic and machine reboot may not flush hardware cache to memory. This means the persistent ring buffer, which relies on a coherent state of memory, may not have its events written to the buffer and they may be lost. Moreover, there may be inconsistency with the counters which are used for validation of the integrity of the persistent ring buffer which may cause all data to be discarded. To avoid this issue, stop recording of the ring buffer on panic and flush the cache of the ring buffer's memory. Fixes: e645535a954a ("tracing: Add option to use memmapped memory for trace boot instance") Cc: stable@vger.kernel.org Cc: Will Deacon Cc: Mathieu Desnoyers Cc: Ian Rogers Link: https://patch.msgid.link/177751969602.2136606.12031934362587643488.stgit@mhiramat.tok.corp.google.com Signed-off-by: Masami Hiramatsu (Google) Acked-by: Catalin Marinas Acked-by: Geert Uytterhoeven Signed-off-by: Steven Rostedt Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

x86/kexec: Disable KCOV instrumentation after load_segments()

2026-06-09T10:26:01+00:00

[ Upstream commit 917e3ad3321e75ca0223d5ccf26ceda116aa51e1 ] The load_segments() function changes segment registers, invalidating GS base (which KCOV relies on for per-cpu data). When CONFIG_KCOV is enabled, any subsequent instrumented C code call (e.g. native_gdt_invalidate()) begins crashing the kernel in an endless loop. To reproduce the problem, it's sufficient to do kexec on a KCOV-instrumented kernel: $ kexec -l /boot/otherKernel $ kexec -e The real-world context for this problem is enabling crash dump collection in syzkaller. For this, the tool loads a panic kernel before fuzzing and then calls makedumpfile after the panic. This workflow requires both CONFIG_KEXEC and CONFIG_KCOV to be enabled simultaneously. Adding safeguards directly to the KCOV fast-path (__sanitizer_cov_trace_pc()) is also undesirable as it would introduce an extra performance overhead. Disabling instrumentation for the individual functions would be too fragile, so disable KCOV instrumentation for the entire machine_kexec_64.c and physaddr.c. If coverage-guided fuzzing ever needs these components in the future, other approaches should be considered. The problem is not relevant for 32 bit kernels as CONFIG_KCOV is not supported there. [ bp: Space out comment for better readability. ] Fixes: 0d345996e4cb ("x86/kernel: increase kcov coverage under arch/x86/kernel folder") Signed-off-by: Aleksandr Nogikh Signed-off-by: Borislav Petkov (AMD) Reviewed-by: Dmitry Vyukov Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260325154825.551191-1-nogikh@google.com Signed-off-by: Sasha Levin

x86/boot: Disable stack protector for early boot code

2026-06-09T10:26:01+00:00

[ Upstream commit a9a76b38aaf577887103e3ebb41d70e6aa5a4b19 ] On 64-bit, this will prevent crashes when the canary access is changed from %gs:40 to %gs:__stack_chk_guard(%rip). RIP-relative addresses from the identity-mapped early boot code will target the wrong address with zero-based percpu. KASLR could then shift that address to an unmapped page causing a crash on boot. This early boot code runs well before user-space is active and does not need stack protector enabled. Signed-off-by: Brian Gerst Signed-off-by: Ingo Molnar Reviewed-by: Ard Biesheuvel Cc: Linus Torvalds Link: https://lore.kernel.org/r/20250123190747.745588-4-brgerst@gmail.com Stable-dep-of: 917e3ad3321e ("x86/kexec: Disable KCOV instrumentation after load_segments()") Signed-off-by: Sasha Levin

KVM: SEV: Don't explicitly pass PSC buffer to snp_begin_psc()

2026-06-09T10:25:51+00:00

commit ebe4b2dc9cfbfb2d8f665667c4d08f4c6c9bec05 upstream. Stop explicitly passing the PSC buffer to snp_begin_psc(): it *must* be the scratch area. This will allow fixing a variety of bugs without further complicating the code. No functional change intended. Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky Reviewed-by: Michael Roth Signed-off-by: Sean Christopherson Message-ID: <20260501202250.2115252-9-seanjc@google.com> Signed-off-by: Paolo Bonzini Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Use READ_ONCE() when reading entries/indices from PSC buffer

2026-06-09T10:25:51+00:00

commit c8cc238093ca6c99267032f6cfe78f59389f3157 upstream. Use READ_ONCE() when reading entries/indices from the guest-accessible Page State Change buffer to defend against TOCTOU bugs. Don't bother with READ_ONCE()/WRITE_ONCE() for cases where KVM is writing (and not consuming the result!), as the guest isn't supposed to touch the buffer while it's being processed. I.e. using READ_ONCE() is all about protecting against misbehaving guests. Fixes: 9b54e248d264 ("KVM: SEV: Add support to handle Page State Change VMGEXIT") Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky Signed-off-by: Sean Christopherson Message-ID: <20260501202250.2115252-11-seanjc@google.com> Signed-off-by: Paolo Bonzini Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Check PSC request indices against the actual size of the buffer

2026-06-09T10:25:51+00:00

commit 121d88de56bc5c0ba0ce2f6381af67f948a7e7c1 upstream. When processing Page State Change (PSC) requests, validate the PSC buffer against the effective size of the scratch area, which could be less than the maximum size if the guest provided a pointer that isn't exactly at the start of the GHCB shared buffer. Fixes: 9b54e248d264 ("KVM: SEV: Add support to handle Page State Change VMGEXIT") Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky Reviewed-by: Michael Roth Signed-off-by: Sean Christopherson Message-ID: <20260501202250.2115252-10-seanjc@google.com> Signed-off-by: Paolo Bonzini Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Compute the correct max length of the in-GHCB scratch area

2026-06-09T10:25:51+00:00

commit 5867d7e202e09f037cefe77f7af4413c7c0fa088 upstream. When setting the length of the GHCB scratch area, and the area is in the GHCB shared buffer, set the effective length of the scratch area to the max possible size given the start of the guest-provided pointer, and the end of the shared buffer. The code was "fine" when first introduced, as KVM doesn't consult the length of the buffer when emulating MMIO, because the passed in @len always specifies the *max* size required. But for PSC requests, the incoming @len is just the minimum length (to process the header), and KVM needs to know the full size of the scratch area to avoid buffer overflows (spoiler alert). Opportunistically rename @len => @min_len to better reflect its role. Fixes: 9b54e248d264 ("KVM: SEV: Add support to handle Page State Change VMGEXIT") Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky Reviewed-by: Michael Roth Signed-off-by: Sean Christopherson Message-ID: <20260501202250.2115252-7-seanjc@google.com> Signed-off-by: Paolo Bonzini Signed-off-by: Greg Kroah-Hartman

KVM: SEV: WARN if KVM attempts to setup scratch area with min_len==0

2026-06-09T10:25:51+00:00

commit f185e05dce6f170f83c4ba602e969b1c3c7a22e6 upstream. Now that all paths in KVM properly validate the length needed for the scratch area, and are guaranteed to pass in a non-zero length, WARN if KVM attempts to configured the scratch area with min_len==0 to guard against future bugs. Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky Reviewed-by: Michael Roth Signed-off-by: Sean Christopherson Message-ID: <20260501202250.2115252-8-seanjc@google.com> Signed-off-by: Paolo Bonzini Signed-off-by: Greg Kroah-Hartman