kernel/linux.git/arch/x86/kvm, branch linux-7.2.y

KVM: SEV: Wire up kvm_x86_ops.gmem_xxx() if and only if CONFIG_KVM_AMD_SEV=y

2026-09-02T12:33:18+00:00

commit 01a96ff30dde5127c37497f1e098e639e7ae152f upstream. Wire up the SEV-SNP guest_memfd kvm_x86_ops hooks if and only if SEV is actually enabled, and drop the now-unnecessary stubs. Leaving the hooks NULL allows the static call infrastructure to elide the CALL+RET, and more importantly, referencing the hooks if and only if SEV support is enabled will allow conditionally definining the hooks using their corresponding HAVE_KVM_ARCH_GMEM_XXX Kconfig. No functional change intended. Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Ackerley Tng Link: https://patch.msgid.link/20260709204948.1988414-5-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Mark vCPU RUNNABLE after AP_CREATE, even if VMSA is unusable

2026-09-02T12:33:18+00:00

commit 98ade8c48c28c227fe2e80e545ff0c57cd4712a3 upstream. Always mark the vCPU as RUNNABLE after responding to AP_CREATE, even if the guest-specified VMSA is unusable, e.g. isn't backed by a memslot or doesn't have a backing guest_memfd page. If the VMSA is unusable, leaving the vCPU in a non-running state will effectively hang the vCPU instead of reporting an error to userspace. This will also allow retrying the VMSA load in the future, to fix a bug where KVM doesn't honor guest_memfd invalidation events, e.g. if AP_CREATION races with PUNCH_HOLE. Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Michael Roth Link: https://patch.msgid.link/20260709204948.1988414-4-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Extract loading of guest-provided VMSA to a separate helper

2026-09-02T12:33:18+00:00

commit 0060569e4f18a7dee2dd8728595e909f19a23c24 upstream. Extract the loading/retrieval of a guest-provided VMSA to a separate helper so that KVM can reuse the core logic when refreshing the VMSA after an MMU invalidation from guest_memfd. No functional change intended. Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Michael Roth Link: https://patch.msgid.link/20260709204948.1988414-3-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Track the GPA of the guest-controlled VMSA used for SNP guests

2026-09-02T12:33:18+00:00

commit 42a39ad5d592aec87a70527a4e694f6210694482 upstream. Track the GPA of the guest-provided VMSA used after AP_CREATION events when running SNP guests, instead of simply tracking whether or not the vCPU is using a guest-provided VMSA. KVM needs to know the GPA of the VMSA that's actively being used so that it can react to MMU invalidation events, i.e. so that KVM can drop the VMSA if its backing guest_memfd page is punched out of existence. Opportunistically rename snp_vmsa_gpa to clarify that it tracks the pending VMSA GPA, whereas snp_guest_vmsa_gpa now tracks the in-use VMSA GPA. Note! Take care to track the GPA, not the GFN, as VALID_PAGE() won't behave correctly if an invalid GFN is converted to a GPA for checking. Note #2! Keep snp_has_guest_vmsa so that switching to a guest-provided VMSA is sticky, even if the guest-provided VMSA becomes invalid. No functional change intended. Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Michael Roth Link: https://patch.msgid.link/20260709204948.1988414-2-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Drop FOLL_WRITE for encrypted region registration

2026-09-02T12:33:18+00:00

commit ee1a586dd1fa2f245b3b753a3e44d9263a49240b upstream. When pinning SEV guest memory, drop FOLL_WRITE and rely on FOLL_LONGTERM to break CoW, as *KVM* doesn't actually to the memory using the GUP'd pages. Omitting FOLL_WRITE fixes a regression when using file-backed guest memory that was introduced when KVM (correctly) added FOLL_LONG (e.g. to ensure anonymous memory is migrated out of MIGRATE_CMA/ZONE_MOVABLE before a long term pin). Unfortunately, as of commits: 8ac268436e6d ("mm/gup: disallow FOLL_LONGTERM GUP-nonfast writing to file-backed mappings") a6e79df92e4a ("mm/gup: disallow FOLL_LONGTERM GUP-fast writing to file-backed mappings") GUP uses FOLL_LONGTERM as a canary of sorts to detect pins that are likely to be problematic, and disallows WRITE+LONGTERM pins for file-backed memory. As a result, backing SEV+ guests with file-backed memory, e.g. virtio-pmem, fails due to the disallowed FOLL_LONGTERM+FOLL_WRITE combination. Note, in the past, FOLL_WRITE was required to trigger CoW unsharing, to prevent replacing the page in the (primary MMU's) page tables during a later write fault after already having pinned a (shared) page in MAP_PRIVATE mappings. FOLL_LONGTERM does that nowadays, even without FOLL_WRITE (see gup_must_unshare()). Fixes: 7e066cb9b71a ("KVM: SEV: Use long-term pin when registering encrypted memory regions") Cc: stable@vger.kernel.org Suggested-by: "David Hildenbrand (Arm)" Link: https://lore.kernel.org/all/ad784f05-b36c-4e91-9f17-4c5b826735d0@kernel.org/ Signed-off-by: Pankaj Gupta Acked-by: David Hildenbrand (Arm) Acked-by: Lorenzo Stoakes (ARM) Link: https://patch.msgid.link/20260715063626.65899-1-pankaj.gupta@amd.com [sean: massage changelog, add comment about CoW unsharing] Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: SEV: Allocate full pages for {DE,EN}CRYPT ops on SNP-enabled hosts

2026-09-02T12:33:18+00:00

commit a33c40b93ccf5177e042253807d40e0b92e7f206 upstream. When {de,en}crypting memory of an SEV or SEV-ES guest on an SNP-enabled host via a temporary buffer, allocate a full 4KiB page for the buffer to ensure the page containing the buffer is wholly owned by KVM, i.e. won't be concurrently allocated and accessed by other kernel code while KVM is using the buffer to {de,en}crypt memory. On SNP-enabled platforms, when sending SEV/SEV-ES commands that trigger firmware writes to memory, the to-be-written page(s) must be (temporarily) assigned to Firmware (as required by the SNP architecture, to guard against using such commands as gadgets to attack SNP guests). See snp_map_cmd_buf_desc() and friends. Unfortunately, transferring ownership of a page to Firmware makes the page inaccessible to software, and thus writes generate RMP #PF violations. If KVM uses a sub-page allocation for its temporary buffer, some other actor in the kernel can allocate and use the other portions of the page, and thus trigger unexpected (and seemingly spurious) RMP #PF violations due to software attempting to access a Firmware-owned page. BUG: unable to handle page fault for address: ffff906ae30f0300 #PF: supervisor write access in kernel mode #PF: error_code(0x80000003) - RMP violation PGD 6b1b80d067 P4D 6b1b80d067 PUD 100231e2063 PMD 10055a88063 PTE 80000100630f0163 SEV-SNP: PFN 0x100630f0 unassigned, dumping non-zero entries in 2M PFN region: [0x10063000 - 0x10063200] Oops: Oops: 0003 [#1] SMP CPU: 70 UID: 0 PID: 10658 Comm: svw_WaiterThrea Tainted: G U W O 7.1.0-smp--c22293789940-seanjc-next #1 PREEMPTLAZY Tainted: [U]=USER, [W]=WARN, [O]=OOT_MODULE Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026 RIP: 0010:memset+0xf/0x20 Call Trace: __kvmalloc_node_noprof+0x2a4/0x710 do_getxattr+0x4e/0x130 path_getxattrat+0x125/0x1b0 do_syscall_64+0x10a/0x480 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7f3a22cb6daa Modules linked in: kvm_amd kvm irqbypass vfat fat ccp k10temp sha3 libsha3 i2c_piix4 gq(O) cdc_acm xhci_pci xhci_hcd gsmi: Log Shutdown Reason 0x03 CR2: ffff906ae30f0300 ---[ end trace 0000000000000000 ]--- RIP: 0010:memset+0xf/0x20 Kernel panic - not syncing: Fatal exception Kernel Offset: 0x39e00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) gsmi: Log Shutdown Reason 0x02 Fixes: 4c735bf1bc22 ("KVM: SEV: Allocate only as many bytes as needed for temp crypt buffers") Cc: stable@vger.kernel.org Cc: Michael Roth Debugged-by: Michael Roth Link: https://patch.msgid.link/20260723211306.75397-1-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: x86/mmu: WARN and clear role.invalid when creating a child shadow page

2026-08-07T07:26:45+00:00

Explicitly clear role.invalid when deriving a child shadow page's role from its parent to harden against bugs elsewhere in KVM, as violating KVM's invariant that invalid pages are NOT on the list of active MMU pages leads to use-after-free due to __kvm_mmu_prepare_zap_page() using list_add() instead of list_move() when processing an invalid shadow page, i.e. makes a bad situation far worse. Yell loudly if the parent is invalid, as it means KVM has missed a validity check, i.e. KVM is attempting to map memory using an invalid/obsolete root, but continue on as the child is otherwise still a valid shadow page. ================================================================== BUG: KASAN: slab-use-after-free in __kvm_mmu_get_shadow_page+0x1817/0x1860 [kvm] Write of size 8 at addr ff11000153dd1368 by task repro/853 CPU: 1 UID: 1000 PID: 853 Comm: repro Not tainted 7.2.0-rc2-3aec122bdcaf-next-vm #5 PREEMPT Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 Call Trace: dump_stack_lvl+0x4b/0x70 print_report+0x153/0x49c kasan_report+0xbc/0xf0 __kvm_mmu_get_shadow_page+0x1817/0x1860 [kvm] mmu_alloc_root+0x141/0x320 [kvm] kvm_mmu_load+0x612/0x20f0 [kvm] kvm_arch_vcpu_ioctl_run+0x3dd5/0x6150 [kvm] kvm_vcpu_ioctl+0x5e4/0x10d0 [kvm] __x64_sys_ioctl+0x131/0x1b0 do_syscall_64+0x67/0x5f0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Allocated by task 853: kasan_save_stack+0x20/0x40 kasan_save_track+0x14/0x30 __kasan_slab_alloc+0x5f/0x70 kmem_cache_alloc_noprof+0xfe/0x2e0 __kvm_mmu_topup_memory_cache+0x135/0x530 [kvm] paging64_page_fault+0x318/0x1e30 [kvm] kvm_mmu_do_page_fault+0x21d/0x630 [kvm] kvm_mmu_page_fault+0x18c/0x17b0 [kvm] kvm_arch_vcpu_ioctl_run+0x1f35/0x6150 [kvm] kvm_vcpu_ioctl+0x5e4/0x10d0 [kvm] __x64_sys_ioctl+0x131/0x1b0 do_syscall_64+0x67/0x5f0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Freed by task 853: kasan_save_stack+0x20/0x40 kasan_save_track+0x14/0x30 kasan_save_free_info+0x3b/0x60 __kasan_slab_free+0x43/0x70 kmem_cache_free+0xe2/0x400 kvm_mmu_commit_zap_page.part.0+0x1e2/0x310 [kvm] kvm_mmu_free_roots+0x283/0x560 [kvm] kvm_arch_vcpu_ioctl_run+0x33c8/0x6150 [kvm] kvm_vcpu_ioctl+0x5e4/0x10d0 [kvm] __x64_sys_ioctl+0x131/0x1b0 do_syscall_64+0x67/0x5f0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Reported-by: Hyunwoo Kim Fixes: a770f6f28b1a ("KVM: MMU: Inherit a shadow page's guest level count from vcpu setup") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Signed-off-by: Paolo Bonzini

KVM: SVM: Serialize accesses to the owner and mirror list with separate lock

2026-08-06T14:50:02+00:00

Interaction between KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM and KVM_CAP_VM_COPY_ENC_CONTEXT_FROM can cause two separate issues: - in sev_migrate_from(), when the destination KVM is a mirror, the mirror entry is moved from the source's list to the owner's mirror_vms list, without holding the owner's lock unlike other writers of the owner's mirror list (sev_vm_copy_enc_context_from(), sev_vm_destroy()). A concurrent COPY or destroy can race with sev_migrate_from() and corrupt the list. - In sev_vm_destroy(), the *owner* is still active and could receive concurrently a KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM that causes sev->enc_context_owner to change. In this case the incorrect VM receives kvm_put_kvm(). The second issue needs particular care because the owner could disappear altogether (even though the race window is impossibly small) between reading it and locking it. There is thus no way to perform the checks under the owner lock without putting struct kvm under SLAB_TYPESAFE_BY_RCU (which would allow kvm_get_kvm_safe() under RCU critical section). It is much simpler to just use a global lock, since the critical sections are so small and the new lock is always a leaf lock. Fixes: b2125513dfc0 ("KVM: SEV: Allow SEV intra-host migration of VM with mirrors") Cc: stable@vger.kernel.org Reported-by: Shen Yongchao Link: https://lore.kernel.org/kvm/tencent_625C0F42824E542C72B34733392AF2C49709@qq.com/ Link: https://lore.kernel.org/kvm/tencent_DDC4E4352EC91CAC05A9A8F4E55E8C96730A@qq.com/ Signed-off-by: Paolo Bonzini

KVM: SVM: make svm_flush_tlb_gva do a full asid flush if NPT enabled

2026-08-06T13:46:46+00:00

Red Hat is seeing multiple reports of Windows memory corruptions (and consequent BSODs) with hv-tlbflush=on, on AMD processors only. The crashes, while extremely rare, happen even with a stock configuration, but with Driver Verifier enabled they can be detected after approximately 200 VM hours. In particular, Alexander Lougovski measured the following: - on AMD Turin, 15 crashes in 3300 VM hours - on AMD Milan, 2 crashes in 500 VM hours (there are fewer hours here due to the host being smaller) - on Intel Sapphire Rapids, 0 crashes in 8000 VM hours - on AMD Turin with full TLB flush (not exactly this patch but similar), no crashes in ~2 weeks of run time which should also be ~7000 VM hours For Turin, the microcode version was 0x0b002162, which (assuming this is the same issue) should not be affected by the problem listed in https://knowledge.broadcom.com/external/article/419026/bsod-on-virtual-machines-running-on-amd.html; on the other hand that problem should not apply to earlier processors. AMD has not provided any information or analysis yet, and when we asked we didn't know yet that it reproduced on Milan as well. As to the workload, Alexander threw more or less everything at the same time at the VM: - a full Windows Defender scan every 30 minutes - a disk I/O job - a loop doing repeated mmap of system files (mostly to hope that it triggers some consistency check in the Windows memory manager) - SQL Express 2022 + StressDB (1.6M rows), with the host doing queries (75% write/25% read) via sqlcmd Driver Verifier is able to detect BSODs more or less at the same time as the pages are freed. They mostly happen in the Windows Defender filter driver, but occasionally also in the networking stack (e.g., afd.sys) or elsewhere in the filesystem stack (e.g., fltmgr.sys). The flush is issued from kvm_hv_vcpu_flush_tlb(), which receives the cross-CPU requests from the Hyper-V TLB flush hypercalls via a kfifo and is invoked by the KVM_REQ_HV_TLB_FLUSH request. The mechanism is the same for both Intel and AMD, and the handler for both vendors is a simple INVVPID(ADDR)/INVLPGA instruction. Because the request is handled on the destination CPU, there is a question of what happens if the VM is migrated across physical CPUs. In that case, the INVLPGA instruction would use a stale svm->vmcb->control.asid; but if anything that might do an *unnecessary* flush (on an asid that's being used for another VM) and then pre_svm_run() would force a full TLB rebuild. So, for lack of better ideas, this patch forces a full ASID bump in svm_flush_tlb_gva(). To avoid paying the price on Intel and also to avoid unnecessary loops on AMD, the flush_tlb_gva op now returns whether it did a full flush or not; kvm_hv_vcpu_flush_tlb() takes note and exits its loops immediately. While there is an obvious performance impact, about half of the benefit from Hyper-V tlbflush is preserved (10% vs. 20% on the SQL Server workload). kvm_mmu_invalidate_addr() is the only other caller of the flush_tlb_gva op. The change would have a performance impact on every intercepted INVLPG and, for nested SVM, on every L1 INVLPGA. For INVLPGA specifically, this covers the same suspected issue but for nested hypervisors, so it is correct to apply the workaround; for INVLPG on shadow paging, instead, the impact would be stronger and, due to lack of data, for now the use of INVLPGA is left in place in svm_flush_tlb_gva(). Analyzed-by: Vitaly Kuznetsov Analyzed-by: Alexander Lougovski Signed-off-by: Paolo Bonzini

KVM: SVM: Update x2APIC MSR intercepts if AVIC is inhibited while L2 is active

2026-07-29T21:39:36+00:00

Always update x2APIC MSR intercepts for L1 when AVIC is deactivated, even if L2 is active and KVM is using a separate MSR bitmap to run L2. If AVIC is fully enabled prior to running L2, and is then inhibited while L2 is active (for a VM-scoped inhibit), then KVM will run L1 with AVIC disabled, but with x2APIC MSR intercepts disabled, i.e. will allow L1 to read most of the host's APIC state, send arbitrary interrupts, change task priority, and ultimately trivially DoS the host. E.g. sending a self-IPI in L1 on HYPERV_REENLIGHTENMENT_VECTOR, 0xee, with CONFIG_HYPERV=n in the host kernel as a "safe" PoC, yields: Spurious interrupt (vector 0xee) on CPU#425. Acked And hacking KVM to abuse kvm_set_posted_intr_wakeup_handler() to register a handler and WARN on POSTED_INTR_WAKEUP_VECTOR yields: ------------[ cut here ]------------ WARNING: arch/x86/kvm/svm/svm.c:5594 at pi_wakeup_handler+0x9/0x10 [kvm_amd], CPU#156: nested_x2apic_t/316940 CPU: 156 UID: 0 PID: 316940 Comm: nested_x2apic_t Tainted: G S U Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER Hardware name: Google Astoria-Turin/astoria, BIOS 0.20260209.0-0 02/09/2026 RIP: 0010:pi_wakeup_handler+0x9/0x10 [kvm_amd] Call Trace: sysvec_kvm_posted_intr_wakeup_ipi+0x64/0x80 asm_sysvec_kvm_posted_intr_wakeup_ipi+0x1a/0x20 RIP: 0010:vcpu_run+0x1430/0x1e40 [kvm] kvm_arch_vcpu_ioctl_run+0x2c1/0x600 [kvm] kvm_vcpu_ioctl+0x580/0x6b0 [kvm] __se_sys_ioctl+0x6d/0xb0 do_syscall_64+0x10a/0x480 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x46ff4b ---[ end trace 0000000000000000 ]--- Fixes: 091abbf578f9 ("KVM: x86: nSVM: optimize svm_set_x2apic_msr_interception") Cc: stable@vger.kernel.org Cc: Yosry Ahmed Signed-off-by: Sean Christopherson Link: https://patch.msgid.link/20260729213558.639074-1-pbonzini@redhat.com/ Signed-off-by: Paolo Bonzini