kernel/linux.git/arch/arm64/include, branch v6.6.145

arm64/mm: convert set_pte_at() to set_ptes(..., 1)

2026-07-24T14:03:26+00:00

[ Upstream commit 659e193027910a5d3083e34b488ab459d2ef5082 ] Since set_ptes() was introduced, set_pte_at() has been implemented as a generic macro around set_ptes(..., 1). So this change should continue to generate the same code. However, making this change prepares us for the transparent contpte support. It means we can reroute set_ptes() to __set_ptes(). Since set_pte_at() is a generic macro, there will be no equivalent __set_pte_at() to reroute to. Note that a couple of calls to set_pte_at() remain in the arch code. This is intentional, since those call sites are acting on behalf of core-mm and should continue to call into the public set_ptes() rather than the arch-private __set_ptes(). Link: https://lkml.kernel.org/r/20240215103205.2607016-9-ryan.roberts@arm.com Signed-off-by: Ryan Roberts Tested-by: John Hubbard Acked-by: Mark Rutland Acked-by: Catalin Marinas Cc: Alistair Popple Cc: Andrey Ryabinin Cc: Ard Biesheuvel Cc: Barry Song <21cnbao@gmail.com> Cc: Borislav Petkov (AMD) Cc: Dave Hansen Cc: David Hildenbrand Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Morse Cc: Kefeng Wang Cc: Marc Zyngier Cc: Matthew Wilcox (Oracle) Cc: Thomas Gleixner Cc: Will Deacon Cc: Yang Shi Cc: Zi Yan Signed-off-by: Andrew Morton Stable-dep-of: ff4c5a0de1f2 ("arm64/mm: Optimize TLB flush in unmap_hotplug_[pmd|pud]_range()") Signed-off-by: Sasha Levin

arm64/mm: convert READ_ONCE(*ptep) to ptep_get(ptep)

2026-07-24T14:03:26+00:00

[ Upstream commit 532736558e8ef2865eae1d84b52dda4422cac810 ] There are a number of places in the arch code that read a pte by using the READ_ONCE() macro. Refactor these call sites to instead use the ptep_get() helper, which itself is a READ_ONCE(). Generated code should be the same. This will benefit us when we shortly introduce the transparent contpte support. In this case, ptep_get() will become more complex so we now have all the code abstracted through it. Link: https://lkml.kernel.org/r/20240215103205.2607016-8-ryan.roberts@arm.com Signed-off-by: Ryan Roberts Tested-by: John Hubbard Acked-by: Mark Rutland Acked-by: Catalin Marinas Cc: Alistair Popple Cc: Andrey Ryabinin Cc: Ard Biesheuvel Cc: Barry Song <21cnbao@gmail.com> Cc: Borislav Petkov (AMD) Cc: Dave Hansen Cc: David Hildenbrand Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Morse Cc: Kefeng Wang Cc: Marc Zyngier Cc: Matthew Wilcox (Oracle) Cc: Thomas Gleixner Cc: Will Deacon Cc: Yang Shi Cc: Zi Yan Signed-off-by: Andrew Morton Stable-dep-of: ff4c5a0de1f2 ("arm64/mm: Optimize TLB flush in unmap_hotplug_[pmd|pud]_range()") Signed-off-by: Sasha Levin

KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

2026-07-24T14:02:56+00:00

commit 8cc8bbbfab14c22c5551d0dd19b208a44b141c76 upstream. flush_hyp_vcpu() copies the host vGIC state into the hyp's private vCPU on every run. The vGIC list register save and restore use used_lrs as their loop bound and expect it to stay within the number of implemented list registers. While this is generally the case, flush_hyp_vcpu() copies vgic_v3 verbatim and does not enforce this, so a value provided by the host is used at EL2 to index vgic_lr[] and access ICH_LR_EL2 (host -> EL2). Fix by clamping used_lrs to the number of implemented list registers after the copy, as the trusted path already does in vgic_flush_lr_state(). The number of implemented list registers is constant after init, so it is replicated once from kvm_vgic_global_state.nr_lr into hyp_gicv3_nr_lr rather than read on every entry. Cc: stable@vger.kernel.org Fixes: be66e67f1750 ("KVM: arm64: Use the pKVM hyp vCPU structure in handle___kvm_vcpu_run()") Signed-off-by: Hyunwoo Kim Reviewed-by: Fuad Tabba Tested-by: Fuad Tabba Link: https://patch.msgid.link/20260606175614.83273-3-imv4bel@gmail.com Signed-off-by: Marc Zyngier [ tabba: adjust context in flush_hyp_vcpu() and kvm_hyp.h ] Signed-off-by: Fuad Tabba Signed-off-by: Sasha Levin

arm64: Revamp HCR_EL2.E2H RES1 detection

2026-07-24T14:02:39+00:00

[ Upstream commit ca88ecdce5f51874a7c151809bd2c936ee0d3805 ] We currently have two ways to identify CPUs that only implement FEAT_VHE and not FEAT_E2H0: - either they advertise it via ID_AA64MMFR4_EL1.E2H0, - or the HCR_EL2.E2H bit is RAO/WI However, there is a third category of "cpus" that fall between these two cases: on CPUs that do not implement FEAT_FGT, it is IMPDEF whether an access to ID_AA64MMFR4_EL1 can trap to EL2 when the register value is zero. A consequence of this is that on systems such as Neoverse V2, a NV guest cannot reliably detect that it is in a VHE-only configuration (E2H is writable, and ID_AA64MMFR0_EL1 is 0), despite the hypervisor's best effort to repaint the id register. Replace the RAO/WI test by a sequence that makes use of the VHE register remnapping between EL1 and EL2 to detect this situation, and work out whether we get the VHE behaviour even after having set HCR_EL2.E2H to 0. This solves the NV problem, and provides a more reliable acid test for CPUs that do not completely follow the letter of the architecture while providing a RES1 behaviour for HCR_EL2.E2H. Suggested-by: Mark Rutland Acked-by: Mark Rutland Acked-by: Catalin Marinas Reviewed-by: Oliver Upton Tested-by: Jan Kotas Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/15A85F2B-1A0C-4FA7-9FE4-EEC2203CC09E@global.cadence.com [ Backport: Resolved conflict in arch/arm64/include/asm/el2_setup.h by replacing msr_hcr_el2 macro usages with raw msr hcr_el2 (since the macro is missing in 6.6.y). ] Signed-off-by: Colton Lewis Signed-off-by: Sasha Levin

KVM: arm64: Initialize SCTLR_EL1 in __kvm_hyp_init_cpu()

2026-07-24T14:02:39+00:00

[ Upstream commit 3855a7b91d42ebf3513b7ccffc44807274978b3d ] When KVM is in protected mode, host calls to PSCI are proxied via EL2, and cold entries from CPU_ON, CPU_SUSPEND, and SYSTEM_SUSPEND bounce through __kvm_hyp_init_cpu() at EL2 before entering the host kernel's entry point at EL1. While __kvm_hyp_init_cpu() initializes SPSR_EL2 for the exception return to EL1, it does not initialize SCTLR_EL1. Due to this, it's possible to enter EL1 with SCTLR_EL1 in an UNKNOWN state. In practice this has been seen to result in kernel crashes after CPU_ON as a result of SCTLR_EL1.M being 1 in violation of the initial core configuration specified by PSCI. Fix this by initializing SCTLR_EL1 for cold entry to the host kernel. As it's necessary to write to SCTLR_EL12 in VHE mode, this initialization is moved into __kvm_host_psci_cpu_entry() where we can use write_sysreg_el1(). The remnants of the '__init_el2_nvhe_prepare_eret' macro are folded into its only caller, as this is clearer than having the macro. Fixes: cdf367192766ad11 ("KVM: arm64: Intercept host's CPU_ON SMCs") Reported-by: Leo Yan Signed-off-by: Ahmed Genidi [ Mark: clarify commit message, handle E2H, move to C, remove macro ] Signed-off-by: Mark Rutland Cc: Ahmed Genidi Cc: Ben Horgan Cc: Catalin Marinas Cc: Leo Yan Cc: Marc Zyngier Cc: Oliver Upton Cc: Will Deacon Reviewed-by: Leo Yan Link: https://lore.kernel.org/r/20250227180526.1204723-3-mark.rutland@arm.com Signed-off-by: Marc Zyngier [ Backport: Resolved context conflicts when removing the __init_el2_nvhe_prepare_eret macro and invocation: - arch/arm64/include/asm/el2_setup.h: conflicted because 6.6.y lacks later GCS/MPAM macros (__init_el2_gcs / __init_el2_mpam) surrounding the definition. - arch/arm64/kvm/hyp/nvhe/hyp-init.S: conflicted because __kvm_init_el2_state does not exist in 6.6.y (EL2 state is initialized inline). ] Signed-off-by: Colton Lewis Signed-off-by: Sasha Levin

KVM: arm64: Initialize HCR_EL2.E2H early

2026-07-24T14:02:39+00:00

[ Upstream commit 7a68b55ff39b0a1638acb1694c185d49f6077a0d ] On CPUs without FEAT_E2H0, HCR_EL2.E2H is RES1, but may reset to an UNKNOWN value out of reset and consequently may not read as 1 unless it has been explicitly initialized. We handled this for the head.S boot code in commits: 3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative") b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented") Unfortunately, we forgot to apply a similar fix to the KVM PSCI entry points used when relaying CPU_ON, CPU_SUSPEND, and SYSTEM SUSPEND. When KVM is entered via these entry points, the value of HCR_EL2.E2H may be consumed before it has been initialized (e.g. by the 'init_el2_state' macro). Initialize HCR_EL2.E2H early in these paths such that it can be consumed reliably. The existing code in head.S is factored out into a new 'init_el2_hcr' macro, and this is used in the __kvm_hyp_init_cpu() function common to all the relevant PSCI entry points. For clarity, I've tweaked the assembly used to check whether ID_AA64MMFR4_EL1.E2H0 is negative. The bitfield is extracted as a signed value, and this is checked with a signed-greater-or-equal (GE) comparison. As the hyp code will reconfigure HCR_EL2 later in ___kvm_hyp_init(), all bits other than E2H are initialized to zero in __kvm_hyp_init_cpu(). Fixes: 3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative") Fixes: b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented") Signed-off-by: Mark Rutland Cc: Ahmed Genidi Cc: Ben Horgan Cc: Catalin Marinas Cc: Leo Yan Cc: Marc Zyngier Cc: Oliver Upton Cc: Will Deacon Link: https://lore.kernel.org/r/20250227180526.1204723-2-mark.rutland@arm.com [maz: fixed LT->GE thinko] Signed-off-by: Marc Zyngier [ Backport: Resolved trivial conflict in arch/arm64/kvm/hyp/nvhe/hyp-init.S caused by __kvm_init_el2_state not existing in 6.6.y (EL2 state is initialized inline via init_el2_state / finalise_el2_state); placed init_el2_hcr 0 directly before the inline sequence. ] Signed-off-by: Colton Lewis Signed-off-by: Sasha Levin

arm64: cputype: Add C1-Premium definitions

2026-06-19T11:39:43+00:00

commit d28413bfc5a255957241f1df5d7fd0c2cd74fe18 upstream. Add cputype definitions for C1-Premium. These will be used for errata detection in subsequent patches. These values can be found in the C1-Premium TRM: https://developer.arm.com/documentation/109416/0100/ ... in section A.5.1 ("MIDR_EL1, Main ID Register"). Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Will Deacon [Mark: backport to v6.6.y] Signed-off-by: Mark Rutland Signed-off-by: Greg Kroah-Hartman

arm64: cputype: Add C1-Ultra definitions

2026-06-19T11:39:43+00:00

commit 60349e64a6c65f9f0aa118af711b3c7e137f07ff upstream. Add cputype definitions for C1-Ultra. These will be used for errata detection in subsequent patches. These values can be found in the C1-Ultra TRM: https://developer.arm.com/documentation/108014/0100/ ... in section A.5.1 ("MIDR_EL1, Main ID Register"). Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Will Deacon [Mark: backport to v6.6.y] Signed-off-by: Mark Rutland Signed-off-by: Greg Kroah-Hartman

arm64: cputype: Add NVIDIA Olympus definitions

2026-06-19T11:39:43+00:00

commit e185c8a0d84236d14af61faff8147c953a878a77 upstream. Add cpu part and model macro definitions for NVIDIA Olympus core. Signed-off-by: Shanker Donthineni Signed-off-by: Will Deacon [Mark: backport to v6.6.y] Signed-off-by: Mark Rutland Signed-off-by: Greg Kroah-Hartman

arm64: tlb: Flush walk cache when unsharing PMD tables

2026-06-19T11:39:39+00:00

[ Upstream commit c2ff4764e03e7a8d758352f4aceb8fe1be6ac971 ] When huge_pmd_unshare() is called to unshare a PMD table, the tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true but the aarch64 tlb_flush() only checked tlb->freed_tables to determine whether to use TLBF_NONE (vae1is, invalidates walk cache) or TLBF_NOWALKCACHE (vale1is, leaf-only). This caused the stale PMD page table entry to remain in the walk cache after unshare, potentially leading to incorrect page table walks. Fix by including unshared_tables in the check, so that when unsharing tables, TLBF_NONE is used and the walk cache is properly invalidated. Here is the detailed distinction between vae1is and vale1is: | Instruction Combination | Actual Invalidation Scope | | ------------------------ | --------------------------------------------------| | `VAE1IS` + TTL=`0` | All entries at all levels (full invalidation) | | `VAE1IS` + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2 | | `VALE1IS` + TTL=`0` | Leaf entries at all levels (non-leaf not cleared) | | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only | Signed-off-by: Zeng Heng Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather") Cc: Signed-off-by: Catalin Marinas Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman