kernel/linux.git/drivers/iommu/amd, branch v6.6.151

iommu/amd: Don't split flush for amd_iommu_domain_flush_all()

2026-08-03T09:15:33+00:00

[ Upstream commit 69fe699afe1afcb730164b86c228483c2da05f94 ] We have observed multiple full invalidations occurring during device detach when we are done using the vfio-device. blocked_domain_attach_device() -> detach_device() -> amd_iommu_domain_flush_all() -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) while (size != 0) { -> __domain_flush_pages( flush_size /* power of 2 flush_size */) -> domain_flush_pages_v1() -> build_inv_iommu_pages() -> build_inv_address() } build_inv_address() will trigger a full invalidation if the chunk size > (1 << 51). Consequently, the guest will issue multiple full invalidations for a single call to amd_iommu_domain_flush_all() Without this patch, we will see 10 time instead of 1 time full invalidations for every amd_iommu_domain_flush_all(). Cc: stable@vger.kernel.org Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") Suggested-by: Josef Bacik Suggested-by: Jason Gunthorpe Signed-off-by: Weinan Liu Reviewed-by: Wei Wang Reviewed-by: Jason Gunthorpe Reviewed-by: Samiullah Khawaja Reviewed-by: Suravee Suthikulpanit Reviewed-by: Vasant Hegde Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

iommu/amd: Bound the early ACPI HID map

2026-08-03T09:15:20+00:00

[ Upstream commit fb80117fddb5b477218dc99bb53911b72c3847f8 ] The ivrs_acpihid command-line parser appends entries to a fixed four-element early_acpihid_map array. Unlike the sibling IOAPIC and HPET parsers, it does not reject a fifth entry before incrementing the map size. Check the capacity at the common found label before parsing the HID and UID or writing the entry. Fixes: ca3bf5d47cec ("iommu/amd: Introduces ivrs_acpihid kernel parameter") Signed-off-by: Pengpeng Hou Reviewed-by: Ankit Soni Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Wait for completion instead of returning early in iommu_completion_wait()

2026-08-03T09:15:19+00:00

[ Upstream commit 1e75a8255f11c81fb07e81e5029cfd75804350a0 ] need_sync is a per-IOMMU flag shared by all domains and devices behind that IOMMU. It is set whenever a command is queued with sync == true and cleared when a completion-wait (CWAIT) command is queued. However, a cleared need_sync only means that a covering CWAIT has been queued, not that all previously queued commands have actually completed in hardware. iommu_completion_wait() read need_sync locklessly and returned early when it was false. This breaks the "block until all previously queued commands have completed" contract in a multi-CPU scenario: CPU2: queue inv-B => need_sync = true CPU1: queue CWAIT(N); need_sync = false; then wait_on_sem(N) CPU2: read need_sync == false => return 0 (no wait!) CPU2 returns without waiting for any sequence number even though its inv-B may not have completed yet (CWAIT(N), queued after inv-B, has not been signaled). CPU2 then proceeds to, for example, free page-table pages while the IOMMU can still walk stale translations, opening a use-after-free window. This is a logical race in the meaning of the flag, not a memory-visibility issue, so barriers alone do not help. Fix it without losing the optimization of avoiding redundant CWAIT commands: take iommu->lock before testing need_sync, and when it is false do not return early but wait for the last allocated sequence number (cmd_sem_val). Since need_sync == false implies no sync command was queued after the last CWAIT, that CWAIT is FIFO-ordered after every not-yet-completed command, so waiting for its sequence number guarantees all prior commands (possibly queued by another CPU) have completed. The common path with pending work is unchanged and no extra hardware command is issued. Signed-off-by: Guanghui Feng Fixes: 815b33fdc279 ("x86/amd-iommu: Cleanup completion-wait handling") Reviewed-by: Vasant Hegde Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Fix a stale comment about which legacy mode is user visible

2026-07-24T14:03:00+00:00

[ Upstream commit 4bf53c2d0c08bbdaa32f2114281f1ddab61902bf ] Update a stale comment about which of the legacy modes is visible to the user, i.e. can be forced via amd_iommu_intr=legacy. Fixes: b74aa02d7a30 ("iommu/amd: Fix legacy interrupt remapping for x2APIC-enabled system") Signed-off-by: Sean Christopherson Reviewed-by: Vasant Hegde Reviewed-by: Wei Wang Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu/amd: serialize sequence allocation under concurrent TLB invalidations

2026-05-17T15:13:35+00:00

commit 9e249c48412828e807afddc21527eb734dc9bd3d upstream. With concurrent TLB invalidations, completion wait randomly gets timed out because cmd_sem_val was incremented outside the IOMMU spinlock, allowing CMD_COMPL_WAIT commands to be queued out of sequence and breaking the ordering assumption in wait_on_sem(). Move the cmd_sem_val increment under iommu->lock so completion sequence allocation is serialized with command queuing. And remove the unnecessary return. Fixes: d2a0cac10597 ("iommu/amd: move wait_on_sem() out of spinlock") Tested-by: Srikanth Aithal Reported-by: Srikanth Aithal Signed-off-by: Ankit Soni Reviewed-by: Vasant Hegde Signed-off-by: Joerg Roedel [Salvatore Bonaccorso: Backport to v6.12.y where f32fe7cb0198 ("iommu/amd: Add support to remap/unmap IOMMU buffers for kdump") is not present] Signed-off-by: Salvatore Bonaccorso Signed-off-by: Sasha Levin

iommu/amd: Use atomic64_inc_return() in iommu.c

2026-05-17T15:13:35+00:00

commit 5ce73c524f5fb5abd7b1bfed0115474b4fb437b4 upstream. Use atomic64_inc_return(&ref) instead of atomic64_add_return(1, &ref) to use optimized implementation and ease register pressure around the primitive for targets that implement optimized variant. Signed-off-by: Uros Bizjak Cc: Joerg Roedel Cc: Suravee Suthikulpanit Cc: Will Deacon Cc: Robin Murphy Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241007084356.47799-1-ubizjak@gmail.com Signed-off-by: Joerg Roedel Signed-off-by: Salvatore Bonaccorso Stable-dep-of: 9e249c48412828e807afddc21527eb734dc9bd3d ("iommu/amd: serialize sequence allocation under concurrent TLB invalidations") Signed-off-by: Sasha Levin

iommu/amd: move wait_on_sem() out of spinlock

2026-03-04T12:20:45+00:00

[ Upstream commit d2a0cac10597068567d336e85fa3cbdbe8ca62bf ] With iommu.strict=1, the existing completion wait path can cause soft lockups under stressed environment, as wait_on_sem() busy-waits under the spinlock with interrupts disabled. Move the completion wait in iommu_completion_wait() out of the spinlock. wait_on_sem() only polls the hardware-updated cmd_sem and does not require iommu->lock, so holding the lock during the busy wait unnecessarily increases contention and extends the time with interrupts disabled. Signed-off-by: Ankit Soni Reviewed-by: Vasant Hegde Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

Revert "iommu/amd: Skip enabling command/event buffers for kdump"

2026-01-11T14:22:33+00:00

This reverts commit 44a764aec64b3f3235b9cbac2525222f69685418 which is commit 9be15fbfc6c5c89c22cf6e209f66ea43ee0e58bb upstream. This causes problems in older kernel trees as SNP host kdump is not supported in them, so drop it from the stable branches. Reported-by: Ashish Kalra Link: https://lore.kernel.org/r/dacdff7f-0606-4ed5-b056-2de564404d51@amd.com Cc: Vasant Hegde Cc: Sairaj Kodilkar Cc: Joerg Roedel Cc: Sasha Levin Signed-off-by: Greg Kroah-Hartman

iommu/amd: Fix pci_segment memleak in alloc_pci_segment()

2026-01-11T14:22:10+00:00

commit 75ba146c2674ba49ed8a222c67f9abfb4a4f2a4f upstream. Fix a memory leak of struct amd_iommu_pci_segment in alloc_pci_segment() when system memory (or contiguous memory) is insufficient. Fixes: 04230c119930 ("iommu/amd: Introduce per PCI segment device table") Fixes: eda797a27795 ("iommu/amd: Introduce per PCI segment rlookup table") Fixes: 99fc4ac3d297 ("iommu/amd: Introduce per PCI segment alias_table") Cc: stable@vger.kernel.org Signed-off-by: Jinhui Guo Signed-off-by: Joerg Roedel Signed-off-by: Greg Kroah-Hartman

iommu/amd: Skip enabling command/event buffers for kdump

2025-11-24T09:29:34+00:00

[ Upstream commit 9be15fbfc6c5c89c22cf6e209f66ea43ee0e58bb ] After a panic if SNP is enabled in the previous kernel then the kdump kernel boots with IOMMU SNP enforcement still enabled. IOMMU command buffers and event buffer registers remain locked and exclusive to the previous kernel. Attempts to enable command and event buffers in the kdump kernel will fail, as hardware ignores writes to the locked MMIO registers as per AMD IOMMU spec Section 2.12.2.1. Skip enabling command buffers and event buffers for kdump boot as they are already enabled in the previous kernel. Reviewed-by: Vasant Hegde Tested-by: Sairaj Kodilkar Signed-off-by: Ashish Kalra Link: https://lore.kernel.org/r/576445eb4f168b467b0fc789079b650ca7c5b037.1756157913.git.ashish.kalra@amd.com Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin