kernel/linux.git/drivers/iommu/amd, branch v7.1.6

iommu/amd: Bound the early ACPI HID map

2026-08-03T09:25:31+00:00

[ Upstream commit fb80117fddb5b477218dc99bb53911b72c3847f8 ] The ivrs_acpihid command-line parser appends entries to a fixed four-element early_acpihid_map array. Unlike the sibling IOAPIC and HPET parsers, it does not reject a fifth entry before incrementing the map size. Check the capacity at the common found label before parsing the HID and UID or writing the entry. Fixes: ca3bf5d47cec ("iommu/amd: Introduces ivrs_acpihid kernel parameter") Signed-off-by: Pengpeng Hou Reviewed-by: Ankit Soni Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Wait for completion instead of returning early in iommu_completion_wait()

2026-08-03T09:25:31+00:00

[ Upstream commit 1e75a8255f11c81fb07e81e5029cfd75804350a0 ] need_sync is a per-IOMMU flag shared by all domains and devices behind that IOMMU. It is set whenever a command is queued with sync == true and cleared when a completion-wait (CWAIT) command is queued. However, a cleared need_sync only means that a covering CWAIT has been queued, not that all previously queued commands have actually completed in hardware. iommu_completion_wait() read need_sync locklessly and returned early when it was false. This breaks the "block until all previously queued commands have completed" contract in a multi-CPU scenario: CPU2: queue inv-B => need_sync = true CPU1: queue CWAIT(N); need_sync = false; then wait_on_sem(N) CPU2: read need_sync == false => return 0 (no wait!) CPU2 returns without waiting for any sequence number even though its inv-B may not have completed yet (CWAIT(N), queued after inv-B, has not been signaled). CPU2 then proceeds to, for example, free page-table pages while the IOMMU can still walk stale translations, opening a use-after-free window. This is a logical race in the meaning of the flag, not a memory-visibility issue, so barriers alone do not help. Fix it without losing the optimization of avoiding redundant CWAIT commands: take iommu->lock before testing need_sync, and when it is false do not return early but wait for the last allocated sequence number (cmd_sem_val). Since need_sync == false implies no sync command was queued after the last CWAIT, that CWAIT is FIFO-ordered after every not-yet-completed command, so waiting for its sequence number guarantees all prior commands (possibly queued by another CPU) have completed. The common path with pending work is unchanged and no extra hardware command is issued. Signed-off-by: Guanghui Feng Fixes: 815b33fdc279 ("x86/amd-iommu: Cleanup completion-wait handling") Reviewed-by: Vasant Hegde Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Fix nested domain leak

2026-08-03T09:25:28+00:00

[ Upstream commit 283c5c4c34b4c8d1ebd038d8f360c5ba7fcc767b ] A couple of runs of different AI tools have generated something like the following bug report: In nested_domain_free(), when refcount_dec_and_test() returns false (other nested domains still reference the same gdom_info), the function returns without calling kfree(ndom), leaking the nested_domain structure. This problem wasn't introduced by this patch, but exists in the code from commit 757d2b1fdf5b that the patch modifies. Each nested_domain (ndom) is allocated individually in amd_iommu_alloc_domain_nested() via kzalloc_obj(*ndom). The .free callback is the sole point responsible for freeing this domain. When the refcount is > 0, only the xa_unlock_irqrestore is performed and the function returns, leaving ndom permanently allocated. This leak occurs every time a nested domain sharing a gDomID is destroyed while other domains still use that gDomID. There is a similar leak later in this function in the WARN_ON() test when the mapping is already NULL. Switch to a RAII-based cleanup for ndom, since it should always be freed in this function. Fixes: 757d2b1fdf5b ("iommu/amd: Introduce gDomID-to-hDomID Mapping and handle parent domain invalidation") Signed-off-by: Tycho Andersen (AMD) Reviewed-by: Ankit Soni Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Fix IRQ unsafe locking in gdom allocation

2026-08-03T09:25:28+00:00

[ Upstream commit 0db3a430d9681fdb29890bef6934cd89cd1745d0 ] Lockdep complains: [ 259.410489] ===================================================== [ 259.417287] WARNING: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected [ 259.424667] 7.0.0-g51db1d8d2113 #54 Not tainted [ 259.429718] ----------------------------------------------------- [ 259.436516] qemu-system-x86/10143 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: [ 259.444670] ff3b2b1c60305170 (&xa->xa_lock#25){+.+.}-{3:3}, at: __domain_flush_pages+0x17c/0x4b0 [ 259.454485] and this task is already holding: [ 259.460991] ff3b2b1c98504cc0 (&domain->lock){-.-.}-{3:3}, at: amd_iommu_iotlb_sync+0x25/0x60 [ 259.470408] which would create a new lock dependency: [ 259.476041] (&domain->lock){-.-.}-{3:3} -> (&xa->xa_lock#25){+.+.}-{3:3} [ 259.483615] but this new dependency connects a HARDIRQ-irq-safe lock: [ 259.492447] (&domain->lock){-.-.}-{3:3} [ 259.492449] ... which became HARDIRQ-irq-safe at: [ 259.503705] lock_acquire+0xb6/0x2e0 [ 259.507790] _raw_spin_lock_irqsave+0x3e/0x60 [ 259.512748] amd_iommu_flush_iotlb_all+0x20/0x50 [ 259.517996] iommu_dma_free_iova.isra.0+0x1b8/0x1e0 [ 259.523534] __iommu_dma_unmap+0xc2/0x140 [ 259.528100] iommu_dma_unmap_phys+0x55/0xc0 [ 259.532863] dma_unmap_phys+0x274/0x2e0 [ 259.537238] dma_unmap_page_attrs+0x17/0x30 [ 259.542000] nvme_unmap_data+0x13e/0x280 [ 259.546473] nvme_pci_complete_batch+0x45/0x70 [ 259.551524] nvme_irq+0x83/0x90 [ 259.555123] __handle_irq_event_percpu+0x92/0x360 [ 259.560466] handle_irq_event+0x39/0x80 [ 259.564841] handle_edge_irq+0xb2/0x1a0 [ 259.569214] __common_interrupt+0x4e/0x130 [ 259.573882] common_interrupt+0x88/0xa0 [ 259.578256] asm_common_interrupt+0x27/0x40 [ 259.583019] cpuidle_enter_state+0x119/0x5d0 [ 259.587877] cpuidle_enter+0x2e/0x50 [ 259.591962] do_idle+0x153/0x2c0 [ 259.595657] cpu_startup_entry+0x29/0x30 [ 259.600128] start_secondary+0x118/0x150 [ 259.604601] common_startup_64+0x13e/0x141 [ 259.609266] to a HARDIRQ-irq-unsafe lock: [ 259.615384] (&xa->xa_lock#25){+.+.}-{3:3} [ 259.615386] ... which became HARDIRQ-irq-unsafe at: [ 259.627039] ... [ 259.627039] lock_acquire+0xb6/0x2e0 [ 259.633071] _raw_spin_lock+0x2f/0x50 [ 259.637250] amd_iommu_alloc_domain_nested+0x140/0x3c0 [ 259.643078] iommufd_hwpt_alloc+0x272/0x800 [iommufd] [ 259.648813] iommufd_fops_ioctl+0x14e/0x200 [iommufd] [ 259.654547] __x64_sys_ioctl+0x9d/0xf0 ... Since amd_iommu_domain_flush_pages() necessarily holds domain->lock to do the flush, switch the allocation side in gdom_info_load_or_alloc_locked() to HARDIRQ-safe allocation. The IOMMU_DESTROY->free path has the same issue, so switch that path to HARDIRQ-safe locking as well. Fixes: 757d2b1fdf5b ("iommu/amd: Introduce gDomID-to-hDomID Mapping and handle parent domain invalidation") Signed-off-by: Tycho Andersen (AMD) Reviewed-by: Ankit Soni Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Fix premature break in init_iommu_one()

2026-07-24T14:19:22+00:00

[ Upstream commit 283d245468a2b61c41aa8b582f25ed5615d1c304 ] In init_iommu_one(), when processing IOMMU EFR attributes, the code checks whether GASUP is enabled. If GASUP is not enabled, the code falls back to legacy guest IR mode and then breaks out of the switch statement. This break incorrectly skips the subsequent initialization steps that follow the GASUP check. These initializations are independent of GASUP support and must always be performed. Fix this by replacing the early break with a conditional else block, ensuring that the XTSUP check is only skipped when GASUP is not available. Fixes: a44092e326d4 ("iommu/amd: Use IVHD EFR for early initialization of IOMMU features") Reported-by: Sudheer Dantuluri Tested-by: Dheeraj Kumar Srivastava Signed-off-by: Vasant Hegde Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu/amd: Fix a stale comment about which legacy mode is user visible

2026-07-24T14:19:11+00:00

[ Upstream commit 4bf53c2d0c08bbdaa32f2114281f1ddab61902bf ] Update a stale comment about which of the legacy modes is visible to the user, i.e. can be forced via amd_iommu_intr=legacy. Fixes: b74aa02d7a30 ("iommu/amd: Fix legacy interrupt remapping for x2APIC-enabled system") Signed-off-by: Sean Christopherson Reviewed-by: Vasant Hegde Reviewed-by: Wei Wang Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu/amd: Don't split flush for amd_iommu_domain_flush_all()

2026-07-18T14:55:45+00:00

commit 69fe699afe1afcb730164b86c228483c2da05f94 upstream. We have observed multiple full invalidations occurring during device detach when we are done using the vfio-device. blocked_domain_attach_device() -> detach_device() -> amd_iommu_domain_flush_all() -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) while (size != 0) { -> __domain_flush_pages( flush_size /* power of 2 flush_size */) -> domain_flush_pages_v1() -> build_inv_iommu_pages() -> build_inv_address() } build_inv_address() will trigger a full invalidation if the chunk size > (1 << 51). Consequently, the guest will issue multiple full invalidations for a single call to amd_iommu_domain_flush_all() Without this patch, we will see 10 time instead of 1 time full invalidations for every amd_iommu_domain_flush_all(). Cc: stable@vger.kernel.org Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") Suggested-by: Josef Bacik Suggested-by: Jason Gunthorpe Signed-off-by: Weinan Liu Reviewed-by: Wei Wang Reviewed-by: Jason Gunthorpe Reviewed-by: Samiullah Khawaja Reviewed-by: Suravee Suthikulpanit Reviewed-by: Vasant Hegde Signed-off-by: Joerg Roedel Signed-off-by: Greg Kroah-Hartman

iommu/amd: Bounds-check devid in __rlookup_amd_iommu()

2026-05-11T08:07:52+00:00

iommu_device_register() walks every device on the PCI bus via bus_for_each_dev() and calls amd_iommu_probe_device() for each. The inlined check_device() path computes the device's sbdf, calls rlookup_amd_iommu() to find the owning IOMMU, and only afterwards verifies devid <= pci_seg->last_bdf. __rlookup_amd_iommu() indexes rlookup_table[devid] with no bounds check of its own, so for a PCI device whose BDF is not described by the IVRS, the lookup reads past the end of the allocation before the caller's bounds check can run. This was harmless before commit e874c666b15b ("iommu/amd: Change rlookup, irq_lookup, and alias to use kvalloc()"): the table was a zeroed page-order allocation, so the over-read returned NULL and the caller's NULL check skipped the device. After that commit the table is a tight kvcalloc() and the over-read returns adjacent slab contents, which check_device() then dereferences as a struct amd_iommu *, causing a boot-time GPF. Seen on Google Compute Engine ct6e VMs, where the virtualized IVRS describes only the four TPU endpoints 00:04.0-07.0; the gVNIC at 00:08.0 (devid 0x40) indexes 56 bytes past the 456-byte allocation, into the adjacent kmalloc-512 slab object: pci 0000:00:04.0: Adding to iommu group 0 pci 0000:00:05.0: Adding to iommu group 1 pci 0000:00:06.0: Adding to iommu group 2 pci 0000:00:07.0: Adding to iommu group 3 Oops: general protection fault, probably for non-canonical address 0x3a64695f78746382: 0000 [#1] SMP NOPTI CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.18.22 #1 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/06/2025 RIP: 0010:amd_iommu_probe_device+0x54/0x3a0 Call Trace: __iommu_probe_device+0x107/0x520 probe_iommu_group+0x29/0x50 bus_for_each_dev+0x7e/0xe0 iommu_device_register+0xc9/0x240 iommu_go_to_state+0x9c0/0x1c60 amd_iommu_init+0x14/0x40 pci_iommu_init+0x16/0x60 do_one_initcall+0x47/0x2f0 Guard the array access in __rlookup_amd_iommu(). With the fix applied on 6.18.22, the gVNIC at 00:08.0 is skipped cleanly and the VM boots. Fixes: e874c666b15b ("iommu/amd: Change rlookup, irq_lookup, and alias to use kvalloc()") Cc: stable@vger.kernel.org Reported-by: Ziyuan Chen Tested-by: Ziyuan Chen Reviewed-by: Josef Bacik Assisted-by: Claude:unspecified Signed-off-by: Jose Fernandez (Anthropic) Signed-off-by: Joerg Roedel

iommu/amd: Remove latent out-of-bounds access in IOMMU debugfs

2026-05-11T07:52:54+00:00

In iommu_mmio_write() and iommu_capability_write(), the variables dbg_mmio_offset and dbg_cap_offset are declared as int. However, they are populated using kstrtou32_from_user(). If a user provides a sufficiently large value, it can become a negative integer. Prior to this patch, the AMD IOMMU debugfs implementation was already protected by different mechanisms. 1. #define OFS_IN_SZ 8 ensures the user string <= 8 bytes, so e.g. 0xffffffff isn't a valid input. if (cnt > OFS_IN_SZ) return -EINVAL; 2. Implicit type promotion in iommu_mmio_write(), dbg_mmio_offset is int and iommu->mmio_phys_end is u64 if (dbg_mmio_offset > iommu->mmio_phys_end - sizeof(u64)) return -EINVAL; 3. The show handlers would currently catch the negative number and refuse to perform the read. Replace kstrtou32_from_user() with kstrtos32_from_user() to parse the input, and check for negative values to explicitly prevent out-of-bounds memory accesses directly in iommu_mmio_write() and iommu_capability_write(). Signed-off-by: Eder Zulian Fixes: 7a4ee419e8c1 ("iommu/amd: Add debugfs support to dump IOMMU MMIO registers") Cc: stable@vger.kernel.org Signed-off-by: Joerg Roedel

iommu/amd: Fix precedence order in set_dte_passthrough()

2026-05-04T08:26:16+00:00

Bitwise OR | operator has a higher precedence than the ternary ?: operatior. It will be incorrectly evaluated as: new->data[1] |= (FIELD_PREP(...) | dev_data->ats_enabled) ? DTE_FLAG_IOTLB : 0; Wrap the conditional operation in parentheses to enforce the correct evaluation order. Fixes: 93eee2a49c1b ("iommu/amd: Refactor logic to program the host page table in DTE") Signed-off-by: Weinan Liu Reviewed-by: Jason Gunthorpe Reviewed-by: Vasant Hegde Signed-off-by: Joerg Roedel