kernel/linux.git/drivers/iommu, branch v6.1.186

iommu/amd: Don't split flush for amd_iommu_domain_flush_all()

2026-08-23T12:18:20+00:00

[ Upstream commit 69fe699afe1afcb730164b86c228483c2da05f94 ] We have observed multiple full invalidations occurring during device detach when we are done using the vfio-device. blocked_domain_attach_device() -> detach_device() -> amd_iommu_domain_flush_all() -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) while (size != 0) { -> __domain_flush_pages( flush_size /* power of 2 flush_size */) -> domain_flush_pages_v1() -> build_inv_iommu_pages() -> build_inv_address() } build_inv_address() will trigger a full invalidation if the chunk size > (1 << 51). Consequently, the guest will issue multiple full invalidations for a single call to amd_iommu_domain_flush_all() Without this patch, we will see 10 time instead of 1 time full invalidations for every amd_iommu_domain_flush_all(). Cc: stable@vger.kernel.org Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") Suggested-by: Josef Bacik Suggested-by: Jason Gunthorpe Signed-off-by: Weinan Liu Reviewed-by: Wei Wang Reviewed-by: Jason Gunthorpe Reviewed-by: Samiullah Khawaja Reviewed-by: Suravee Suthikulpanit Reviewed-by: Vasant Hegde Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

iommu/vt-d: Disallow SVA if page walk is not coherent

2026-08-19T15:16:08+00:00

commit 780dfed688622ea01be3c9c2c55eec2207f05e04 upstream. Hardware implementations report Scalable-Mode Page-walk Coherency Support via the SMPWCS field in the extended capability register. If the hardware does not support page-walk coherency, a clflush is required every time the page table entries (which are walked by the IOMMU hardware) are updated. In the SVA case, page tables are managed by the CPU mm core, not by the IOMMU driver. Because the IOMMU driver has no way of knowing whether the CPU page table management code has ensured coherency via clflush, the driver must deny SVA if the hardware does not support coherent paging. Fixes: ff3dc6521f78 ("iommu/vt-d: Fix CPU and IOMMU SVM feature matching checks") Cc: stable@vger.kernel.org Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Reviewed-by: Samiullah Khawaja Reviewed-by: Jason Gunthorpe Signed-off-by: Will Deacon Signed-off-by: Greg Kroah-Hartman

iommu/intel: Fix out-of-bounds memset in dmar_latency_disable()

2026-08-19T15:16:00+00:00

[ Upstream commit 754f8efe45f87e3a9c6871b645b2f9d46d1b407b ] dmar_latency_disable() intends to zero out only the single latency_statistic entry for the given type, but the memset size was computed as sizeof(*lstat) * DMAR_LATENCY_NUM, which clears the entire array starting from &lstat[type]. When type > 0, this writes beyond the end of the allocated array, corrupting adjacent memory. Fix by using sizeof(*lstat) to clear only the target entry. Fixes: 55ee5e67a59a ("iommu/vt-d: Add common code for dmar latency performance monitors") Signed-off-by: Li RongQing Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Bound the early ACPI HID map

2026-08-19T15:16:00+00:00

[ Upstream commit fb80117fddb5b477218dc99bb53911b72c3847f8 ] The ivrs_acpihid command-line parser appends entries to a fixed four-element early_acpihid_map array. Unlike the sibling IOAPIC and HPET parsers, it does not reject a fifth entry before incrementing the map size. Check the capacity at the common found label before parsing the HID and UID or writing the entry. Fixes: ca3bf5d47cec ("iommu/amd: Introduces ivrs_acpihid kernel parameter") Signed-off-by: Pengpeng Hou Reviewed-by: Ankit Soni Signed-off-by: Will Deacon Signed-off-by: Sasha Levin

iommu/amd: Fix a stale comment about which legacy mode is user visible

2026-07-24T13:54:34+00:00

[ Upstream commit 4bf53c2d0c08bbdaa32f2114281f1ddab61902bf ] Update a stale comment about which of the legacy modes is visible to the user, i.e. can be forced via amd_iommu_intr=legacy. Fixes: b74aa02d7a30 ("iommu/amd: Fix legacy interrupt remapping for x2APIC-enabled system") Signed-off-by: Sean Christopherson Reviewed-by: Vasant Hegde Reviewed-by: Wei Wang Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu, debugobjects: avoid gcc-16.1 section mismatch warnings

2026-06-19T11:37:02+00:00

commit 4c9ad387aa2d6785299722e54224d34764edaeb3 upstream. gcc-16 has gained some more advanced inter-procedual optimization techniques that enable it to inline the dummy_tlb_add_page() and dummy_tlb_flush() function pointers into a specialized version of __arm_v7s_unmap: WARNING: modpost: vmlinux: section mismatch in reference: __arm_v7s_unmap+0x2cc (section: .text) -> dummy_tlb_add_page (section: .init.text) ERROR: modpost: Section mismatches detected. >From what I can tell, the transformation is correct, as this is only called when __arm_v7s_unmap() is called from arm_v7s_do_selftests(), which is also __init. Since __arm_v7s_unmap() however is not __init, gcc cannot inline the inner function calls directly. In debug_objects_selftest(), the same thing happens. Both the caller and the leaf function are __init, but the IPA pulls it into a non-init one: WARNING: modpost: vmlinux: section mismatch in reference: lookup_object_or_alloc+0x7c (section: .text.lookup_object_or_alloc) -> is_static_object (section: .init.text) Marking the affected functions as not "__init" would reliably avoid this issue but is not a good solution because it removes an otherwise correct annotation. I tried marking the functions as 'noinline', but that ended up not covering all the affected configurations. With some more experimenting, I found that marking these functions as __attribute__((noipa)) is both logical and reliable. In order to keep the syntax readable, add a custom macro for this in include/linux/compiler_attributes.h next to other related macros and use it to annotate both files. Link: https://lore.kernel.org/all/abRB6g-48ZX6Yl2r@willie-the-truck/ Cc: Will Deacon Cc: Thomas Gleixner Cc: Andrew Morton Cc: Miguel Ojeda Cc: linux-kbuild@vger.kernel.org Cc: stable@vger.kernel.org Signed-off-by: Arnd Bergmann Acked-by: Will Deacon Acked-by: Thomas Gleixner Acked-by: Miguel Ojeda Signed-off-by: Joerg Roedel Signed-off-by: Greg Kroah-Hartman

iommu/vt-d: Disable DMAR for Intel Q35 IGFX

2026-06-01T15:39:33+00:00

commit 2cda2e10dc8343ae01eae9e999a876b7e7d37861 upstream. Intel Q35 integrated graphics (8086:29b2) exhibits broken DMAR behaviour similar to other G4x/GM45 devices for which DMAR is already disabled via quirks. When DMAR is enabled, the system may hard lock up during boot or early device initialization, requiring a reset. Add the missing PCI ID to the existing quirk list to disable DMAR for this device. Fixes: 1f76249cc3be ("iommu/vt-d: Declare Broadwell igfx dmar support snafu") Cc: stable@vger.kernel.org Closes: https://bugzilla.kernel.org/show_bug.cgi?id=201185 Closes: https://bugzilla.kernel.org/show_bug.cgi?id=216064 Signed-off-by: Naval Alcalá Link: https://lore.kernel.org/r/20260410161622.13549-1-ari@naval.cat Signed-off-by: Lu Baolu Signed-off-by: Joerg Roedel Signed-off-by: Greg Kroah-Hartman

iommu/vt-d: Fix intel iommu iotlb sync hardlockup and retry

2026-03-25T10:03:13+00:00

commit fe89277c9ceb0d6af0aa665bcf24a41d8b1b79cd upstream. During the qi_check_fault process after an IOMMU ITE event, requests at odd-numbered positions in the queue are set to QI_ABORT, only satisfying single-request submissions. However, qi_submit_sync now supports multiple simultaneous submissions, and can't guarantee that the wait_desc will be at an odd-numbered position. Therefore, if an item times out, IOMMU can't re-initiate the request, resulting in an infinite polling wait. This modifies the process by setting the status of all requests already fetched by IOMMU and recorded as QI_IN_USE status (including wait_desc requests) to QI_ABORT, thus enabling multiple requests to be resubmitted. Fixes: 8a1d82462540 ("iommu/vt-d: Multiple descriptors per qi_submit_sync()") Cc: stable@vger.kernel.org Signed-off-by: Guanghui Feng Tested-by: Shuai Xue Reviewed-by: Shuai Xue Reviewed-by: Samiullah Khawaja Link: https://lore.kernel.org/r/20260306101516.3885775-1-guanghuifeng@linux.alibaba.com Signed-off-by: Lu Baolu Fixes: 8a1d82462540 ("iommu/vt-d: Multiple descriptors per qi_submit_sync()") Signed-off-by: Joerg Roedel Signed-off-by: Greg Kroah-Hartman

iommu/vt-d: Flush dev-IOTLB only when PCIe device is accessible in scalable mode

2026-03-04T12:20:48+00:00

[ Upstream commit 10e60d87813989e20eac1f3eda30b3bae461e7f9 ] Commit 4fc82cd907ac ("iommu/vt-d: Don't issue ATS Invalidation request when device is disconnected") relies on pci_dev_is_disconnected() to skip ATS invalidation for safely-removed devices, but it does not cover link-down caused by faults, which can still hard-lock the system. For example, if a VM fails to connect to the PCIe device, "virsh destroy" is executed to release resources and isolate the fault, but a hard-lockup occurs while releasing the group fd. Call Trace: qi_submit_sync qi_flush_dev_iotlb intel_pasid_tear_down_entry device_block_translation blocking_domain_attach_dev __iommu_attach_device __iommu_device_set_domain __iommu_group_set_domain_internal iommu_detach_group vfio_iommu_type1_detach_group vfio_group_detach_container vfio_group_fops_release __fput Although pci_device_is_present() is slower than pci_dev_is_disconnected(), it still takes only ~70 µs on a ConnectX-5 (8 GT/s, x2) and becomes even faster as PCIe speed and width increase. Besides, devtlb_invalidation_with_pasid() is called only in the paths below, which are far less frequent than memory map/unmap. 1. mm-struct release 2. {attach,release}_dev 3. set/remove PASID 4. dirty-tracking setup The gain in system stability far outweighs the negligible cost of using pci_device_is_present() instead of pci_dev_is_disconnected() to decide when to skip ATS invalidation, especially under GDR high-load conditions. Fixes: 4fc82cd907ac ("iommu/vt-d: Don't issue ATS Invalidation request when device is disconnected") Cc: stable@vger.kernel.org Signed-off-by: Jinhui Guo Link: https://lore.kernel.org/r/20251211035946.2071-3-guojinhui.liam@bytedance.com Signed-off-by: Lu Baolu Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu/arm-smmu-v3: Improve CMDQ lock fairness and efficiency

2026-03-04T12:20:26+00:00

[ Upstream commit df180b1a4cc51011c5f8c52c7ec02ad2e42962de ] The SMMU CMDQ lock is highly contentious when there are multiple CPUs issuing commands and the queue is nearly full. The lock has the following states: - 0: Unlocked - >0: Shared lock held with count - INT_MIN+N: Exclusive lock held, where N is the # of shared waiters - INT_MIN: Exclusive lock held, no shared waiters When multiple CPUs are polling for space in the queue, they attempt to grab the exclusive lock to update the cons pointer from the hardware. If they fail to get the lock, they will spin until either the cons pointer is updated by another CPU. The current code allows the possibility of shared lock starvation if there is a constant stream of CPUs trying to grab the exclusive lock. This leads to severe latency issues and soft lockups. Consider the following scenario where CPU1's attempt to acquire the shared lock is starved by CPU2 and CPU0 contending for the exclusive lock. CPU0 (exclusive) | CPU1 (shared) | CPU2 (exclusive) | `cmdq->lock` -------------------------------------------------------------------------- trylock() //takes | | | 0 | shared_lock() | | INT_MIN | fetch_inc() | | INT_MIN | no return | | INT_MIN + 1 | spins // VAL >= 0 | | INT_MIN + 1 unlock() | spins... | | INT_MIN + 1 set_release(0) | spins... | | 0 see[NOTE] (done) | (sees 0) | trylock() // takes | 0 | *exits loop* | cmpxchg(0, INT_MIN) | 0 | | *cuts in* | INT_MIN | cmpxchg(0, 1) | | INT_MIN | fails // != 0 | | INT_MIN | spins // VAL >= 0 | | INT_MIN | *starved* | | INT_MIN [NOTE] The current code resets the exclusive lock to 0 regardless of the state of the lock. This causes two problems: 1. It opens the possibility of back-to-back exclusive locks and the downstream effect of starving shared lock. 2. The count of shared lock waiters are lost. To mitigate this, we release the exclusive lock by only clearing the sign bit while retaining the shared lock waiter count as a way to avoid starving the shared lock waiters. Also deleted cmpxchg loop while trying to acquire the shared lock as it is not needed. The waiters can see the positive lock count and proceed immediately after the exclusive lock is released. Exclusive lock is not starved in that submitters will try exclusive lock first when new spaces become available. Reviewed-by: Mostafa Saleh Reviewed-by: Nicolin Chen Signed-off-by: Alexander Grest Signed-off-by: Jacob Pan Signed-off-by: Will Deacon Signed-off-by: Sasha Levin