kernel/linux.git/drivers/iommu/amd, branch v6.12.91

iommu/amd: Put list_add/del(dev_data) back under the domain->lock

2026-05-23T11:04:58+00:00

[ Upstream commit 4a552f7890f0870f6d9fd4fbc6c05cea7bfd4503 ] The list domain->dev_list is protected by the domain->lock spinlock. Any iteration, addition or removal must be under the lock. Move the list_del() up into the critical section. pdom_is_sva_capable(), and destroy_gcr3_table() do not interact with the list element. Wrap the list_add() in a lock, it would make more sense if this was under the same critical section as adjusting the refcounts earlier, but that requires more complications. Fixes: d6b47dec3684 ("iommu/amd: Reduce domain lock scope in attach device path") Signed-off-by: Jason Gunthorpe Reviewed-by: Vasant Hegde Link: https://lore.kernel.org/r/1-v1-3b9edcf8067d+3975-amd_dev_list_locking_jgg@nvidia.com Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu/amd: Reorder attach device code

2026-05-23T11:04:58+00:00

[ Upstream commit 0b136493d3ffa1358783dcf5b9f866ceef2ff122 ] Ideally in attach device path, it should take dev_data lock before making changes to device data including IOPF enablement. So far dev_data was using spinlock and it was hitting lock order issue when it tries to enable IOPF. Hence Commit 526606b0a199 ("iommu/amd: Fix Invalid wait context issue") moved IOPF enablement outside dev_data->lock. Previous patch converted dev_data lock to mutex. Now its safe to call amd_iommu_iopf_add_device() with dev_data->mutex. Hence move back PCI device capability enablement (ATS, PRI, PASID) and IOPF enablement code inside the lock. Also in attach_device(), update 'dev_data->domain' at the end so that error handling becomes simple. Signed-off-by: Vasant Hegde Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241030063556.6104-11-vasant.hegde@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: 4a552f7890f0 ("iommu/amd: Put list_add/del(dev_data) back under the domain->lock") Signed-off-by: Sasha Levin

iommu/amd: Fix clone_alias() to use the original device's devid

2026-05-23T11:04:33+00:00

[ Upstream commit faad224fe0f0857a04ff2eb3c90f0de57f47d0f3 ] Currently clone_alias() assumes first argument (pdev) is always the original device pointer. This function is called by pci_for_each_dma_alias() which based on topology decides to send original or alias device details in first argument. This meant that the source devid used to look up and copy the DTE may be incorrect, leading to wrong or stale DTE entries being propagated to alias device. Fix this by passing the original pdev as the opaque data argument to both the direct clone_alias() call and pci_for_each_dma_alias(). Inside clone_alias(), retrieve the original device from data and compute devid from it. Fixes: 3332364e4ebc ("iommu/amd: Support multiple PCI DMA aliases in device table") Signed-off-by: Vasant Hegde Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin

iommu/amd: Introduce helper function get_dte256()

2026-05-23T11:04:33+00:00

[ Upstream commit a2ce608a1eb65c2af99c58b63eae557165a0da87 ] And use it in clone_alias() along with update_dte256(). Also use get_dte256() in dump_dte_entry(). Reviewed-by: Jason Gunthorpe Signed-off-by: Suravee Suthikulpanit Link: https://lore.kernel.org/r/20241118054937.5203-7-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin

iommu/amd: Introduce helper function to update 256-bit DTE

2026-05-23T11:04:33+00:00

[ Upstream commit 8b3f78733814b180089a400743b6f19d118aec62 ] The current implementation does not follow 128-bit write requirement to update DTE as specified in the AMD I/O Virtualization Techonology (IOMMU) Specification. Therefore, modify the struct dev_table_entry to contain union of u128 data array, and introduce a helper functions update_dte256() to update DTE using two 128-bit cmpxchg operations to update 256-bit DTE with the modified structure, and take into account the DTE[V, GV] bits when programming the DTE to ensure proper order of DTE programming and flushing. In addition, introduce a per-DTE spin_lock struct dev_data.dte_lock to provide synchronization when updating the DTE to prevent cmpxchg128 failure. Suggested-by: Jason Gunthorpe Suggested-by: Uros Bizjak Reviewed-by: Jason Gunthorpe Reviewed-by: Uros Bizjak Signed-off-by: Suravee Suthikulpanit Link: https://lore.kernel.org/r/20241118054937.5203-5-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin

iommu/amd: Convert dev_data lock from spinlock to mutex

2026-05-23T11:04:33+00:00

[ Upstream commit e843aedbeb82b17a5fe6172449bff133fc8b68a1 ] Currently in attach device path it takes dev_data->spinlock. But as per design attach device path can sleep. Also if device is PRI capable then it adds device to IOMMU fault handler queue which takes mutex. Hence currently PRI enablement is done outside dev_data lock. Covert dev_data lock from spinlock to mutex so that it follows the design and also PRI enablement can be done properly. Signed-off-by: Vasant Hegde Reviewed-by: Joerg Roedel Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241030063556.6104-10-vasant.hegde@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin

iommu/amd: Rearrange attach device code

2026-05-23T11:04:33+00:00

[ Upstream commit 4b18ef8491b06e353e8801705092cc292582cb7a ] attach_device() is just holding lock and calling do_attach(). There is not need to have another function. Just move do_attach() code to attach_device(). Similarly move do_detach() code to detach_device(). Signed-off-by: Vasant Hegde Reviewed-by: Joerg Roedel Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241030063556.6104-9-vasant.hegde@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin

iommu/amd: Reduce domain lock scope in attach device path

2026-05-23T11:04:33+00:00

[ Upstream commit d6b47dec368400a62d2b9d44c8e136fc15eac72c ] Currently attach device path takes protection domain lock followed by dev_data lock. Most of the operations in this function is specific to device data except pdom_attach_iommu() where it updates protection domain structure. Hence reduce the scope of protection domain lock. Note that this changes the locking order. Now it takes device lock before taking doamin lock (group->mutex -> dev_data->lock -> pdom->lock). dev_data->lock is used only in device attachment path. So changing order is fine. It will not create any issue. Finally move numa node assignment to pdom_attach_iommu(). Signed-off-by: Vasant Hegde Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241030063556.6104-8-vasant.hegde@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin

iommu/amd: Do not detach devices in domain free path

2026-05-23T11:04:33+00:00

[ Upstream commit 07bbd660dbd6ff03907d9ddbdfe9deabbd18ac4d ] All devices attached to a protection domain must be freed before calling domain free. Hence do not try to free devices in domain free path. Continue to throw warning if pdom->dev_list is not empty so that any potential issues can be fixed. Signed-off-by: Vasant Hegde Reviewed-by: Joerg Roedel Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241030063556.6104-7-vasant.hegde@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin

iommu/amd: xarray to track protection_domain->iommu list

2026-05-23T11:04:33+00:00

[ Upstream commit d16041124de1dea4389b5e6b330657f34f8c0492 ] Use xarray to track IOMMU attached to protection domain instead of static array of MAX_IOMMUS. Also add lockdep assertion. Signed-off-by: Vasant Hegde Reviewed-by: Joerg Roedel Reviewed-by: Jason Gunthorpe Link: https://lore.kernel.org/r/20241030063556.6104-5-vasant.hegde@amd.com Signed-off-by: Joerg Roedel Stable-dep-of: faad224fe0f0 ("iommu/amd: Fix clone_alias() to use the original device's devid") Signed-off-by: Sasha Levin