summaryrefslogtreecommitdiff
path: root/drivers/iommu/iommu.c
AgeCommit message (Collapse)AuthorFilesLines
2020-07-29Merge branches 'arm/renesas', 'arm/qcom', 'arm/mediatek', 'arm/omap', ↵Joerg Roedel1-13/+24
'arm/exynos', 'arm/smmu', 'ppc/pamu', 'x86/vt-d', 'x86/amd' and 'core' into next
2020-07-24iommu: Mark __iommu_map_sg() as staticBaolin Wang1-3/+3
Now __iommu_map_sg() is used only in iommu.c file, so mark it static. Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com> Acked-by: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/ab722e9970739929738066777b8ee7930e32abd5.1591930156.git.baolin.wang@linux.alibaba.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-22iommu: Make some functions staticWei Yongjun1-4/+4
The sparse tool complains as follows: drivers/iommu/iommu.c:386:5: warning: symbol 'iommu_insert_resv_region' was not declared. Should it be static? drivers/iommu/iommu.c:2182:5: warning: symbol '__iommu_map' was not declared. Should it be static? Those functions are not used outside of iommu.c, so mark them static. Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Link: https://lore.kernel.org/r/20200713142542.50294-1-weiyongjun1@huawei.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-09iommu: Fix use-after-free in iommu_release_deviceQian Cai1-1/+1
In pci_disable_sriov(), i.e., # echo 0 > /sys/class/net/enp11s0f1np1/device/sriov_numvfs iommu_release_device iommu_group_remove_device arm_smmu_domain_free kfree(smmu_domain) Later, iommu_release_device arm_smmu_release_device arm_smmu_detach_dev spin_lock_irqsave(&smmu_domain->devices_lock, would trigger an use-after-free. Fixed it by call arm_smmu_release_device() first before iommu_group_remove_device(). BUG: KASAN: use-after-free in __lock_acquire+0x3458/0x4440 __lock_acquire at kernel/locking/lockdep.c:4250 Read of size 8 at addr ffff0089df1a6f68 by task bash/3356 CPU: 5 PID: 3356 Comm: bash Not tainted 5.8.0-rc3-next-20200630 #2 Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.11 06/18/2019 Call trace: dump_backtrace+0x0/0x398 show_stack+0x14/0x20 dump_stack+0x140/0x1b8 print_address_description.isra.12+0x54/0x4a8 kasan_report+0x134/0x1b8 __asan_report_load8_noabort+0x2c/0x50 __lock_acquire+0x3458/0x4440 lock_acquire+0x204/0xf10 _raw_spin_lock_irqsave+0xf8/0x180 arm_smmu_detach_dev+0xd8/0x4a0 arm_smmu_detach_dev at drivers/iommu/arm-smmu-v3.c:2776 arm_smmu_release_device+0xb4/0x1c8 arm_smmu_disable_pasid at drivers/iommu/arm-smmu-v3.c:2754 (inlined by) arm_smmu_release_device at drivers/iommu/arm-smmu-v3.c:3000 iommu_release_device+0xc0/0x178 iommu_release_device at drivers/iommu/iommu.c:302 iommu_bus_notifier+0x118/0x160 notifier_call_chain+0xa4/0x128 __blocking_notifier_call_chain+0x70/0xa8 blocking_notifier_call_chain+0x14/0x20 device_del+0x618/0xa00 pci_remove_bus_device+0x108/0x2d8 pci_stop_and_remove_bus_device+0x1c/0x28 pci_iov_remove_virtfn+0x228/0x368 sriov_disable+0x8c/0x348 pci_disable_sriov+0x5c/0x70 mlx5_core_sriov_configure+0xd8/0x260 [mlx5_core] sriov_numvfs_store+0x240/0x318 dev_attr_store+0x38/0x68 sysfs_kf_write+0xdc/0x128 kernfs_fop_write+0x23c/0x448 __vfs_write+0x54/0xe8 vfs_write+0x124/0x3f0 ksys_write+0xe8/0x1b8 __arm64_sys_write+0x68/0x98 do_el0_svc+0x124/0x220 el0_sync_handler+0x260/0x408 el0_sync+0x140/0x180 Allocated by task 3356: save_stack+0x24/0x50 __kasan_kmalloc.isra.13+0xc4/0xe0 kasan_kmalloc+0xc/0x18 kmem_cache_alloc_trace+0x1ec/0x318 arm_smmu_domain_alloc+0x54/0x148 iommu_group_alloc_default_domain+0xc0/0x440 iommu_probe_device+0x1c0/0x308 iort_iommu_configure+0x434/0x518 acpi_dma_configure+0xf0/0x128 pci_dma_configure+0x114/0x160 really_probe+0x124/0x6d8 driver_probe_device+0xc4/0x180 __device_attach_driver+0x184/0x1e8 bus_for_each_drv+0x114/0x1a0 __device_attach+0x19c/0x2a8 device_attach+0x10/0x18 pci_bus_add_device+0x70/0xf8 pci_iov_add_virtfn+0x7b4/0xb40 sriov_enable+0x5c8/0xc30 pci_enable_sriov+0x64/0x80 mlx5_core_sriov_configure+0x58/0x260 [mlx5_core] sriov_numvfs_store+0x1c0/0x318 dev_attr_store+0x38/0x68 sysfs_kf_write+0xdc/0x128 kernfs_fop_write+0x23c/0x448 __vfs_write+0x54/0xe8 vfs_write+0x124/0x3f0 ksys_write+0xe8/0x1b8 __arm64_sys_write+0x68/0x98 do_el0_svc+0x124/0x220 el0_sync_handler+0x260/0x408 el0_sync+0x140/0x180 Freed by task 3356: save_stack+0x24/0x50 __kasan_slab_free+0x124/0x198 kasan_slab_free+0x10/0x18 slab_free_freelist_hook+0x110/0x298 kfree+0x128/0x668 arm_smmu_domain_free+0xf4/0x1a0 iommu_group_release+0xec/0x160 kobject_put+0xf4/0x238 kobject_del+0x110/0x190 kobject_put+0x1e4/0x238 iommu_group_remove_device+0x394/0x938 iommu_release_device+0x9c/0x178 iommu_release_device at drivers/iommu/iommu.c:300 iommu_bus_notifier+0x118/0x160 notifier_call_chain+0xa4/0x128 __blocking_notifier_call_chain+0x70/0xa8 blocking_notifier_call_chain+0x14/0x20 device_del+0x618/0xa00 pci_remove_bus_device+0x108/0x2d8 pci_stop_and_remove_bus_device+0x1c/0x28 pci_iov_remove_virtfn+0x228/0x368 sriov_disable+0x8c/0x348 pci_disable_sriov+0x5c/0x70 mlx5_core_sriov_configure+0xd8/0x260 [mlx5_core] sriov_numvfs_store+0x240/0x318 dev_attr_store+0x38/0x68 sysfs_kf_write+0xdc/0x128 kernfs_fop_write+0x23c/0x448 __vfs_write+0x54/0xe8 vfs_write+0x124/0x3f0 ksys_write+0xe8/0x1b8 __arm64_sys_write+0x68/0x98 do_el0_svc+0x124/0x220 el0_sync_handler+0x260/0x408 el0_sync+0x140/0x180 The buggy address belongs to the object at ffff0089df1a6e00 which belongs to the cache kmalloc-512 of size 512 The buggy address is located 360 bytes inside of 512-byte region [ffff0089df1a6e00, ffff0089df1a7000) The buggy address belongs to the page: page:ffffffe02257c680 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff0089df1a1400 flags: 0x7ffff800000200(slab) raw: 007ffff800000200 ffffffe02246b8c8 ffffffe02257ff88 ffff000000320680 raw: ffff0089df1a1400 00000000002a000e 00000001ffffffff ffff0089df1a5001 page dumped because: kasan: bad access detected page->mem_cgroup:ffff0089df1a5001 Memory state around the buggy address: ffff0089df1a6e00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff0089df1a6e80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >ffff0089df1a6f00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff0089df1a6f80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff0089df1a7000: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc Fixes: a6a4c7e2c5b8 ("iommu: Add probe_device() and release_device() call-backs") Signed-off-by: Qian Cai <cai@lca.pw> Link: https://lore.kernel.org/r/20200704001003.2303-1-cai@lca.pw Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-06-30iommu: Allow page responses without PASIDJean-Philippe Brucker1-6/+17
Some PCIe devices do not expect a PASID value in PRI Page Responses. If the "PRG Response PASID Required" bit in the PRI capability is zero, then the OS should not set the PASID field. Similarly on Arm SMMU, responses to stall events do not have a PASID. Currently iommu_page_response() systematically checks that the PASID in the page response corresponds to the one in the page request. This can't work with virtualization because a page response coming from a guest OS won't have a PASID if the passed-through device does not require one. Add a flag to page requests that declares whether the corresponding response needs to have a PASID. When this flag isn't set, allow page responses without PASID. Reported-by: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Link: https://lore.kernel.org/r/20200616144712.748818-1-jean-philippe@linaro.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-06-04iommu: Check for deferred attach in iommu_group_do_dma_attach()Joerg Roedel1-1/+5
The iommu_group_do_dma_attach() must not attach devices which have deferred_attach set. Otherwise devices could cause IOMMU faults when re-initialized in a kdump kernel. Fixes: deac0b3bed26 ("iommu: Split off default domain allocation from group assignment") Reported-by: Jerry Snitselaar <jsnitsel@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Jerry Snitselaar <jsnitsel@redhat.com> Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com> Link: https://lore.kernel.org/r/20200604091944.26402-1-joro@8bytes.org
2020-06-02Merge branches 'arm/msm', 'arm/allwinner', 'arm/smmu', 'x86/vt-d', ↵Joerg Roedel1-148/+318
'hyper-v', 'core' and 'x86/amd' into next
2020-05-29Merge tag 'v5.7-rc7' into x86/amdJoerg Roedel1-6/+11
Linux 5.7-rc7
2020-05-29iommu: Fix reference count leak in iommu_group_alloc.Qiushi Wu1-1/+1
kobject_init_and_add() takes reference even when it fails. Thus, when kobject_init_and_add() returns an error, kobject_put() must be called to properly clean up the kobject. Fixes: d72e31c93746 ("iommu: IOMMU Groups") Signed-off-by: Qiushi Wu <wu000273@umn.edu> Link: https://lore.kernel.org/r/20200527210020.6522-1-wu000273@umn.edu Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-29iommu: Remove iommu_sva_ops::mm_exit()Jean-Philippe Brucker1-11/+0
After binding a device to an mm, device drivers currently need to register a mm_exit handler. This function is called when the mm exits, to gracefully stop DMA targeting the address space and flush page faults to the IOMMU. This is deemed too complex for the MMU release() notifier, which may be triggered by any mmput() invocation, from about 120 callsites [1]. The upcoming SVA module has an example of such complexity: the I/O Page Fault handler would need to call mmput_async() instead of mmput() after handling an IOPF, to avoid triggering the release() notifier which would in turn drain the IOPF queue and lock up. Another concern is the DMA stop function taking too long, up to several minutes [2]. For some mmput() callers this may disturb other users. For example, if the OOM killer picks the mm bound to a device as the victim and that mm's memory is locked, if the release() takes too long, it might choose additional innocent victims to kill. To simplify the MMU release notifier, don't forward the notification to device drivers. Since they don't stop DMA on mm exit anymore, the PASID lifetime is extended: (1) The device driver calls bind(). A PASID is allocated. Here any DMA fault is handled by mm, and on error we don't print anything to dmesg. Userspace can easily trigger errors by issuing DMA on unmapped buffers. (2) exit_mmap(), for example the process took a SIGKILL. This step doesn't happen during normal operations. Remove the pgd from the PASID table, since the page tables are about to be freed. Invalidate the IOTLBs. Here the device may still perform DMA on the address space. Incoming transactions are aborted but faults aren't printed out. ATS Translation Requests return Successful Translation Completions with R=W=0. PRI Page Requests return with Invalid Request. (3) The device driver stops DMA, possibly following release of a fd, and calls unbind(). PASID table is cleared, IOTLB invalidated if necessary. The page fault queues are drained, and the PASID is freed. If DMA for that PASID is still running here, something went seriously wrong and errors should be reported. For now remove iommu_sva_ops entirely. We might need to re-introduce them at some point, for example to notify device drivers of unhandled IOPF. [1] https://lore.kernel.org/linux-iommu/20200306174239.GM31668@ziepe.ca/ [2] https://lore.kernel.org/linux-iommu/4d68da96-0ad5-b412-5987-2f7a6aa796c3@amd.com/ Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Acked-by: Jacob Pan <jacob.jun.pan@linux.intel.com> Acked-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20200423125329.782066-3-jean-philippe@linaro.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-25iommu: Don't take group reference in iommu_alloc_default_domain()Joerg Roedel1-12/+9
The iommu_alloc_default_domain() function takes a reference to an IOMMU group without releasing it. This causes the group to never be released, with undefined side effects. The function has only one call-site, which takes a group reference on its own, so to fix this leak, do not take another reference in iommu_alloc_default_domain() and pass the group as a function parameter instead. Fixes: 6e1aa2049154 ("iommu: Move default domain allocation to iommu_probe_device()") Reported-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Cc: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Link: https://lore.kernel.org/r/20200525130122.380-1-joro@8bytes.org Reference: https://lore.kernel.org/lkml/20200522130145.30067-1-saiprakash.ranjan@codeaurora.org/
2020-05-25iommu: Don't call .probe_finalize() under group->mutexJoerg Roedel1-10/+18
The .probe_finalize() call-back of some IOMMU drivers calls into arm_iommu_attach_device(). This function will call back into the IOMMU core code, where it tries to take group->mutex again, resulting in a deadlock. As there is no reason why .probe_finalize() needs to be called under that mutex, move it after the lock has been released to fix the deadlock. Fixes: deac0b3bed26 ("iommu: Split off default domain allocation from group assignment") Reported-by: Yong Wu <yong.wu@mediatek.com> Tested-by: Yong Wu <yong.wu@mediatek.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Cc: Yong Wu <yong.wu@mediatek.com> Link: https://lore.kernel.org/r/20200519132824.15163-1-joro@8bytes.org
2020-05-19iommu: Fix deferred domain attachmentJoerg Roedel1-6/+11
The IOMMU core code has support for deferring the attachment of a domain to a device. This is needed in kdump kernels where the new domain must not be attached to a device before the device driver takes it over. When the AMD IOMMU driver got converted to use the dma-iommu implementation, the deferred attaching got lost. The code in dma-iommu.c has support for deferred attaching, but it calls into iommu_attach_device() to actually do it. But iommu_attach_device() will check if the device should be deferred in it code-path and do nothing, breaking deferred attachment. Move the is_deferred_attach() check out of the attach_device path and into iommu_group_add_device() to make deferred attaching work from the dma-iommu code. Fixes: 795bbbb9b6f8 ("iommu/dma-iommu: Handle deferred devices") Reported-by: Jerry Snitselaar <jsnitsel@redhat.com> Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Jerry Snitselaar <jsnitsel@redhat.com> Cc: Jerry Snitselaar <jsnitsel@redhat.com> Cc: Tom Murphy <murphyt7@tcd.ie> Cc: Robin Murphy <robin.murphy@arm.com> Link: https://lore.kernel.org/r/20200519130340.14564-1-joro@8bytes.org
2020-05-15iommu: Remove functions that support private domainSai Praneeth Prakhya1-65/+0
After moving iommu_group setup to iommu core code [1][2] and removing private domain support in vt-d [3], there are no users for functions such as iommu_request_dm_for_dev(), iommu_request_dma_domain_for_dev() and request_default_domain_for_dev(). So, remove these functions. [1] commit dce8d6964ebd ("iommu/amd: Convert to probe/release_device() call-backs") [2] commit e5d1841f18b2 ("iommu/vt-d: Convert to probe/release_device() call-backs") [3] commit 327d5b2fee91 ("iommu/vt-d: Allow 32bit devices to uses DMA domain") Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20200513224721.20504-1-sai.praneeth.prakhya@intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-13Merge tag 'v5.7-rc4' into coreJoerg Roedel1-0/+1
Linux 5.7-rc4
2020-05-13iommu: Do not probe devices on IOMMU-less bussesThierry Reding1-0/+3
The host1x bus implemented on Tegra SoCs is primarily an abstraction to create logical device from multiple platform devices. Since the devices in such a setup are typically hierarchical, DMA setup still needs to be done so that DMA masks can be properly inherited, but we don't actually want to attach the host1x logical devices to any IOMMU. The platform devices that make up the logical device are responsible for memory bus transactions, so it is them that will need to be attached to the IOMMU. Add a check to __iommu_probe_device() that aborts IOMMU setup early for busses that don't have the IOMMU operations pointer set since they will cause a crash otherwise. Signed-off-by: Thierry Reding <treding@nvidia.com> Link: https://lore.kernel.org/r/20200511161000.3853342-1-thierry.reding@gmail.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Unexport iommu_group_get_for_dev()Joerg Roedel1-2/+2
The function is now only used in IOMMU core code and shouldn't be used outside of it anyway, so remove the export for it. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-35-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Move more initialization to __iommu_probe_device()Joerg Roedel1-29/+18
Move the calls to dev_iommu_get() and try_module_get() into __iommu_probe_device(), so that the callers don't have to do it on their own. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-34-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Remove add_device()/remove_device() code-pathsJoerg Roedel1-120/+38
All drivers are converted to use the probe/release_device() call-backs, so the add_device/remove_device() pointers are unused and the code using them can be removed. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-33-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Export bus_iommu_probe() and make is safe for re-probingJoerg Roedel1-1/+9
Add a check to the bus_iommu_probe() call-path to make sure it ignores devices which have already been successfully probed. Then export the bus_iommu_probe() function so it can be used by IOMMU drivers. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-14-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Move iommu_group_create_direct_mappings() out of iommu_group_add_device()Joerg Roedel1-5/+30
After the previous changes the iommu group may not have a default domain when iommu_group_add_device() is called. With no default domain iommu_group_create_direct_mappings() will do nothing and no direct mappings will be created. Rename iommu_group_create_direct_mappings() to iommu_create_device_direct_mappings() to better reflect that the function creates direct mappings only for one device and not for all devices in the group. Then move the call to the places where a default domain actually exists. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-13-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Split off default domain allocation from group assignmentJoerg Roedel1-3/+151
When a bus is initialized with iommu-ops, all devices on the bus are scanned and iommu-groups are allocated for them, and each groups will also get a default domain allocated. Until now this happened as soon as the group was created and the first device added to it. When other devices with different default domain requirements were added to the group later on, the default domain was re-allocated, if possible. This resulted in some back and forth and unnecessary allocations, so change the flow to defer default domain allocation until all devices have been added to their respective IOMMU groups. The default domains are allocated for newly allocated groups after each device on the bus is handled and was probed by the IOMMU driver. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-12-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Move new probe_device path to separate functionJoerg Roedel1-23/+46
This makes it easier to remove to old code-path when all drivers are converted. As a side effect that it also fixes the error cleanup path. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-11-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Keep a list of allocated groups in __iommu_probe_device()Joerg Roedel1-2/+7
This is needed to defer default_domain allocation for new IOMMU groups until all devices have been added to the group. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-10-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Move default domain allocation to iommu_probe_device()Joerg Roedel1-31/+71
Well, not really. The call to iommu_alloc_default_domain() in iommu_group_get_for_dev() has to stay around as long as there are IOMMU drivers using the add/remove_device() call-backs instead of probe/release_device(). Those drivers expect that iommu_group_get_for_dev() returns the device attached to a group and the group set up with a default domain (and the device attached to the groups current domain). But when all drivers are converted this compatability mess can be removed. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-9-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Add probe_device() and release_device() call-backsJoerg Roedel1-6/+57
Add call-backs to 'struct iommu_ops' as an alternative to the add_device() and remove_device() call-backs, which will be removed when all drivers are converted. The new call-backs will not setup IOMMU groups and domains anymore, so also add a probe_finalize() call-back where the IOMMU driver can do per-device setup work which require the device to be set up with a group and a domain. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-8-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Add def_domain_type() callback in iommu_opsSai Praneeth Prakhya1-3/+17
Some devices are reqired to use a specific type (identity or dma) of default domain when they are used with a vendor iommu. When the system level default domain type is different from it, the vendor iommu driver has to request a new default domain with iommu_request_dma_domain_for_dev() and iommu_request_dm_for_dev() in the add_dev() callback. Unfortunately, these two helpers only work when the group hasn't been assigned to any other devices, hence, some vendor iommu driver has to use a private domain if it fails to request a new default one. This adds def_domain_type() callback in the iommu_ops, so that any special requirement of default domain for a device could be aware by the iommu generic layer. Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> [ jroedel@suse.de: Added iommu_get_def_domain_type() function and use it to allocate the default domain ] Co-developed-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-3-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05iommu: Move default domain allocation to separate functionJoerg Roedel1-29/+45
Move the code out of iommu_group_get_for_dev() into a separate function. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com> Link: https://lore.kernel.org/r/20200429133712.31431-2-joro@8bytes.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-01iommu: Properly export iommu_group_get_for_dev()Greg Kroah-Hartman1-1/+1
In commit a7ba5c3d008d ("drivers/iommu: Export core IOMMU API symbols to permit modular drivers") a bunch of iommu symbols were exported, all with _GPL markings except iommu_group_get_for_dev(). That export should also be _GPL like the others. Fixes: a7ba5c3d008d ("drivers/iommu: Export core IOMMU API symbols to permit modular drivers") Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Will Deacon <will@kernel.org> Cc: Joerg Roedel <jroedel@suse.de> Cc: John Garry <john.garry@huawei.com> Cc: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20200430120120.2948448-1-gregkh@linuxfoundation.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-04-29iommu: Fix the memory leak in dev_iommu_free()Kevin Hao1-0/+1
In iommu_probe_device(), we would invoke dev_iommu_free() to free the dev->iommu after the ->add_device() returns failure. But after commit 72acd9df18f1 ("iommu: Move iommu_fwspec to struct dev_iommu"), we also need to free the iommu_fwspec before the dev->iommu is freed. This fixes the following memory leak reported by kmemleak: unreferenced object 0xffff000bc836c700 (size 128): comm "swapper/0", pid 1, jiffies 4294896304 (age 782.120s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 d8 cd 9b ff 0b 00 ff ff ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000df34077b>] kmem_cache_alloc_trace+0x244/0x4b0 [<000000000e560ac0>] iommu_fwspec_init+0x7c/0xb0 [<0000000075eda275>] of_iommu_xlate+0x80/0xe8 [<00000000728d6bf9>] of_pci_iommu_init+0xb0/0xb8 [<00000000d001fe6f>] pci_for_each_dma_alias+0x48/0x190 [<000000006db6bbce>] of_iommu_configure+0x1ac/0x1d0 [<00000000634745f8>] of_dma_configure+0xdc/0x220 [<000000002cbc8ba0>] pci_dma_configure+0x50/0x78 [<00000000cdf6e193>] really_probe+0x8c/0x340 [<00000000fddddc46>] driver_probe_device+0x60/0xf8 [<0000000061bcdb51>] __device_attach_driver+0x8c/0xd0 [<000000009b9ff58e>] bus_for_each_drv+0x80/0xd0 [<000000004b9c8aa3>] __device_attach+0xec/0x148 [<00000000a5c13bf3>] device_attach+0x1c/0x28 [<000000005071e151>] pci_bus_add_device+0x58/0xd0 [<000000002d4f87d1>] pci_bus_add_devices+0x40/0x90 Fixes: 72acd9df18f1 ("iommu: Move iommu_fwspec to struct dev_iommu") Signed-off-by: Kevin Hao <haokexin@gmail.com> Link: https://lore.kernel.org/r/20200402143749.40500-1-haokexin@gmail.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-03-27iommu: Move iommu_fwspec to struct dev_iommuJoerg Roedel1-0/+3
Move the iommu_fwspec pointer in struct device into struct dev_iommu. This is a step in the effort to reduce the iommu related pointers in struct device to one. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Will Deacon <will@kernel.org> # arm-smmu Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Link: https://lore.kernel.org/r/20200326150841.10083-7-joro@8bytes.org
2020-03-27iommu: Rename struct iommu_param to dev_iommuJoerg Roedel1-14/+14
The term dev_iommu aligns better with other existing structures and their accessor functions. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Will Deacon <will@kernel.org> # arm-smmu Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Link: https://lore.kernel.org/r/20200326150841.10083-6-joro@8bytes.org
2020-02-28iommu: Use C99 flexible array in fwspecRobin Murphy1-7/+8
Although the 1-element array was a typical pre-C99 way to implement variable-length structures, and indeed is a fundamental construct in the APIs of certain other popular platforms, there's no good reason for it here (and in particular the sizeof() trick is far too "clever" for its own good). We can just as easily implement iommu_fwspec's preallocation behaviour using a standard flexible array member, so let's make it look the way most readers would expect. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24Merge branches 'iommu/fixes', 'arm/smmu', 'x86/amd', 'x86/vt-d' and 'core' ↵Joerg Roedel1-2/+49
into next
2020-01-07iommu: Remove device link to group on failureJon Derrick1-0/+1
This adds the missing teardown step that removes the device link from the group when the device addition fails. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Fixes: 797a8b4d768c5 ("iommu: Handle default domain attach failure") Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-23iommu: Implement generic_iommu_put_resv_regions()Thierry Reding1-0/+19
Implement a generic function for removing reserved regions. This can be used by drivers that don't do anything fancy with these regions other than allocating memory for them. Signed-off-by: Thierry Reding <treding@nvidia.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-23drivers/iommu: Allow IOMMU bus ops to be unregisteredWill Deacon1-0/+5
'bus_set_iommu()' allows IOMMU drivers to register their ops for a given bus type. Unfortunately, it then doesn't allow them to be removed, which is necessary for modular drivers to shutdown cleanly so that they can be reloaded later on. Allow 'bus_set_iommu()' to take a NULL 'ops' argument, which clear the ops pointer for the selected bus_type. Signed-off-by: Will Deacon <will@kernel.org> Tested-by: John Garry <john.garry@huawei.com> # smmu v3 Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-23drivers/iommu: Take a ref to the IOMMU driver prior to ->add_device()Will Deacon1-2/+17
To avoid accidental removal of an active IOMMU driver module, take a reference to the driver module in 'iommu_probe_device()' immediately prior to invoking the '->add_device()' callback and hold it until the after the device has been removed by '->remove_device()'. Suggested-by: Joerg Roedel <joro@8bytes.org> Signed-off-by: Will Deacon <will@kernel.org> Tested-by: John Garry <john.garry@huawei.com> # smmu v3 Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-23drivers/iommu: Export core IOMMU API symbols to permit modular driversWill Deacon1-0/+8
Building IOMMU drivers as modules requires that the core IOMMU API symbols are exported as GPL symbols. Signed-off-by: Will Deacon <will@kernel.org> Tested-by: John Garry <john.garry@huawei.com> # smmu v3 Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-20Merge tag 'iommu-fixes-v5.5-rc2' of ↵Linus Torvalds1-2/+2
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull iommu fixes from Joerg Roedel: - Fix kmemleak warning in IOVA code - Fix compile warnings on ARM32/64 in dma-iommu code due to dma_mask type mismatches - Make ISA reserved regions relaxable, so that VFIO can assign devices which have such regions defined - Fix mapping errors resulting in IO page-faults in the VT-d driver - Make sure direct mappings for a domain are created after the default domain is updated - Map ISA reserved regions in the VT-d driver with correct permissions - Remove unneeded check for PSI capability in the IOTLB flush code of the VT-d driver - Lockdep fix iommu_dma_prepare_msi() * tag 'iommu-fixes-v5.5-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/dma: Relax locking in iommu_dma_prepare_msi() iommu/vt-d: Remove incorrect PSI capability check iommu/vt-d: Allocate reserved region for ISA with correct permission iommu: set group default domain before creating direct mappings iommu/vt-d: Fix dmar pte read access not set error iommu/vt-d: Set ISA bridge reserved region as relaxable iommu/dma: Rationalise types for DMA masks iommu/iova: Init the struct iova to fix the possible memleak
2019-12-17iommu: set group default domain before creating direct mappingsJerry Snitselaar1-2/+2
iommu_group_create_direct_mappings uses group->default_domain, but right after it is called, request_default_domain_for_dev calls iommu_domain_free for the default domain, and sets the group default domain to a different domain. Move the iommu_group_create_direct_mappings call to after the group default domain is set, so the direct mappings get associated with that domain. Cc: Joerg Roedel <jroedel@suse.de> Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: iommu@lists.linux-foundation.org Cc: stable@vger.kernel.org Fixes: 7423e01741dd ("iommu: Add API to request DMA domain for device") Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-16iommu: fix KASAN use-after-free in iommu_insert_resv_regionEric Auger1-2/+2
In case the new region gets merged into another one, the nr list node is freed. Checking its type while completing the merge algorithm leads to a use-after-free. Use new->type instead. Fixes: 4dbd258ff63e ("iommu: Revisit iommu_insert_resv_region() implementation") Signed-off-by: Eric Auger <eric.auger@redhat.com> Reported-by: Qian Cai <cai@lca.pw> Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com> Cc: Stable <stable@vger.kernel.org> #v5.3+ Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-11-12Merge branches 'iommu/fixes', 'arm/qcom', 'arm/renesas', 'arm/rockchip', ↵Joerg Roedel1-6/+67
'arm/mediatek', 'arm/tegra', 'arm/smmu', 'x86/amd', 'x86/vt-d', 'virtio' and 'core' into next
2019-10-15iommu: Introduce guest PASID bind functionJacob Pan1-0/+20
Guest shared virtual address (SVA) may require host to shadow guest PASID tables. Guest PASID can also be allocated from the host via enlightened interfaces. In this case, guest needs to bind the guest mm, i.e. cr3 in guest physical address to the actual PASID table in the host IOMMU. Nesting will be turned on such that guest virtual address can go through a two level translation: - 1st level translates GVA to GPA - 2nd level translates GPA to HPA This patch introduces APIs to bind guest PASID data to the assigned device entry in the physical IOMMU. See the diagram below for usage explanation. .-------------. .---------------------------. | vIOMMU | | Guest process mm, FL only | | | '---------------------------' .----------------/ | PASID Entry |--- PASID cache flush - '-------------' | | | V | | GP '-------------' Guest ------| Shadow |----------------------- GP->HP* --------- v v | Host v .-------------. .----------------------. | pIOMMU | | Bind FL for GVA-GPA | | | '----------------------' .----------------/ | | PASID Entry | V (Nested xlate) '----------------\.---------------------. | | |Set SL to GPA-HPA | | | '---------------------' '-------------' Where: - FL = First level/stage one page tables - SL = Second level/stage two page tables - GP = Guest PASID - HP = Host PASID * Conversion needed if non-identity GP-HP mapping option is chosen. Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.com> Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-10-15iommu: Introduce cache_invalidate APIYi L Liu1-0/+10
In any virtualization use case, when the first translation stage is "owned" by the guest OS, the host IOMMU driver has no knowledge of caching structure updates unless the guest invalidation activities are trapped by the virtualizer and passed down to the host. Since the invalidation data can be obtained from user space and will be written into physical IOMMU, we must allow security check at various layers. Therefore, generic invalidation data format are proposed here, model specific IOMMU drivers need to convert them into their own format. Signed-off-by: Yi L Liu <yi.l.liu@intel.com> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.com> Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-10-15iommu: Add gfp parameter to iommu_ops::mapTom Murphy1-6/+37
Add a gfp_t parameter to the iommu_ops::map function. Remove the needless locking in the AMD iommu driver. The iommu_ops::map function (or the iommu_map function which calls it) was always supposed to be sleepable (according to Joerg's comment in this thread: https://lore.kernel.org/patchwork/patch/977520/ ) and so should probably have had a "might_sleep()" since it was written. However currently the dma-iommu api can call iommu_map in an atomic context, which it shouldn't do. This doesn't cause any problems because any iommu driver which uses the dma-iommu api uses gfp_atomic in it's iommu_ops::map function. But doing this wastes the memory allocators atomic pools. Signed-off-by: Tom Murphy <murphyt7@tcd.ie> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-09-03iommu: Don't use sme_active() in generic codeJoerg Roedel1-2/+2
Switch to the generic function mem_encrypt_active() because sme_active() is x86 specific and can't be called from generic code on other platforms than x86. Fixes: 2cc13bb4f59f ("iommu: Disable passthrough mode when SME is active") Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-08-30iommu: Remove wrong default domain commentsTom Murphy1-3/+1
These comments are wrong. request_default_domain_for_dev doesn't just handle direct mapped domains. Signed-off-by: Tom Murphy <murphyt7@tcd.ie> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-08-30iommu: Revisit iommu_insert_resv_region() implementationEric Auger1-49/+47
Current implementation is recursive and in case of allocation failure the existing @regions list is altered. A non recursive version looks better for maintainability and simplifies the error handling. We use a separate stack for overlapping segment merging. The elements are sorted by start address and then by type, if their start address match. Note this new implementation may change the region order of appearance in /sys/kernel/iommu_groups/<n>/reserved_regions files but this order has never been documented, see commit bc7d12b91bd3 ("iommu: Implement reserved_regions iommu-group sysfs file"). Signed-off-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-08-23iommu: Disable passthrough mode when SME is activeJoerg Roedel1-0/+5
Using Passthrough mode when SME is active causes certain devices to use the SWIOTLB bounce buffer. The bounce buffer code has an upper limit of 256kb for the size of DMA allocations, which is too small for certain devices and causes them to fail. With this patch we enable IOMMU by default when SME is active in the system, making the default configuration work for more systems than it does now. Users that don't want IOMMUs to be enabled still can disable them with kernel parameters. Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Tested-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
generated by cgit v1.2.3 (git 2.39.0) at 2025-01-29 20:01:34 +0300