summaryrefslogtreecommitdiff
path: root/drivers/iommu/ipmmu-vmsa.c
AgeCommit message (Collapse)AuthorFilesLines
2018-01-17iommu: Clean up of_iommu_init_fnRobin Murphy1-2/+2
Now that no more drivers rely on arbitrary early initialisation via an of_iommu_init_fn hook, let's clean up the redundant remnants. The IOMMU_OF_DECLARE() macro needs to remain for now, as the probe-deferral mechanism has no other nice way to detect built-in drivers before they have registered themselves, such that it can make the right decision. Reviewed-by: Sricharan R <sricharan@codeaurora.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-01-17iommu/ipmmu-vmsa: Remove redundant of_iommu_init_fn hookRobin Murphy1-12/+2
Having of_iommu_init() call ipmmu_init() via ipmmu_vmsa_iommu_of_setup() does nothing that the subsys_initcall wouldn't do slightly later anyway, since probe-deferral of masters means it is no longer critical to register the driver super-early. Clean it up. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-11-06iommu/ipmmu-vmsa: Hook up r8a7795 DT matching codeMagnus Damm1-0/+29
Tie in r8a7795 features and update the IOMMU_OF_DECLARE compat string to include the updated compat string. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Allow two bit SL0Magnus Damm1-1/+13
Introduce support for two bit SL0 bitfield in IMTTBCR by using a separate feature flag. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Make IMBUSCTR setup optionalMagnus Damm1-3/+6
Introduce a feature to allow opt-out of setting up IMBUSCR. The default case is unchanged. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Write IMCTR twiceMagnus Damm1-21/+35
Write IMCTR both in the root device and the leaf node. To allow access of IMCTR introduce the following function: - ipmmu_ctx_write_all() While at it also rename context functions: - ipmmu_ctx_read() -> ipmmu_ctx_read_root() - ipmmu_ctx_write() -> ipmmu_ctx_write_root() Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: IPMMU device is 40-bit bus masterMagnus Damm1-0/+1
The r8a7795 IPMMU supports 40-bit bus mastering. Both the coherent DMA mask and the streaming DMA mask are set to unlock the 40-bit address space for coherent allocations and streaming operations. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Make use of IOMMU_OF_DECLARE()Magnus Damm1-9/+41
Hook up IOMMU_OF_DECLARE() support in case CONFIG_IOMMU_DMA is enabled. The only current supported case for 32-bit ARM is disabled, however for 64-bit ARM usage of OF is required. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Enable multi context supportMagnus Damm1-8/+22
Add support for up to 8 contexts. Each context is mapped to one domain. One domain is assigned one or more slave devices. Contexts are allocated dynamically and slave devices are grouped together based on which IPMMU device they are connected to. This makes slave devices tied to the same IPMMU device share the same IOVA space. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Add optional root device featureMagnus Damm1-17/+73
Add root device handling to the IPMMU driver by allowing certain DT compat strings to enable has_cache_leaf_nodes that in turn will support both root devices with interrupts and leaf devices that face the actual IPMMU consumer devices. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Introduce features, break out aliasMagnus Damm1-7/+24
Introduce struct ipmmu_features to track various hardware and software implementation changes inside the driver for different kinds of IPMMU hardware. Add use_ns_alias_offset as a first example of a feature to control if the secure register bank offset should be used or not. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Unify ipmmu_opsRobin Murphy1-50/+19
The remaining difference between the ARM-specific and iommu-dma ops is in the {add,remove}_device implementations, but even those have some overlap and duplication. By stubbing out the few arm_iommu_*() calls, we can get rid of the rest of the inline #ifdeffery to both simplify the code and improve build coverage. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Clean up struct ipmmu_vmsa_iommu_privRobin Murphy1-24/+12
Now that the IPMMU instance pointer is the only thing remaining in the private data structure, we no longer need the extra level of indirection and can simply stash that directlty in the fwspec. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Simplify group allocationRobin Murphy1-44/+9
We go through quite the merry dance in order to find masters behind the same IPMMU instance, so that we can ensure they are grouped together. None of which is really necessary, since the master's private data already points to the particular IPMMU it is associated with, and that IPMMU instance data is the perfect place to keep track of a per-instance group directly. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Unify domain alloc/freeRobin Murphy1-41/+24
We have two implementations for ipmmu_ops->alloc depending on CONFIG_IOMMU_DMA, the difference being whether they accept the IOMMU_DOMAIN_DMA type or not. However, iommu_dma_get_cookie() is guaranteed to return an error when !CONFIG_IOMMU_DMA, so if ipmmu_domain_alloc_dma() was actually checking and handling the return value correctly, it would behave the same as ipmmu_domain_alloc() anyway. Similarly for freeing; iommu_put_dma_cookie() is robust by design. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-11-06iommu/ipmmu-vmsa: Fix return value check in ipmmu_find_group_dma()weiyongjun (A)1-1/+1
In case of error, the function iommu_group_get() returns NULL pointer not ERR_PTR(). The IS_ERR() test in the return value check should be replaced with NULL test. Fixes: 3ae47292024f ("iommu/ipmmu-vmsa: Add new IOMMU_DOMAIN_DMA ops") Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2017-10-02iommu/io-pgtable-arm: Convert to IOMMU API TLB syncRobin Murphy1-0/+10
Now that the core API issues its own post-unmap TLB sync call, push that operation out from the io-pgtable-arm internals into the users. For now, we leave the invalidation implicit in the unmap operation, since none of the current users would benefit much from any change to that. CC: Magnus Damm <damm+renesas@opensource.se> CC: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-30iommu/ipmmu-vmsa: Make ipmmu_gather_ops constBhumika Goyal1-1/+1
Make these const as they are not modified anywhere. Signed-off-by: Bhumika Goyal <bhumirks@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-30iommu/ipmmu-vmsa: Rereserving a free context before setting up a pagetableOleksandr Tyshchenko1-21/+21
Reserving a free context is both quicker and more likely to fail (due to limited hardware resources) than setting up a pagetable. What is more the pagetable init/cleanup code could require the context to be set up. Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> CC: Robin Murphy <robin.murphy@arm.com> CC: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> CC: Joerg Roedel <joro@8bytes.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-22iommu/ipmmu-vmsa: Use iommu_device_sysfs_add()/remove()Magnus Damm1-0/+6
Extend the driver to make use of iommu_device_sysfs_add()/remove() functions to hook up initial sysfs support. Suggested-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Clean up device trackingRobin Murphy1-12/+0
Get rid of now unused device tracking code. Future code should instead be able to use driver_for_each_device() for this purpose. This is a simplified version of the following patch from Robin [PATCH] iommu/ipmmu-vmsa: Clean up group allocation Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Replace local utlb code with fwspec idsMagnus Damm1-85/+19
Now when both 32-bit and 64-bit code inside the driver is using fwspec it is possible to replace the utlb handling with fwspec ids that get populated from ->of_xlate(). Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Use fwspec on both 32 and 64-bit ARMRobin Murphy1-18/+3
Consolidate the 32-bit and 64-bit code to make use of fwspec instead of archdata for the 32-bit ARM case. This is a simplified version of the fwspec handling code from Robin posted as [PATCH] iommu/ipmmu-vmsa: Convert to iommu_fwspec Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Consistent ->of_xlate() handlingMagnus Damm1-34/+17
The 32-bit ARM code gets updated to make use of ->of_xlate() and the code is shared between 64-bit and 32-bit ARM. The of_device_is_available() check gets dropped since it is included in of_iommu_xlate(). Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Use iommu_device_register()/unregister()Magnus Damm1-0/+10
Extend the driver to make use of iommu_device_register()/unregister() functions together with iommu_device_set_ops() and iommu_set_fwnode(). These used to be part of the earlier posted 64-bit ARM (r8a7795) series but it turns out that these days they are required on 32-bit ARM as well. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Fix pgsize_bitmap semicolon typoMagnus Damm1-1/+1
Fix comma-instead-of-semicolon typo error present in the latest version of the IPMMU driver. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Use fwspec iommu_priv on ARM64Magnus Damm1-39/+58
Convert from archdata to iommu_priv via iommu_fwspec on ARM64 but let 32-bit ARM keep on using archdata for now. Once the 32-bit ARM code and the IPMMU driver is able to move over to CONFIG_IOMMU_DMA=y then coverting to fwspec via ->of_xlate() will be easy. For now fwspec ids and num_ids are not used to allow code sharing between 32-bit and 64-bit ARM code inside the driver. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Add new IOMMU_DOMAIN_DMA opsMagnus Damm1-9/+155
Introduce an alternative set of iommu_ops suitable for 64-bit ARM as well as 32-bit ARM when CONFIG_IOMMU_DMA=y. Also adjust the Kconfig to depend on ARM or IOMMU_DMA. Initialize the device from ->xlate() when CONFIG_IOMMU_DMA=y. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Break out domain allocation codeMagnus Damm1-4/+9
Break out the domain allocation code into a separate function. This is preparation for future code sharing. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Reviewed-by: Joerg Roedel <jroedel@suse.de> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Break out utlb parsing codeMagnus Damm1-23/+41
Break out the utlb parsing code and dev_data allocation into a separate function. This is preparation for future code sharing. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Reviewed-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Rework interrupt code and use bitmap for contextMagnus Damm1-10/+66
Introduce a bitmap for context handing and convert the interrupt routine to handle all registered contexts. At this point the number of contexts are still limited. Also remove the use of the ARM specific mapping variable from ipmmu_irq() to allow compile on ARM64. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Reviewed-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-05-17iommu/ipmmu-vmsa: Remove platform data handlingMagnus Damm1-5/+0
The IPMMU driver is using DT these days, and platform data is no longer used by the driver. Remove unused code. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Joerg Roedel <jroedel@suse.de> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-02-06iommu/ipmmu-vmsa: Restrict IOMMU Domain Geometry to 32-bit address spaceGeert Uytterhoeven1-0/+2
Currently, the IPMMU/VMSA driver supports 32-bit I/O Virtual Addresses only, and thus sets io_pgtable_cfg.ias = 32. However, it doesn't force a 32-bit IOVA space through the IOMMU Domain Geometry. Hence if a device (e.g. SYS-DMAC) rightfully configures a 40-bit DMA mask, it will still be handed out a 40-bit IOVA, outside the 32-bit IOVA space, leading to out-of-bounds accesses of the PGD when mapping the IOVA. Force a 32-bit IOMMU Domain Geometry to fix this. Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/ipmmu-vmsa: Fix wrong error handle of ipmmu_add_deviceShawn Lin1-1/+1
Let's fix the error handle of ipmmu_add_device when failing to find utlbs, otherwise we take a risk of pontential memleak. Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-01-19Merge branches 's390', 'arm/renesas', 'arm/msm', 'arm/shmobile', 'arm/smmu', ↵Joerg Roedel1-2/+2
'x86/amd' and 'x86/vt-d' into next
2015-12-28iommu/ipmmu-vmsa: Don't truncate ttbr if LPAE is not enabledGeert Uytterhoeven1-1/+1
If CONFIG_PHYS_ADDR_T_64BIT=n: drivers/iommu/ipmmu-vmsa.c: In function 'ipmmu_domain_init_context': drivers/iommu/ipmmu-vmsa.c:434:2: warning: right shift count >= width of type ipmmu_ctx_write(domain, IMTTUBR0, ttbr >> 32); ^ As io_pgtable_cfg.arm_lpae_s1_cfg.ttbr[] is an array of u64s, assigning it to a phys_addr_t may truncates it. Make ttbr u64 to fix this. Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-17iommu/io-pgtable: Indicate granule for TLB maintenanceRobin Murphy1-2/+2
IOMMU hardware with range-based TLB maintenance commands can work happily with the iova and size arguments passed via the tlb_add_flush callback, but for IOMMUs which require separate commands per entry in the range, it is not straightforward to infer the necessary granularity when it comes to issuing the actual commands. Add an additional argument indicating the granularity for the benefit of drivers needing to know, and update the ARM LPAE code appropriately (for non-leaf invalidations we currently just assume the worst-case page granularity rather than walking the table to check). Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-08-06iommu/ipmmu-vmsa: Clean up DMA API usageRobin Murphy1-14/+5
With the correct DMA API calls now integrated into the io-pgtable code, let that handle the flushing of non-coherent page table updates. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-04-02Merge branches 'iommu/fixes', 'x86/vt-d', 'x86/amd', 'arm/smmu', 'arm/tegra' ↵Joerg Roedel1-18/+23
and 'core' into next Conflicts: drivers/iommu/amd_iommu.c drivers/iommu/tegra-gart.c drivers/iommu/tegra-smmu.c
2015-03-31iommu/ipmmu-vmsa: Make use of domain_alloc and domain_freeJoerg Roedel1-18/+23
Implement domain_alloc and domain_free iommu-ops as a replacement for domain_init/domain_destroy. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-23iommu: ipmmu-vmsa: Add terminating entry for ipmmu_of_idsAxel Lin1-0/+1
The of_device_id table is supposed to be zero-terminated. Signed-off-by: Axel Lin <axel.lin@ingics.com> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-30iommu/ipmmu-vmsa: Fix IOMMU lookup when multiple IOMMUs are registeredLaurent Pinchart1-28/+21
When adding a new device the driver loops over all registered IOMMUs and calls the ipmmu_find_utlbs() function to parse the DT iommus attribute. The function returns an error when the IOMMU referenced in DT doesn't match the current IOMMU. The caller incorrectly breaks from the loop immediately when the error is reported, resulting in only the first IOMMU being considered. Fix this, and while at it move code that isn't specific to an IOMMU instance out of the loop. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-26iommu/ipmmu-vmsa: Use the ARM LPAE page table allocatorLaurent Pinchart1-494/+60
Replace the custom page table allocation implementation with the standard allocator. The driver loses the ability to map 64kB chunkgs using the PTE contiguous hint, hence the removal of the SZ_64K page size from the IOMMU page sizes bitmap. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-16iommu/ipmmu-vmsa: Remove platform data supportLaurent Pinchart1-24/+0
No board file instantiates the IPMMU using platform data. Now that we have DT support, get rid of platform data. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
2015-01-16iommu/ipmmu-vmsa: Support multiple micro TLBs per deviceLaurent Pinchart1-29/+86
Devices such as the system DMA controller are connected to multiple micro TLBs of the same IOMMU. Support this. Selective enabling of micro TLBs based on runtime device usage isn't possible at the moment due to lack of support in the IOMMU and DMA mapping APIs. Support for devices connected to different IOMMUs is also unsupported for the same reason. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
2015-01-16iommu/ipmmu-vmsa: Add device tree supportLaurent Pinchart1-8/+47
Make platform data optional when the device is instantiated from DT and look up the micro-TLB number in the bus master DT node. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
2015-01-16iommu/ipmmu-vmsa: Invalidate TLB after unmappingLaurent Pinchart1-3/+1
The TLB must be invalidated after unmapping memory to remove stale TLB entries. this was supposed to be performed already, but a bug in the driver prevented the TLB invalidate function from being called. Fix it. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
2015-01-16iommu/ipmmu-vmsa: Flush P[UM]D entry before freeing the child page tableLaurent Pinchart1-8/+11
When clearing PUD or PMD entries the child page table (if any) is freed and the PUD or PMD entry is then cleared. This result in a small race condition window during which a free page table could be accessed by the IPMMU. Fix it by clearing and flushing the PUD or PMD entry before freeing the child page table. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
2015-01-16iommu/ipmmu-vmsa: Cleanup failures of ARM mapping creation or attachmentLaurent Pinchart1-1/+3
The ARM IOMMU mapping needs to be released when attaching the device fails. Add arm_iommu_release_mapping() to the error code path. This is safe to call with a NULL mapping, so no specific check is needed. Cleanup is also missing when failing to create a mapping. Jump to the error code path in that case instead of returning immediately. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
2015-01-05iommu/ipmmu-vmsa: Change IOMMU_EXEC to IOMMU_NOEXECLaurent Pinchart1-3/+3
Commit a720b41c41f5a7e4 ("iommu/arm-smmu: change IOMMU_EXEC to IOMMU_NOEXEC") has inverted and replaced the IOMMU_EXEC flag with IOMMU_NOEXEC. Update the driver accordingly. Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>