summaryrefslogtreecommitdiff
path: root/drivers/iommu
AgeCommit message (Collapse)AuthorFilesLines
2016-05-28remove lots of IS_ERR_VALUE abusesArnd Bergmann2-13/+13
Most users of IS_ERR_VALUE() in the kernel are wrong, as they pass an 'int' into a function that takes an 'unsigned long' argument. This happens to work because the type is sign-extended on 64-bit architectures before it gets converted into an unsigned type. However, anything that passes an 'unsigned short' or 'unsigned int' argument into IS_ERR_VALUE() is guaranteed to be broken, as are 8-bit integers and types that are wider than 'unsigned long'. Andrzej Hajda has already fixed a lot of the worst abusers that were causing actual bugs, but it would be nice to prevent any users that are not passing 'unsigned long' arguments. This patch changes all users of IS_ERR_VALUE() that I could find on 32-bit ARM randconfig builds and x86 allmodconfig. For the moment, this doesn't change the definition of IS_ERR_VALUE() because there are probably still architecture specific users elsewhere. Almost all the warnings I got are for files that are better off using 'if (err)' or 'if (err < 0)'. The only legitimate user I could find that we get a warning for is the (32-bit only) freescale fman driver, so I did not remove the IS_ERR_VALUE() there but changed the type to 'unsigned long'. For 9pfs, I just worked around one user whose calling conventions are so obscure that I did not dare change the behavior. I was using this definition for testing: #define IS_ERR_VALUE(x) ((unsigned long*)NULL == (typeof (x)*)NULL && \ unlikely((unsigned long long)(x) >= (unsigned long long)(typeof(x))-MAX_ERRNO)) which ends up making all 16-bit or wider types work correctly with the most plausible interpretation of what IS_ERR_VALUE() was supposed to return according to its users, but also causes a compile-time warning for any users that do not pass an 'unsigned long' argument. I suggested this approach earlier this year, but back then we ended up deciding to just fix the users that are obviously broken. After the initial warning that caused me to get involved in the discussion (fs/gfs2/dir.c) showed up again in the mainline kernel, Linus asked me to send the whole thing again. [ Updated the 9p parts as per Al Viro - Linus ] Signed-off-by: Arnd Bergmann <arnd@arndb.de> Cc: Andrzej Hajda <a.hajda@samsung.com> Cc: Andrew Morton <akpm@linux-foundation.org> Link: https://lkml.org/lkml/2016/1/7/363 Link: https://lkml.org/lkml/2016/5/27/486 Acked-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> # For nvmem part Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-27Merge git://git.infradead.org/intel-iommuLinus Torvalds2-118/+617
Pull intel IOMMU updates from David Woodhouse: "This patchset improves the scalability of the Intel IOMMU code by resolving two spinlock bottlenecks and eliminating the linearity of the IOVA allocator, yielding up to ~5x performance improvement and approaching 'iommu=off' performance" * git://git.infradead.org/intel-iommu: iommu/vt-d: Use per-cpu IOVA caching iommu/iova: introduce per-cpu caching to iova allocation iommu/vt-d: change intel-iommu to use IOVA frame numbers iommu/vt-d: avoid dev iotlb logic for domains with no dev iotlbs iommu/vt-d: only unmap mapped entries iommu/vt-d: correct flush_unmaps pfn usage iommu/vt-d: per-cpu deferred invalidation queues iommu/vt-d: refactoring of deferred flush entries
2016-05-21Merge tag 'devicetree-for-4.7' of ↵Linus Torvalds1-8/+30
git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux Pull devicetree updates from Rob Herring: - Rewrite of the unflattening code to avoid recursion and lessen the stack usage. - Rewrite of the phandle args parsing code to get rid of the fixed args size. This is needed for IOMMU code. - Sync to latest dtc which adds more dts style checking. These warnings are enabled with "W=1" compiles. - Tegra documentation updates related to the above warnings. - A bunch of spelling and other doc fixes. - Various vendor prefix additions. * tag 'devicetree-for-4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (52 commits) devicetree: Add Creative Technology vendor id gpio: dt-bindings: add ibm,ppc4xx-gpio binding of/unittest: Remove unnecessary module.h header inclusion drivers/of: Fix build warning in populate_node() drivers/of: Fix depth when unflattening devicetree of: dynamic: changeset prop-update revert fix drivers/of: Export of_detach_node() drivers/of: Return allocated memory from of_fdt_unflatten_tree() drivers/of: Specify parent node in of_fdt_unflatten_tree() drivers/of: Rename unflatten_dt_node() drivers/of: Avoid recursively calling unflatten_dt_node() drivers/of: Split unflatten_dt_node() of: include errno.h in of_graph.h of: document refcount incrementation of of_get_cpu_node() Documentation: dt: soc: fix spelling mistakes Documentation: dt: power: fix spelling mistake Documentation: dt: pinctrl: fix spelling mistake Documentation: dt: opp: fix spelling mistake Documentation: dt: net: fix spelling mistakes Documentation: dt: mtd: fix spelling mistake ...
2016-05-20Merge tag 'iommu-updates-v4.7' of ↵Linus Torvalds19-393/+891
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU updates from Joerg Roedel: "The updates include: - rate limiting for the VT-d fault handler - remove statistics code from the AMD IOMMU driver. It is unused and should be replaced by something more generic if needed - per-domain pagesize-bitmaps in IOMMU core code to support systems with different types of IOMMUs - support for ACPI devices in the AMD IOMMU driver - 4GB mode support for Mediatek IOMMU driver - ARM-SMMU updates from Will Deacon: - support for 64k pages with SMMUv1 implementations (e.g MMU-401) - remove open-coded 64-bit MMIO accessors - initial support for 16-bit VMIDs, as supported by some ThunderX SMMU implementations - a couple of errata workarounds for silicon in the field - various fixes here and there" * tag 'iommu-updates-v4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (44 commits) iommu/arm-smmu: Use per-domain page sizes. iommu/amd: Remove statistics code iommu/dma: Finish optimising higher-order allocations iommu: Allow selecting page sizes per domain iommu: of: enforce const-ness of struct iommu_ops iommu: remove unused priv field from struct iommu_ops iommu/dma: Implement scatterlist segment merging iommu/arm-smmu: Clear cache lock bit of ACR iommu/arm-smmu: Support SMMUv1 64KB supplement iommu/arm-smmu: Decouple context format from kernel config iommu/arm-smmu: Tidy up 64-bit/atomic I/O accesses io-64-nonatomic: Add relaxed accessor variants iommu/arm-smmu: Work around MMU-500 prefetch errata iommu/arm-smmu: Convert ThunderX workaround to new method iommu/arm-smmu: Differentiate specific implementations iommu/arm-smmu: Workaround for ThunderX erratum #27704 iommu/arm-smmu: Add support for 16 bit VMID iommu/amd: Move get_device_id() and friends to beginning of file iommu/amd: Don't use IS_ERR_VALUE to check integer values iommu/amd: Signedness bug in acpihid_device_group() ...
2016-05-19Merge tag 'pci-v4.7-changes' of ↵Linus Torvalds2-9/+4
git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci Pull PCI updates from Bjorn Helgaas: "Enumeration: - Refine PCI support check in pcibios_init() (Adrian-Ken Rueegsegger) - Provide common functions for ECAM mapping (Jayachandran C) - Allow all PCIe services on non-ACPI host bridges (Jon Derrick) - Remove return values from pcie_port_platform_notify() and relatives (Jon Derrick) - Widen portdrv service type from 4 bits to 8 bits (Keith Busch) - Add Downstream Port Containment portdrv service type (Keith Busch) - Add Downstream Port Containment driver (Keith Busch) Resource management: - Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs (Alex Williamson) - Supply CPU physical address (not bus address) to iomem_is_exclusive() (Bjorn Helgaas) - alpha: Call iomem_is_exclusive() for IORESOURCE_MEM, but not IORESOURCE_IO (Bjorn Helgaas) - Mark Broadwell-EP Home Agent 1 as having non-compliant BARs (Prarit Bhargava) - Disable all BAR sizing for devices with non-compliant BARs (Prarit Bhargava) - Move PCI I/O space management from OF to PCI core code (Tomasz Nowicki) PCI device hotplug: - acpiphp_ibm: Avoid uninitialized variable reference (Dan Carpenter) - Use cached copy of PCI_EXP_SLTCAP_HPC bit (Lukas Wunner) Virtualization: - Mark Intel i40e NIC INTx masking as broken (Alex Williamson) - Reverse standard ACS vs device-specific ACS enabling (Alex Williamson) - Work around Intel Sunrise Point PCH incorrect ACS capability (Alex Williamson) IOMMU: - Add pci_add_dma_alias() to abstract implementation (Bjorn Helgaas) - Move informational printk to pci_add_dma_alias() (Bjorn Helgaas) - Add support for multiple DMA aliases (Jacek Lawrynowicz) - Add DMA alias quirk for mic_x200_dma (Jacek Lawrynowicz) Thunderbolt: - Fix double free of drom buffer (Andreas Noever) - Add Intel Thunderbolt device IDs (Lukas Wunner) - Fix typos and magic number (Lukas Wunner) - Support 1st gen Light Ridge controller (Lukas Wunner) Generic host bridge driver: - Use generic ECAM API (Jayachandran C) Cavium ThunderX host bridge driver: - Don't clobber read-only bits in bridge config registers (David Daney) - Use generic ECAM API (Jayachandran C) Freescale i.MX6 host bridge driver: - Use enum instead of bool for variant indicator (Andrey Smirnov) - Implement reset sequence for i.MX6+ (Andrey Smirnov) - Factor out ref clock enable (Bjorn Helgaas) - Add initial imx6sx support (Christoph Fritz) - Add reset-gpio-active-high boolean property to DT (Petr Štetiar) - Add DT property for link gen, default to Gen1 (Tim Harvey) - dts: Specify imx6qp version of PCIe core (Andrey Smirnov) - dts: Fix PCIe reset GPIO polarity on Toradex Apalis Ixora (Petr Štetiar) Marvell Armada host bridge driver: - add DT binding for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni) - Add driver for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni) Marvell MVEBU host bridge driver: - Constify mvebu_pcie_pm_ops structure (Jisheng Zhang) - Use SET_NOIRQ_SYSTEM_SLEEP_PM_OPS for mvebu_pcie_pm_ops (Jisheng Zhang) Microsoft Hyper-V host bridge driver: - Report resources release after stopping the bus (Vitaly Kuznetsov) - Add explicit barriers to config space access (Vitaly Kuznetsov) Renesas R-Car host bridge driver: - Select PCI_MSI_IRQ_DOMAIN (Arnd Bergmann) Synopsys DesignWare host bridge driver: - Remove incorrect RC memory base/limit configuration (Gabriele Paoloni) - Move Root Complex setup code to dw_pcie_setup_rc() (Jisheng Zhang) TI Keystone host bridge driver: - Add error IRQ handler (Murali Karicheri) - Remove unnecessary goto statement (Murali Karicheri) Miscellaneous: - Fix spelling errors (Colin Ian King)" * tag 'pci-v4.7-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (48 commits) PCI: Disable all BAR sizing for devices with non-compliant BARs x86/PCI: Mark Broadwell-EP Home Agent 1 as having non-compliant BARs PCI: Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs PCI, of: Move PCI I/O space management to PCI core code PCI: generic, thunder: Use generic ECAM API PCI: Provide common functions for ECAM mapping PCI: hv: Add explicit barriers to config space access PCI: Use cached copy of PCI_EXP_SLTCAP_HPC bit PCI: Add Downstream Port Containment driver PCI: Add Downstream Port Containment portdrv service type PCI: Widen portdrv service type from 4 bits to 8 bits PCI: designware: Remove incorrect RC memory base/limit configuration PCI: hv: Report resources release after stopping the bus ARM: dts: imx6qp: Specify imx6qp version of PCIe core PCI: imx6: Implement reset sequence for i.MX6+ PCI: imx6: Use enum instead of bool for variant indicator PCI: thunder: Don't clobber read-only bits in bridge config registers thunderbolt: Fix double free of drom buffer PCI: rcar: Select PCI_MSI_IRQ_DOMAIN PCI: armada: Add driver for Marvell Armada 7K/8K PCIe controller ...
2016-05-09Merge branches 'arm/io-pgtable', 'arm/rockchip', 'arm/omap', 'x86/vt-d', ↵Joerg Roedel15-401/+938
'ppc/pamu', 'core' and 'x86/amd' into next
2016-05-09iommu/arm-smmu: Use per-domain page sizes.Robin Murphy2-21/+25
Now that we can accurately reflect the context format we choose for each domain, do that instead of imposing the global lowest-common-denominator restriction and potentially ending up with nothing. We currently have a strict 1:1 correspondence between domains and context banks, so we don't need to entertain the possibility of multiple formats _within_ a domain. Signed-off-by: Will Deacon <will.deacon@arm.com> [rm: split from original patch, added SMMUv3] Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09iommu/amd: Remove statistics codeJoerg Roedel3-131/+0
The statistics are not really used for anything and should be replaced by generic and per-device statistic counters. Remove the code for now. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09iommu/dma: Finish optimising higher-order allocationsRobin Murphy1-21/+39
Now that we know exactly which page sizes our caller wants to use in the given domain, we can restrict higher-order allocation attempts to just those sizes, if any, and avoid wasting any time or effort on other sizes which offer no benefit. In the same vein, this also lets us accommodate a minimum order greater than 0 for special cases. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Yong Wu <yong.wu@mediatek.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09iommu: Allow selecting page sizes per domainRobin Murphy3-12/+14
Many IOMMUs support multiple page table formats, meaning that any given domain may only support a subset of the hardware page sizes presented in iommu_ops->pgsize_bitmap. There are also certain use-cases where the creator of a domain may want to control which page sizes are used, for example to force the use of hugepage mappings to reduce pagetable walk depth. To this end, add a per-domain pgsize_bitmap to represent the subset of page sizes actually in use, to make it possible for domains with different requirements to coexist. Signed-off-by: Will Deacon <will.deacon@arm.com> [rm: hijacked and rebased original patch with new commit message] Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09iommu: of: enforce const-ness of struct iommu_opsRobin Murphy1-7/+7
As a set of driver-provided callbacks and static data, there is no compelling reason for struct iommu_ops to be mutable in core code, so enforce const-ness throughout. Acked-by: Thierry Reding <treding@nvidia.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09Merge branch 'arm/smmu' into coreJoerg Roedel1-81/+210
2016-05-09iommu/dma: Implement scatterlist segment mergingRobin Murphy1-23/+61
Stop wasting IOVA space by over-aligning scatterlist segments for a theoretical worst-case segment boundary mask, and instead take the real limits into account to merge consecutive segments wherever appropriate, so our callers can benefit from getting back nicely simplified lists. This also represents the last piece of functionality wanted by users of the current arch/arm implementation, thus brings us a small step closer to converting that over to the common code. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09Merge branch 'for-joerg/arm-smmu/updates' of ↵Joerg Roedel1-81/+210
git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu
2016-05-05Merge tag 'v4.6-rc6' into x86/asm, to refresh the treeIngo Molnar2-18/+91
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-05-03iommu/arm-smmu: Clear cache lock bit of ACRPeng Fan1-1/+22
According MMU-500r2 TRM, section 3.7.1 Auxiliary Control registers, You can modify ACTLR only when the ACR.CACHE_LOCK bit is 0. So before clearing ARM_MMU500_ACTLR_CPRE of each context bank, need clear CACHE_LOCK bit of ACR register first. Since CACHE_LOCK bit is only present in MMU-500r2 onwards, need to check the major number of IDR7. Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Peng Fan <van.freenix@gmail.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Support SMMUv1 64KB supplementRobin Murphy1-8/+14
The 64KB Translation Granule Supplement to the SMMUv1 architecture allows an SMMUv1 implementation to support 64KB pages for stage 2 translations, using a constrained VMSAv8 descriptor format limited to 40-bit addresses. Now that we can freely mix and match context formats, we can actually handle having 4KB pages via an AArch32 context but 64KB pages via an AArch64 context, so plumb it in. It is assumed that any implementations will have hardware capabilities matching the format constraints, thus obviating the need for excessive sanity-checking; this is the case for MMU-401, the only ARM Ltd. implementation. CC: Eric Auger <eric.auger@linaro.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Decouple context format from kernel configRobin Murphy1-22/+72
The way the driver currently forces an AArch32 or AArch64 context format based on the kernel config and SMMU architecture version is suboptimal, in that it makes it very hard to support oddball mix-and-match cases like the SMMUv1 64KB supplement, or situations where the reduced table depth of an AArch32 short descriptor context may be desirable under an AArch64 kernel. It also only happens to work on current implementations which do support all the relevant formats. Introduce an explicit notion of context format, so we can manage that independently and get rid of the inflexible #ifdeffery. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Tidy up 64-bit/atomic I/O accessesRobin Murphy1-29/+19
With {read,write}q_relaxed now able to fall back to the common nonatomic-hi-lo helper, make use of that so that we don't have to open-code our own. In the process, also convert the other remaining split accesses, and repurpose the custom accessor to smooth out the couple of troublesome instances where we really want to avoid nonatomic writes (and a 64-bit access is unnecessary in the 32-bit context formats we would use on a 32-bit CPU). This paves the way for getting rid of some of the assumptions currently baked into the driver which make it really awkward to use 32-bit context formats with SMMUv2 under a 64-bit kernel. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Work around MMU-500 prefetch errataRobin Murphy1-1/+15
MMU-500 erratum #841119 is tickled by a particular set of circumstances interacting with the next-page prefetcher. Since said prefetcher is quite dumb and actually detrimental to performance in some cases (by causing unwanted TLB evictions for non-sequential access patterns), we lose very little by turning it off, and what we gain is a guarantee that the erratum is never hit. As a bonus, the same workaround will also prevent erratum #826419 once v7 short descriptor support is implemented. CC: Catalin Marinas <catalin.marinas@arm.com> CC: Will Deacon <will.deacon@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Convert ThunderX workaround to new methodRobin Murphy1-13/+14
With a framework for implementation-specific funtionality in place, the currently-FDT-dependent ThunderX workaround gets to be the first user. Acked-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Differentiate specific implementationsRobin Murphy1-7/+26
As the inevitable reality of implementation-specific errata workarounds begin to accrue alongside our integration quirk handling, it's about time the driver had a decent way of keeping track. Extend the per-SMMU data so we can identify specific implementations in an efficient and firmware-agnostic manner. Acked-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Workaround for ThunderX erratum #27704Tirumalesh Chalamarla1-11/+28
Due to erratum #27704, the CN88xx SMMUv2 implementation supports only shared ASID and VMID numberspaces. This patch ensures that ASID and VMIDs are unique across all SMMU instances on affected Cavium systems. Signed-off-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com> Signed-off-by: Akula Geethasowjanya <Geethasowjanya.Akula@caviumnetworks.com> [will: commit message, comments and formatting] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03iommu/arm-smmu: Add support for 16 bit VMIDTirumalesh Chalamarla1-6/+17
This patch adds support for 16-bit VMIDs on implementations of SMMUv2 that support it. Signed-off-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com> [will: commit messsage and comments] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-04-21iommu/amd: Move get_device_id() and friends to beginning of fileJoerg Roedel1-54/+54
They will be needed there later. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21iommu/amd: Don't use IS_ERR_VALUE to check integer valuesJoerg Roedel1-10/+10
Use the better 'var < 0' check. Fixes: 7aba6cb9ee9d ('iommu/amd: Make call-sites of get_device_id aware of its return value') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21iommu/arm-smmu: Don't allocate resources for bypass domainsRobin Murphy1-1/+7
Until we get fully plumbed into of_iommu_configure, our default IOMMU_DOMAIN_DMA domains just bypass translation. Since we achieve that by leaving the stream table entries set to bypass instead of pointing at a translation context, the context bank we allocate for the domain is completely wasted. Context banks are typically a rather limited resource, so don't hog ones we don't need. Reported-by: Eric Auger <eric.auger@linaro.org> Tested-by: Eric Auger <eric.auger@linaro.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21iommu/arm-smmu: Fix stream-match conflict with IOMMU_DOMAIN_DMAWill Deacon1-6/+8
Commit cbf8277ef456 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now") ignores requests to attach a device to the default domain since, without IOMMU-basked DMA ops available everywhere, the default domain will just lead to unexpected transaction faults being reported. Unfortunately, the way this was implemented on SMMUv2 causes a regression with VFIO PCI device passthrough under KVM on AMD Seattle. On this system, the host controller device is associated with both a pci_dev *and* a platform_device, and can therefore end up with duplicate SMR entries, resulting in a stream-match conflict at runtime. This patch amends the original fix so that attaching to IOMMU_DOMAIN_DMA is rejected even before configuring the SMRs. This restores the old behaviour for now, but we'll need to look at handing host controllers specially when we come to supporting the default domain fully. Reported-by: Eric Auger <eric.auger@linaro.org> Tested-by: Eric Auger <eric.auger@linaro.org> Tested-by: Yang Shi <yang.shi@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-20iommu/vt-d: Use per-cpu IOVA cachingOmer Peleg1-12/+35
Commit 9257b4a2 ('iommu/iova: introduce per-cpu caching to iova allocation') introduced per-CPU IOVA caches to massively improve scalability. Use them. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased, cleaned up and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> [dwmw2: split out VT-d part into a separate patch] Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/iova: introduce per-cpu caching to iova allocationOmer Peleg1-24/+393
IOVA allocation has two problems that impede high-throughput I/O. First, it can do a linear search over the allocated IOVA ranges. Second, the rbtree spinlock that serializes IOVA allocations becomes contended. Address these problems by creating an API for caching allocated IOVA ranges, so that the IOVA allocator isn't accessed frequently. This patch adds a per-CPU cache, from which CPUs can alloc/free IOVAs without taking the rbtree spinlock. The per-CPU caches are backed by a global cache, to avoid invoking the (linear-time) IOVA allocator without needing to make the per-CPU cache size excessive. This design is based on magazines, as described in "Magazines and Vmem: Extending the Slab Allocator to Many CPUs and Arbitrary Resources" (currently available at https://www.usenix.org/legacy/event/usenix01/bonwick.html) Adding caching on top of the existing rbtree allocator maintains the property that IOVAs are densely packed in the IO virtual address space, which is important for keeping IOMMU page table usage low. To keep the cache size reasonable, we bound the IOVA space a CPU can cache by 32 MiB (we cache a bounded number of IOVA ranges, and only ranges of size <= 128 KiB). The shared global cache is bounded at 4 MiB of IOVA space. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased, cleaned up and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> [dwmw2: split out VT-d part into a separate patch] Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/vt-d: change intel-iommu to use IOVA frame numbersOmer Peleg1-32/+29
Make intel-iommu map/unmap/invalidate work with IOVA pfns instead of pointers to "struct iova". This avoids using the iova struct from the IOVA red-black tree and the resulting explicit find_iova() on unmap. This patch will allow us to cache IOVAs in the next patch, in order to avoid rbtree operations for the majority of map/unmap operations. Note: In eliminating the find_iova() operation, we have also eliminated the sanity check previously done in the unmap flow. Arguably, this was overhead that is better avoided in production code, but it could be brought back as a debug option for driver development. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased, fixed to not break iova api, and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/vt-d: avoid dev iotlb logic for domains with no dev iotlbsOmer Peleg1-0/+34
This patch avoids taking the device_domain_lock in iommu_flush_dev_iotlb() for domains with no dev iotlb devices. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [gvdl@google.com: fixed locking issues] Signed-off-by: Godfrey van der Linden <gvdl@google.com> [mad@cs.technion.ac.il: rebased and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/vt-d: only unmap mapped entriesOmer Peleg1-11/+25
Current unmap implementation unmaps the entire area covered by the IOVA range, which is a power-of-2 aligned region. The corresponding map, however, only maps those pages originally mapped by the user. This discrepancy can lead to unmapping of already unmapped entries, which is unneeded work. With this patch, only mapped pages are unmapped. This is also a baseline for a map/unmap implementation based on IOVAs and not iova structures, which will allow caching. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/vt-d: correct flush_unmaps pfn usageOmer Peleg1-1/+2
Change flush_unmaps() to correctly pass iommu_flush_iotlb_psi() dma addresses. (x86_64 mm and dma have the same size for pages at the moment, but this usage improves consistency.) Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/vt-d: per-cpu deferred invalidation queuesOmer Peleg1-41/+92
The IOMMU's IOTLB invalidation is a costly process. When iommu mode is not set to "strict", it is done asynchronously. Current code amortizes the cost of invalidating IOTLB entries by batching all the invalidations in the system and performing a single global invalidation instead. The code queues pending invalidations in a global queue that is accessed under the global "async_umap_flush_lock" spinlock, which can result is significant spinlock contention. This patch splits this deferred queue into multiple per-cpu deferred queues, and thus gets rid of the "async_umap_flush_lock" and its contention. To keep existing deferred invalidation behavior, it still invalidates the pending invalidations of all CPUs whenever a CPU reaches its watermark or a timeout occurs. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased, cleaned up and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/vt-d: refactoring of deferred flush entriesOmer Peleg1-19/+29
Currently, deferred flushes' info is striped between several lists in the flush tables. Instead, move all information about a specific flush to a single entry in this table. This patch does not introduce any functional change. Signed-off-by: Omer Peleg <omer@cs.technion.ac.il> [mad@cs.technion.ac.il: rebased and reworded the commit message] Signed-off-by: Adam Morrison <mad@cs.technion.ac.il> Reviewed-by: Shaohua Li <shli@fb.com> Reviewed-by: Ben Serebrin <serebrin@google.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20iommu/arm-smmu: Make use of phandle iterators in device-tree parsingJoerg Roedel1-8/+30
Remove the usage of of_parse_phandle_with_args() and replace it by the phandle-iterator implementation so that we can parse out all of the potentially present 128 stream-ids. Signed-off-by: Joerg Roedel <jroedel@suse.de> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Rob Herring <robh@kernel.org>
2016-04-19Merge tag 'v4.6-rc4' into x86/asm, to pick up fixesIngo Molnar4-8/+9
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-04-15iommu/amd: Signedness bug in acpihid_device_group()Dan Carpenter1-1/+1
"devid" needs to be signed for the error handling to work. Fixes: b097d11a0fa3f ('iommu/amd: Manage iommu_group for ACPI HID devices') Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-13x86/cpufeature: Replace cpu_has_apic with boot_cpu_has() usageBorislav Petkov1-1/+1
Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: iommu@lists.linux-foundation.org Cc: linux-pm@vger.kernel.org Cc: oprofile-list@lists.sf.net Link: http://lkml.kernel.org/r/1459801503-15600-8-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-04-11PCI: Add support for multiple DMA aliasesJacek Lawrynowicz1-7/+3
Solve IOMMU support issues with PCIe non-transparent bridges that use Requester ID look-up tables (RID-LUT), e.g., the PEX8733. The NTB connects devices in two independent PCI domains. Devices separated by the NTB are not able to discover each other. A PCI packet being forwared from one domain to another has to have its RID modified so it appears on correct bus and completions are forwarded back to the original domain through the NTB. The RID is translated using a preprogrammed table (LUT) and the PCI packet propagates upstream away from the NTB. If the destination system has IOMMU enabled, the packet will be discarded because the new RID is unknown to the IOMMU. Adding a DMA alias for the new RID allows IOMMU to properly recognize the packet. Each device behind the NTB has a unique RID assigned in the RID-LUT. The current DMA alias implementation supports only a single alias, so it's not possible to support mutiple devices behind the NTB when IOMMU is enabled. Enable all possible aliases on a given bus (256) that are stored in a bitset. Alias devfn is directly translated to a bit number. The bitset is not allocated for devices that have no need for DMA aliases. More details can be found in the following article: http://www.plxtech.com/files/pdf/technical/expresslane/RTC_Enabling%20MulitHostSystemDesigns.pdf Signed-off-by: Jacek Lawrynowicz <jacek.lawrynowicz@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: David Woodhouse <David.Woodhouse@intel.com> Acked-by: Joerg Roedel <jroedel@suse.de>
2016-04-11iommu/amd: Fix checking of pci dma aliasesJoerg Roedel1-11/+76
Commit 61289cb ('iommu/amd: Remove old alias handling code') removed the old alias handling code from the AMD IOMMU driver because this is now handled by the IOMMU core code. But this also removed the handling of PCI aliases, which is not handled by the core code. This caused issues with PCI devices that have hidden PCIe-to-PCI bridges that rewrite the request-id. Fix this bug by re-introducing some of the removed functions from commit 61289cbaf6c8 and add a alias field 'struct iommu_dev_data'. This field carrys the return value of the get_alias() function and uses that instead of the amd_iommu_alias_table[] array in the code. Fixes: 61289cbaf6c8 ('iommu/amd: Remove old alias handling code') Cc: stable@vger.kernel.org # v4.4+ Tested-by: Tomasz Golinski <tomaszg@math.uwb.edu.pl> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu/io-pgtable-arm-v7s: Support IOMMU_MMIO flagRobin Murphy1-5/+11
Teach the short-descriptor format to create Device mappings when asked. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu/io-pgtable-arm: Support IOMMU_MMIO flagRobin Murphy1-2/+7
Teach the LPAE format to create Device mappings when asked. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Eric Auger <eric.auger@linaro.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu/vt-d: Silence an uninitialized variable warningDan Carpenter1-1/+1
My static checker complains that "dma_alias" is uninitialized unless we are dealing with a pci device. This is true but harmless. Anyway, we can flip the condition around to silence the warning. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu/rockchip: Fix "is stall active" checkJohn Keeping1-4/+4
Since commit cd6438c5f844 ("iommu/rockchip: Reconstruct to support multi slaves") rk_iommu_is_stall_active() always returns false because the bitwise AND operates on the boolean flag promoted to an integer and a value that is either zero or BIT(2). Explicitly convert the right-hand value to a boolean so that both sides are guaranteed to be either zero or one. rk_iommu_is_paging_enabled() does not suffer from the same problem since RK_MMU_STATUS_PAGING_ENABLED is BIT(0), but let's apply the same change for consistency and to make it clear that it's correct without needing to lookup the value. Fixes: cd6438c5f844 ("iommu/rockchip: Reconstruct to support multi slaves") Signed-off-by: John Keeping <john@metanate.com> Reviewed-by: Heiko Stuebner <heiko@sntech.de> Tested-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07powerpc: Fix incorrect PPC32 PAMU dependencyAndy Fleming1-2/+1
The Freescale PAMU can be enabled on both 32 and 64-bit Power chips. Commit 477ab7a19ce restricted PAMU to PPC32. PPC covers both. Signed-off-by: Andy Fleming <afleming@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu: Don't overwrite domain pointer when there is no default_domainJoerg Roedel1-1/+2
IOMMU drivers that do not support default domains, but make use of the the group->domain pointer can get that pointer overwritten with NULL on device add/remove. Make sure this can't happen by only overwriting the domain pointer when it is NULL. Cc: stable@vger.kernel.org # v4.4+ Fixes: 1228236de5f9 ('iommu: Move default domain allocation to iommu_group_get_for_dev()') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu/amd: Set AMD iommu callbacks for amba busWan Zongshun1-1/+12
AMD Uart DMA belongs to ACPI HID type device, and its driver is basing on AMBA Bus, need also IOMMU support. This patch is just to set the AMD iommu callbacks for amba bus. Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07iommu/amd: Manage iommu_group for ACPI HID devicesWan Zongshun1-1/+32
This patch creates a new function for finding or creating an IOMMU group for acpihid(ACPI Hardware ID) device. The acpihid devices with the same devid will be put into same group and there will have the same domain id and share the same page table. Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
generated by cgit v1.2.3 (git 2.39.0) at 2024-12-22 08:56:06 +0300