summaryrefslogtreecommitdiff
path: root/drivers/iommu
AgeCommit message (Collapse)AuthorFilesLines
2019-02-09Merge tag 'iommu-fixes-v5.0-rc5' of ↵Linus Torvalds1-4/+4
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU fix from Joerg Roedel: "Intel decided to leave the newly added Scalable Mode Feature default-disabled for now. The patch here accomplishes that" * tag 'iommu-fixes-v5.0-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/vt-d: Leave scalable mode default off
2019-01-30Merge tag 'iommu-fixes-v5.0-rc4' of ↵Linus Torvalds3-7/+18
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU fixes from Joerg Roedel: "A few more fixes this time: - Two patches to fix the error path of the map_sg implementation of the AMD IOMMU driver. - Also a missing IOTLB flush is fixed in the AMD IOMMU driver. - Memory leak fix for the Intel IOMMU driver. - Fix a regression in the Mediatek IOMMU driver which caused device initialization to fail (seen as broken HDMI output)" * tag 'iommu-fixes-v5.0-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/amd: Fix IOMMU page flush when detach device from a domain iommu/mediatek: Use correct fwspec in mtk_iommu_add_device() iommu/vt-d: Fix memory leak in intel_iommu_put_resv_regions() iommu/amd: Unmap all mapped pages in error path of map_sg iommu/amd: Call free_iova_fast with pfn in map_sg
2019-01-30iommu/vt-d: Leave scalable mode default offLu Baolu1-4/+4
Commit 765b6a98c1de3 ("iommu/vt-d: Enumerate the scalable mode capability") enables VT-d scalable mode if hardware advertises the capability. As we will bring up different features and use cases to upstream in different patch series, it will leave some intermediate kernel versions which support partial features. Hence, end user might run into problems when they use such kernels on bare metals or virtualization environments. This leaves scalable mode default off and end users could turn it on with "intel-iommu=sm_on" only when they have clear ideas about which scalable features are supported in the kernel. Cc: Liu Yi L <yi.l.liu@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Suggested-by: Ashok Raj <ashok.raj@intel.com> Suggested-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-24iommu/amd: Fix IOMMU page flush when detach device from a domainSuravee Suthikulpanit1-4/+11
When a VM is terminated, the VFIO driver detaches all pass-through devices from VFIO domain by clearing domain id and page table root pointer from each device table entry (DTE), and then invalidates the DTE. Then, the VFIO driver unmap pages and invalidate IOMMU pages. Currently, the IOMMU driver keeps track of which IOMMU and how many devices are attached to the domain. When invalidate IOMMU pages, the driver checks if the IOMMU is still attached to the domain before issuing the invalidate page command. However, since VFIO has already detached all devices from the domain, the subsequent INVALIDATE_IOMMU_PAGES commands are being skipped as there is no IOMMU attached to the domain. This results in data corruption and could cause the PCI device to end up in indeterministic state. Fix this by invalidate IOMMU pages when detach a device, and before decrementing the per-domain device reference counts. Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Suggested-by: Joerg Roedel <joro@8bytes.org> Co-developed-by: Brijesh Singh <brijesh.singh@amd.com> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Fixes: 6de8ad9b9ee0 ('x86/amd-iommu: Make iommu_flush_pages aware of multiple IOMMUs') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-23iommu/mediatek: Use correct fwspec in mtk_iommu_add_device()Joerg Roedel1-0/+4
The mtk_iommu_add_device() function keeps the fwspec in an on-stack pointer and calls mtk_iommu_create_mapping(), which might change its source, dev->iommu_fwspec. This causes the on-stack pointer to be obsoleted and the device initialization to fail. Update the on-stack fwspec pointer after mtk_iommu_create_mapping() has been called. Reported-by: Frank Wunderlich <frank-w@public-files.de> Fixes: a9bf2eec5a6f ('iommu/mediatek: Use helper functions to access dev->iommu_fwspec') Tested-by: Frank Wunderlich <frank-w@public-files.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-22iommu/vt-d: Fix memory leak in intel_iommu_put_resv_regions()Gerald Schaefer1-1/+1
Commit 9d3a4de4cb8d ("iommu: Disambiguate MSI region types") changed the reserved region type in intel_iommu_get_resv_regions() from IOMMU_RESV_RESERVED to IOMMU_RESV_MSI, but it forgot to also change the type in intel_iommu_put_resv_regions(). This leads to a memory leak, because now the check in intel_iommu_put_resv_regions() for IOMMU_RESV_RESERVED will never be true, and no allocated regions will be freed. Fix this by changing the region type in intel_iommu_put_resv_regions() to IOMMU_RESV_MSI, matching the type of the allocated regions. Fixes: 9d3a4de4cb8d ("iommu: Disambiguate MSI region types") Cc: <stable@vger.kernel.org> # v4.11+ Signed-off-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-22iommu/amd: Unmap all mapped pages in error path of map_sgJerry Snitselaar1-1/+1
In the error path of map_sg there is an incorrect if condition for breaking out of the loop that searches the scatterlist for mapped pages to unmap. Instead of breaking out of the loop once all the pages that were mapped have been unmapped, it will break out of the loop after it has unmapped 1 page. Fix the condition, so it breaks out of the loop only after all the mapped pages have been unmapped. Fixes: 80187fd39dcb ("iommu/amd: Optimize map_sg and unmap_sg") Cc: Joerg Roedel <joro@8bytes.org> Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-22iommu/amd: Call free_iova_fast with pfn in map_sgJerry Snitselaar1-1/+1
In the error path of map_sg, free_iova_fast is being called with address instead of the pfn. This results in a bad value getting into the rcache, and can result in hitting a BUG_ON when iova_magazine_free_pfns is called. Cc: Joerg Roedel <joro@8bytes.org> Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com> Fixes: 80187fd39dcb ("iommu/amd: Optimize map_sg and unmap_sg") Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-21Merge tag 'iommu-fixes-v5.0-rc3' of ↵Linus Torvalds1-1/+1
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU fix from Joerg Roedel: "One fix only for now: Fix probe deferral in iommu/of code (broke with recent changes to iommu_ops->add_device invocation)" * tag 'iommu-fixes-v5.0-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/of: Fix probe-deferral
2019-01-11iommu/of: Fix probe-deferralRobin Murphy1-1/+1
Whilst iommu_probe_device() does check for non-NULL ops as the previous code did, it does not do so in the same order relative to the other checks, and as a result means that -EPROBE_DEFER returned by of_xlate() (plus any real error condition too) gets overwritten with -EINVAL and leads to various misbehaviour. Reinstate the original logic, but without implicitly relying on ops being set to infer !err as the initial condition (now that the validity of ops for its own sake is checked elsewhere). Fixes: 641fb0efbff0 ("iommu/of: Don't call iommu_ops->add_device directly") Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-01-08cross-tree: phase out dma_zalloc_coherent()Luis Chamberlain1-3/+2
We already need to zero out memory for dma_alloc_coherent(), as such using dma_zalloc_coherent() is superflous. Phase it out. This change was generated with the following Coccinelle SmPL patch: @ replace_dma_zalloc_coherent @ expression dev, size, data, handle, flags; @@ -dma_zalloc_coherent(dev, size, handle, flags) +dma_alloc_coherent(dev, size, handle, flags) Suggested-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> [hch: re-ran the script on the latest tree] Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-01-02Merge tag 'iommu-updates-v4.21' of ↵Linus Torvalds27-786/+1414
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU updates from Joerg Roedel: - Page table code for AMD IOMMU now supports large pages where smaller page-sizes were mapped before. VFIO had to work around that in the past and I included a patch to remove it (acked by Alex Williamson) - Patches to unmodularize a couple of IOMMU drivers that would never work as modules anyway. - Work to unify the the iommu-related pointers in 'struct device' into one pointer. This work is not finished yet, but will probably be in the next cycle. - NUMA aware allocation in iommu-dma code - Support for r8a774a1 and r8a774c0 in the Renesas IOMMU driver - Scalable mode support for the Intel VT-d driver - PM runtime improvements for the ARM-SMMU driver - Support for the QCOM-SMMUv2 IOMMU hardware from Qualcom - Various smaller fixes and improvements * tag 'iommu-updates-v4.21' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (78 commits) iommu: Check for iommu_ops == NULL in iommu_probe_device() ACPI/IORT: Don't call iommu_ops->add_device directly iommu/of: Don't call iommu_ops->add_device directly iommu: Consolitate ->add/remove_device() calls iommu/sysfs: Rename iommu_release_device() dmaengine: sh: rcar-dmac: Use device_iommu_mapped() xhci: Use device_iommu_mapped() powerpc/iommu: Use device_iommu_mapped() ACPI/IORT: Use device_iommu_mapped() iommu/of: Use device_iommu_mapped() driver core: Introduce device_iommu_mapped() function iommu/tegra: Use helper functions to access dev->iommu_fwspec iommu/qcom: Use helper functions to access dev->iommu_fwspec iommu/of: Use helper functions to access dev->iommu_fwspec iommu/mediatek: Use helper functions to access dev->iommu_fwspec iommu/ipmmu-vmsa: Use helper functions to access dev->iommu_fwspec iommu/dma: Use helper functions to access dev->iommu_fwspec iommu/arm-smmu: Use helper functions to access dev->iommu_fwspec ACPI/IORT: Use helper functions to access dev->iommu_fwspec iommu: Introduce wrappers around dev->iommu_fwspec ...
2018-12-29Merge tag 'char-misc-4.21-rc1' of ↵Linus Torvalds2-3/+78
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc Pull char/misc driver updates from Greg KH: "Here is the big set of char and misc driver patches for 4.21-rc1. Lots of different types of driver things in here, as this tree seems to be the "collection of various driver subsystems not big enough to have their own git tree" lately. Anyway, some highlights of the changes in here: - binderfs: is it a rule that all driver subsystems will eventually grow to have their own filesystem? Binder now has one to handle the use of it in containerized systems. This was discussed at the Plumbers conference a few months ago and knocked into mergable shape very fast by Christian Brauner. Who also has signed up to be another binder maintainer, showing a distinct lack of good judgement :) - binder updates and fixes - mei driver updates - fpga driver updates and additions - thunderbolt driver updates - soundwire driver updates - extcon driver updates - nvmem driver updates - hyper-v driver updates - coresight driver updates - pvpanic driver additions and reworking for more device support - lp driver updates. Yes really, it's _finally_ moved to the proper parallal port driver model, something I never thought I would see happen. Good stuff. - other tiny driver updates and fixes. All of these have been in linux-next for a while with no reported issues" * tag 'char-misc-4.21-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc: (116 commits) MAINTAINERS: add another Android binder maintainer intel_th: msu: Fix an off-by-one in attribute store stm class: Add a reference to the SyS-T document stm class: Fix a module refcount leak in policy creation error path char: lp: use new parport device model char: lp: properly count the lp devices char: lp: use first unused lp number while registering char: lp: detach the device when parallel port is removed char: lp: introduce list to save port number bus: qcom: remove duplicated include from qcom-ebi2.c VMCI: Use memdup_user() rather than duplicating its implementation char/rtc: Use of_node_name_eq for node name comparisons misc: mic: fix a DMA pool free failure ptp: fix an IS_ERR() vs NULL check genwqe: Fix size check binder: implement binderfs binder: fix use-after-free due to ksys_close() during fdget() bus: fsl-mc: remove duplicated include files bus: fsl-mc: explicitly define the fsl_mc_command endianness misc: ti-st: make array read_ver_cmd static, shrinks object size ...
2018-12-20Merge branches 'iommu/fixes', 'arm/renesas', 'arm/mediatek', 'arm/tegra', ↵Joerg Roedel26-782/+1414
'arm/omap', 'arm/smmu', 'x86/vt-d', 'x86/amd' and 'core' into next
2018-12-20iommu: Check for iommu_ops == NULL in iommu_probe_device()Joerg Roedel1-1/+5
This check needs to be there and got lost at some point during development. Add it again. Fixes: 641fb0efbff0 ('iommu/of: Don't call iommu_ops->add_device directly') Reported-by: Marek Szyprowski <m.szyprowski@samsung.com> Reported-by: kernelci.org bot <bot@kernelci.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/of: Don't call iommu_ops->add_device directlyJoerg Roedel1-3/+3
Make sure to invoke this call-back through the proper function of the IOMMU-API. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu: Consolitate ->add/remove_device() callsJoerg Roedel1-26/+25
Put them into separate functions and call those where the plain ops have been called before. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/sysfs: Rename iommu_release_device()Joerg Roedel1-6/+6
Remove the iommu_ prefix from the function and a few other static data structures so that the iommu_release_device name can be re-used in iommu core code. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/of: Use device_iommu_mapped()Joerg Roedel1-1/+1
Use Use device_iommu_mapped() to check if the device is already mapped by an IOMMU. Acked-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/tegra: Use helper functions to access dev->iommu_fwspecJoerg Roedel1-1/+1
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Cc: Thierry Reding <thierry.reding@gmail.com> Acked-by: Thierry Reding <treding@nvidia.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/qcom: Use helper functions to access dev->iommu_fwspecJoerg Roedel1-8/+10
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Cc: Rob Clark <robdclark@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/of: Use helper functions to access dev->iommu_fwspecJoerg Roedel1-3/+7
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/mediatek: Use helper functions to access dev->iommu_fwspecJoerg Roedel2-21/+28
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Cc: Matthias Brugger <matthias.bgg@gmail.com> Tested-by: Yong Wu <yong.wu@mediatek.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/ipmmu-vmsa: Use helper functions to access dev->iommu_fwspecJoerg Roedel1-4/+8
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/dma: Use helper functions to access dev->iommu_fwspecJoerg Roedel1-1/+1
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/arm-smmu: Use helper functions to access dev->iommu_fwspecJoerg Roedel2-13/+15
Use the new helpers dev_iommu_fwspec_get()/set() to access the dev->iommu_fwspec pointer. This makes it easier to move that pointer later into another struct. Cc: Will Deacon <will.deacon@arm.com> Cc: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu: Introduce wrappers around dev->iommu_fwspecJoerg Roedel1-7/+7
These wrappers will be used to easily change the location of the field later when all users are converted. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17iommu/ipmmu-vmsa: Hook up r8a774c0 DT matching codeFabrizio Castro1-0/+5
Support RZ/G2E (a.k.a. R8A774C0) IPMMU. Signed-off-by: Fabrizio Castro <fabrizio.castro@bp.renesas.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Simon Horman <horms+renesas@verge.net.au> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-17Revert "iommu/io-pgtable-arm: Check for v7s-incapable systems"Yong Wu1-4/+0
This reverts commit 82db33dc5e49fb625262d81125625d07a0d6184e. After the commit 29859aeb8a6e ("iommu/io-pgtable-arm-v7s: Abort allocation when table address overflows the PTE"), v7s will return fail if the page table allocation isn't expected. this PHYS_OFFSET check is unnecessary now. And this check may lead to fail. For example, If CONFIG_RANDOMIZE_BASE is enabled, the "memstart_addr" will be updated randomly, then the PHYS_OFFSET may be random. Reported-by: CK Hu <ck.hu@mediatek.com> Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-13dma-mapping: bypass indirect calls for dma-directChristoph Hellwig1-12/+1
Avoid expensive indirect calls in the fast path DMA mapping operations by directly calling the dma_direct_* ops if we are using the directly mapped DMA operations. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Jesper Dangaard Brouer <brouer@redhat.com> Tested-by: Jesper Dangaard Brouer <brouer@redhat.com> Tested-by: Tony Luck <tony.luck@intel.com>
2018-12-11iommu/vt-d: Remove deferred invalidationLu Baolu2-46/+0
Deferred invalidation is an ECS specific feature. It will not be supported when IOMMU works in scalable mode. As we deprecated the ECS support, remove deferred invalidation and cleanup the code. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Cc: Liu Yi L <yi.l.liu@intel.com> Cc: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Shared virtual address in scalable modeLu Baolu4-80/+19
This patch enables the current SVA (Shared Virtual Address) implementation to work in the scalable mode. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Add first level page table interfaceLu Baolu2-0/+91
This adds an interface to setup the PASID entries for first level page table translation. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Setup context and enable RID2PASID supportLu Baolu2-30/+107
This patch enables the translation for requests without PASID in the scalable mode by setting up the root and context entries. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Pass pasid table to context mappingLu Baolu1-3/+11
So that the pasid related info, such as the pasid table and the maximum of pasid could be used during setting up scalable mode context. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Cc: Liu Yi L <yi.l.liu@intel.com> Cc: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Setup pasid entry for RID2PASID supportLu Baolu2-0/+21
when the scalable mode is enabled, there is no second level page translation pointer in the context entry any more (for DMA request without PASID). Instead, a new RID2PASID field is introduced in the context entry. Software can choose any PASID value to set RID2PASID and then setup the translation in the corresponding PASID entry. Upon receiving a DMA request without PASID, hardware will firstly look at this RID2PASID field and then treat this request as a request with a pasid value specified in RID2PASID field. Though software is allowed to use any PASID for the RID2PASID, we will always use the PASID 0 as a sort of design decision. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Add second level page table interfaceLu Baolu3-1/+289
This adds the interfaces to setup or tear down the structures for second level page table translations. This includes types of second level only translation and pass through. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Reserve a domain id for FL and PT modesLu Baolu2-0/+16
Vt-d spec rev3.0 (section 6.2.3.1) requires that each pasid entry for first-level or pass-through translation should be programmed with a domain id different from those used for second-level or nested translation. It is recommended that software could use a same domain id for all first-only and pass-through translations. This reserves a domain id for first-level and pass-through translations. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Cc: Liu Yi L <yi.l.liu@intel.com> Cc: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Add 256-bit invalidation descriptor supportLu Baolu3-59/+114
Intel vt-d spec rev3.0 requires software to use 256-bit descriptors in invalidation queue. As the spec reads in section 6.5.2: Remapping hardware supporting Scalable Mode Translations (ECAP_REG.SMTS=1) allow software to additionally program the width of the descriptors (128-bits or 256-bits) that will be written into the Queue. Software should setup the Invalidation Queue for 256-bit descriptors before progra- mming remapping hardware for scalable-mode translation as 128-bit descriptors are treated as invalid descriptors (see Table 21 in Section 6.5.2.10) in scalable-mode. This patch adds 256-bit invalidation descriptor support if the hardware presents scalable mode capability. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Move page table helpers into headerLu Baolu1-43/+0
So that they could also be used in other source files. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Cc: Liu Yi L <yi.l.liu@intel.com> Cc: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Manage scalalble mode PASID tablesLu Baolu4-31/+97
In scalable mode, pasid structure is a two level table with a pasid directory table and a pasid table. Any pasid entry can be identified by a pasid value in below way. 1 9 6 5 0 .-----------------------.-------. | PASID | | '-----------------------'-------' .-------------. | | | | | | | | | | | | | .-----------. | .-------------. | | | |----->| PASID Entry | | | | | '-------------' | | | |Plus | | | .-----------. | | | |---->| DIR Entry |-------->| | | '-----------' '-------------' .---------. |Plus | | | Context | | | | | Entry |------->| | '---------' '-----------' This changes the pasid table APIs to support scalable mode PASID directory and PASID table. It also adds a helper to get the PASID table entry according to the pasid value. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/vt-d: Enumerate the scalable mode capabilityLu Baolu1-47/+17
The Intel vt-d spec rev3.0 introduces a new translation mode called scalable mode, which enables PASID-granular translations for first level, second level, nested and pass-through modes. At the same time, the previous Extended Context (ECS) mode is deprecated (no production ever implements ECS). This patch adds enumeration for Scalable Mode and removes the deprecated ECS enumeration. It provides a boot time option to disable scalable mode even hardware claims to support it. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Ashok Raj <ashok.raj@intel.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-11iommu/dma: Use NUMA aware memory allocations in __iommu_dma_alloc_pages()Ganapatrao Kulkarni1-10/+10
Change function __iommu_dma_alloc_pages() to allocate pages for DMA from respective device NUMA node. The ternary operator which would be for alloc_pages_node() is tidied along with this. The motivation for this change is to have a policy for page allocation consistent with direct DMA mapping, which attempts to allocate pages local to the device, as mentioned in [1]. In addition, for certain workloads it has been observed a marginal performance improvement. The patch caused an observation of 0.9% average throughput improvement for running tcrypt with HiSilicon crypto engine. We also include a modification to use kvzalloc() for kzalloc()/vzalloc() combination. [1] https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1692998.html Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com> [JPG: Added kvzalloc(), drop pages ** being device local, remove ternary operator, update message] Signed-off-by: John Garry <john.garry@huawei.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-12-10iommu/arm-smmu: Add support for qcom,smmu-v2 variantVivek Gautam1-0/+3
qcom,smmu-v2 is an arm,smmu-v2 implementation with specific clock and power requirements. On msm8996, multiple cores, viz. mdss, video, etc. use this smmu. On sdm845, this smmu is used with gpu. Add bindings for the same. Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org> Reviewed-by: Rob Herring <robh@kernel.org> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10iommu/arm-smmu: Add the device_link between masters and smmuSricharan R1-0/+3
Finally add the device link between the master device and smmu, so that the smmu gets runtime enabled/disabled only when the master needs it. This is done from add_device callback which gets called once when the master is added to the smmu. Signed-off-by: Sricharan R <sricharan@codeaurora.org> Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10iommu/arm-smmu: Invoke pm_runtime across the driverSricharan R1-10/+98
Enable pm-runtime on devices that implement a pm domain. Then, add pm runtime hooks to several iommu_ops to power cycle the smmu device for explicit TLB invalidation requests, and register space accesses, etc. We need these hooks when the smmu, linked to its master through device links, has to be powered-up without the master device being in context. Signed-off-by: Sricharan R <sricharan@codeaurora.org> [vivek: Cleanup pm runtime calls] Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10iommu/arm-smmu: Add pm_runtime/sleep opsSricharan R1-3/+55
The smmu needs to be functional only when the respective master's using it are active. The device_link feature helps to track such functional dependencies, so that the iommu gets powered when the master device enables itself using pm_runtime. So by adapting the smmu driver for runtime pm, above said dependency can be addressed. This patch adds the pm runtime/sleep callbacks to the driver and the corresponding bulk clock handling for all the clocks needed by smmu. Also, while we enable the runtime pm, add a pm sleep suspend callback that pushes devices to low power state by turning the clocks off in a system sleep. Add corresponding clock enable path in resume callback as well. Signed-off-by: Sricharan R <sricharan@codeaurora.org> Signed-off-by: Archit Taneja <architt@codeaurora.org> [Thor: Rework to get clocks from device tree] Signed-off-by: Thor Thayer <thor.thayer@linux.intel.com> [vivek: rework for clock and pm ops] Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Thor Thayer <thor.thayer@linux.intel.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10iommu/arm-smmu-v3: Use explicit mb() when moving cons pointerWill Deacon1-1/+7
After removing an entry from a queue (e.g. reading an event in arm_smmu_evtq_thread()) it is necessary to advance the MMIO consumer pointer to free the queue slot back to the SMMU. A memory barrier is required here so that all reads targetting the queue entry have completed before the consumer pointer is updated. The implementation of queue_inc_cons() relies on a writel() to complete the previous reads, but this is incorrect because writel() is only guaranteed to complete prior writes. This patch replaces the call to writel() with an mb(); writel_relaxed() sequence, which gives us the read->write ordering which we require. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10iommu/arm-smmu-v3: Avoid memory corruption from Hisilicon MSI payloadsZhen Lei1-1/+5
The GITS_TRANSLATER MMIO doorbell register in the ITS hardware is architected to be 4 bytes in size, yet on hi1620 and earlier, Hisilicon have allocated the adjacent 4 bytes to carry some IMPDEF sideband information which results in an 8-byte MSI payload being delivered when signalling an interrupt: MSIAddr: |----4bytes----|----4bytes----| | MSIData | IMPDEF | This poses no problem for the ITS hardware because the adjacent 4 bytes are reserved in the memory map. However, when delivering MSIs to memory, as we do in the SMMUv3 driver for signalling the completion of a SYNC command, the extended payload will corrupt the 4 bytes adjacent to the "sync_count" member in struct arm_smmu_device. Fortunately, the current layout allocates these bytes to padding, but this is fragile and we should make this explicit. Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> [will: Rewrote commit message and comment] Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10iommu/arm-smmu-v3: Fix big-endian CMD_SYNC writesRobin Murphy1-1/+7
When we insert the sync sequence number into the CMD_SYNC.MSIData field, we do so in CPU-native byte order, before writing out the whole command as explicitly little-endian dwords. Thus on big-endian systems, the SMMU will receive and write back a byteswapped version of sync_nr, which would be perfect if it were targeting a similarly-little-endian ITS, but since it's actually writing back to memory being polled by the CPUs, they're going to end up seeing the wrong thing. Since the SMMU doesn't care what the MSIData actually contains, the minimal-overhead solution is to simply add an extra byteswap initially, such that it then writes back the big-endian format directly. Cc: <stable@vger.kernel.org> Fixes: 37de98f8f1cf ("iommu/arm-smmu-v3: Use CMD_SYNC completion MSI") Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
generated by cgit v1.2.3 (git 2.39.0) at 2024-12-03 01:21:01 +0300