summaryrefslogtreecommitdiff
path: root/arch/arc/mm/dma.c
AgeCommit message (Collapse)AuthorFilesLines
2018-09-04ARC: don't check for HIGHMEM pages in arch_dma_allocEugeniy Paltsev1-18/+14
__GFP_HIGHMEM flag is cleared by upper layer functions (in include/linux/dma-mapping.h) so we'll never get a __GFP_HIGHMEM flag in arch_dma_alloc gfp argument. That's why alloc_pages will never return highmem page here. Get rid of highmem pages handling and cleanup arch_dma_alloc and arch_dma_free functions. Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2018-09-04ARC: dma [IOC] Enable per device io coherencyEugeniy Paltsev1-25/+29
So far the IOC treatment was global on ARC, being turned on (or off) for all devices in the system. With this patch, this can now be done per device using the "dma-coherent" DT property; IOW with this patch we can use both HW-coherent and regular DMA peripherals simultaneously. The changes involved are too many so enlisting the summary below: 1. common code calls ARC arch_setup_dma_ops() per device. 2. For coherent dma (IOC) it plugs in generic @dma_direct_ops which doesn't need any arch specific backend: No need for any explicit cache flushes or MMU mappings to provide for uncached access - dma_(map|sync)_single* return early as corresponding dma ops callbacks are NULL in generic code. So arch_sync_dma_*() -> dma_cache_*() need not handle the coherent dma case, hence drop ARC __dma_cache_*_ioc() which were no-op anyways 3. For noncoherent dma (non IOC) generic @dma_noncoherent_ops is used which in turns calls ARC specific routines - arch_dma_alloc() no longer checks for @ioc_enable since this is called only for !IOC case. Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> [vgupta: rewrote changelog]
2018-07-27ARC: dma [non IOC]: fix arc_dma_sync_single_for_(device|cpu)Eugeniy Paltsev1-2/+47
ARC backend for dma_sync_single_for_(device|cpu) was broken as it was not honoring the @dir argument and simply forcing it based on the call: - arc_dma_sync_single_for_device(dir) assumed DMA_TO_DEVICE (cache wback) - arc_dma_sync_single_for_cpu(dir) assumed DMA_FROM_DEVICE (cache inv) This is not true given the DMA API programming model and has been discussed here [1] in some detail. Interestingly while the deficiency has been there forever, it only started showing up after 4.17 dma common ops rework, commit a8eb92d02dd7 ("arc: fix arc_dma_{map,unmap}_page") which wired up these calls under the more commonly used dma_map_page API triggering the issue. [1]: https://lkml.org/lkml/2018/5/18/979 Fixes: commit a8eb92d02dd7 ("arc: fix arc_dma_{map,unmap}_page") Cc: stable@kernel.org # v4.17+ Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> [vgupta: reworked changelog]
2018-05-19arc: use generic dma_noncoherent_opsChristoph Hellwig1-127/+13
Switch to the generic noncoherent direct mapping implementation. Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Alexey Brodkin <abrodkin@synopsys.com> Acked-by: Vineet Gupta <vgupta@synopsys.com>
2018-05-19arc: fix arc_dma_{map,unmap}_pageChristoph Hellwig1-25/+2
These functions should perform the same cache synchronoization as calling arc_dma_sync_single_for_{cpu,device} in addition to doing any required address translation or mapping [1]. Ensure they actually do that by calling arc_dma_sync_single_for_{cpu,device} instead of passing the dir argument along to _dma_cache_sync. The now unused _dma_cache_sync function is removed as well. [1] in fact various drivers rely on that by passing DMA_ATTR_SKIP_CPU_SYNC to the map/unmap routines and doing the cache synchronization manually. Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Alexey Brodkin <abrodkin@synopsys.com> Acked-by: Vineet Gupta <vgupta@synopsys.com>
2018-05-19arc: fix arc_dma_sync_sg_for_{cpu,device}Christoph Hellwig1-2/+3
These functions should perform the same functionality as calling arc_dma_sync_single_for_{cpu,device} on each S/G list element. Ensure they actually do that by calling arc_dma_sync_single_for_{cpu,device}. Otherwise we could be passing a different dir argument. Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Alexey Brodkin <abrodkin@synopsys.com> Acked-by: Vineet Gupta <vgupta@synopsys.com>
2018-05-19arc: simplify arc_dma_sync_single_for_{cpu,device}Christoph Hellwig1-12/+12
Remove the indirection through _dma_cache_sync. Also move the functions up a bit in the source file as we'll need them in more places soon. Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Alexey Brodkin <abrodkin@synopsys.com> Acked-by: Vineet Gupta <vgupta@synopsys.com>
2018-01-09arc: remove CONFIG_ARC_PLAT_NEEDS_PHYS_TO_DMAChristoph Hellwig1-7/+7
We always use the stub definitions, so remove the unused other code. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Vineet Gupta <vgupta@synopsys.com>
2017-08-21Merge tag 'arc-4.13-rc7-fixes' of ↵Linus Torvalds1-0/+45
git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc Pull ARC fixes from Vineet Gupta: - PAE40 related updates - SLC errata for region ops - intc line masking by default * tag 'arc-4.13-rc7-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc: arc: Mask individual IRQ lines during core INTC init ARCv2: PAE40: set MSB even if !CONFIG_ARC_HAS_PAE40 but PAE exists in SoC ARCv2: PAE40: Explicitly set MSB counterpart of SLC region ops addresses ARC: dma: implement dma_unmap_page and sg variant ARCv2: SLC: Make sure busy bit is set properly for region ops ARC: [plat-sim] Include this platform unconditionally ARC: [plat-axs10x]: prepare dts files for enabling PAE40 on axs103 ARC: defconfig: Cleanup from old Kconfig options
2017-08-04ARC: dma: implement dma_unmap_page and sg variantVineet Gupta1-0/+45
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2017-07-20dma-coherent: introduce interface for default DMA poolVladimir Murzin1-1/+1
Christoph noticed [1] that default DMA pool in current form overload the DMA coherent infrastructure. In reply, Robin suggested [2] to split the per-device vs. global pool interfaces, so allocation/release from default DMA pool is driven by dma ops implementation. This patch implements Robin's idea and provide interface to allocate/release/mmap the default (aka global) DMA pool. To make it clear that existing *_from_coherent routines work on per-device pool rename them to *_from_dev_coherent. [1] https://lkml.org/lkml/2017/7/7/370 [2] https://lkml.org/lkml/2017/7/7/431 Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Ralf Baechle <ralf@linux-mips.org> Suggested-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Andras Szemzo <sza@esh.hu> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
2017-01-24treewide: Constify most dma_map_ops structuresBart Van Assche1-1/+1
Most dma_map_ops structures are never modified. Constify these structures such that these can be write-protected. This patch has been generated as follows: git grep -l 'struct dma_map_ops' | xargs -d\\n sed -i \ -e 's/struct dma_map_ops/const struct dma_map_ops/g' \ -e 's/const struct dma_map_ops {/struct dma_map_ops {/g' \ -e 's/^const struct dma_map_ops;$/struct dma_map_ops;/' \ -e 's/const const struct dma_map_ops /const struct dma_map_ops /g'; sed -i -e 's/const \(struct dma_map_ops intel_dma_ops\)/\1/' \ $(git grep -l 'struct dma_map_ops intel_dma_ops'); sed -i -e 's/const \(struct dma_map_ops dma_iommu_ops\)/\1/' \ $(git grep -l 'struct dma_map_ops' | grep ^arch/powerpc); sed -i -e '/^struct vmd_dev {$/,/^};$/ s/const \(struct dma_map_ops[[:blank:]]dma_ops;\)/\1/' \ -e '/^static void vmd_setup_dma_ops/,/^}$/ s/const \(struct dma_map_ops \*dest\)/\1/' \ -e 's/const \(struct dma_map_ops \*dest = \&vmd->dma_ops\)/\1/' \ drivers/pci/host/*.c sed -i -e '/^void __init pci_iommu_alloc(void)$/,/^}$/ s/dma_ops->/intel_dma_ops./' arch/ia64/kernel/pci-dma.c sed -i -e 's/static const struct dma_map_ops sn_dma_ops/static struct dma_map_ops sn_dma_ops/' arch/ia64/sn/pci/pci_dma.c sed -i -e 's/(const struct dma_map_ops \*)//' drivers/misc/mic/bus/vop_bus.c Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: David Woodhouse <dwmw2@infradead.org> Cc: Juergen Gross <jgross@suse.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: linux-arch@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: Russell King <linux@armlinux.org.uk> Cc: x86@kernel.org Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-12-15arch/arc: add option to skip sync on DMA mappingAlexander Duyck1-1/+4
Patch series "Add support for DMA writable pages being writable by the network stack", v3. The first 19 patches in the set add support for the DMA attribute DMA_ATTR_SKIP_CPU_SYNC on multiple platforms/architectures. This is needed so that we can flag the calls to dma_map/unmap_page so that we do not invalidate cache lines that do not currently belong to the device. Instead we have to take care of this in the driver via a call to sync_single_range_for_cpu prior to freeing the Rx page. Patch 20 adds support for dma_map_page_attrs and dma_unmap_page_attrs so that we can unmap and map a page using the DMA_ATTR_SKIP_CPU_SYNC attribute. Patch 21 adds support for freeing a page that has multiple references being held by a single caller. This way we can free page fragments that were allocated by a given driver. The last 2 patches use these updates in the igb driver, and lay the groundwork to allow for us to reimplement the use of build_skb. This patch (of 23): This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to avoid invoking cache line invalidation if the driver will just handle it later via a sync_for_cpu or sync_for_device call. Link: http://lkml.kernel.org/r/20161110113419.76501.38491.stgit@ahduyck-blue-test.jf.intel.com Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com> Acked-by: Vineet Gupta <vgupta@synopsys.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-11-03arc: Implement arch-specific dma_map_ops.mmapAlexey Brodkin1-0/+26
We used to use generic implementation of dma_map_ops.mmap which is dma_common_mmap() but that only worked for simpler cached mappings when vaddr = paddr. If a driver requests uncached DMA buffer kernel maps it to virtual address so that MMU gets involved and page uncached status takes into account. In that case usage of dma_common_mmap() lead to mapping of vaddr to vaddr for user-space which is obviously wrong. For more detals please refer to verbose explanation here [1]. So here we implement our own version of mmap() which always deals with dma_addr and maps underlying memory to user-space properly (note that DMA buffer mapped to user-space is always uncached because there's no way to properly manage cache from user-space). [1] https://lkml.org/lkml/2016/10/26/973 Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: <stable@vger.kernel.org> #4.5+ Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-10-24ARCv2: IOC: use @ioc_enable not @ioc_exist where intendedVineet Gupta1-2/+2
if user disables IOC from debugger at startup (by clearing @ioc_enable), @ioc_exists is cleared too. This means boot prints don't capture the fact that IOC was present but disabled which could be misleading. So invert how we use @ioc_enable and @ioc_exists and make it more canonical. @ioc_exists represent whether hardware is present or not and stays same whether enabled or not. @ioc_enable is still user driven, but will be auto-disabled if IOC hardware is not present, i.e. if @ioc_exist=0. This is opposite to what we were doing before, but much clearer. This means @ioc_enable is now the "exported" toggle in rest of code such as dma mapping API. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-08-04dma-mapping: use unsigned long for dma_attrsKrzysztof Kozlowski1-6/+6
The dma-mapping core and the implementations do not change the DMA attributes passed by pointer. Thus the pointer can point to const data. However the attributes do not have to be a bitfield. Instead unsigned long will do fine: 1. This is just simpler. Both in terms of reading the code and setting attributes. Instead of initializing local attributes on the stack and passing pointer to it to dma_set_attr(), just set the bits. 2. It brings safeness and checking for const correctness because the attributes are passed by value. Semantic patches for this change (at least most of them): virtual patch virtual context @r@ identifier f, attrs; @@ f(..., - struct dma_attrs *attrs + unsigned long attrs , ...) { ... } @@ identifier r.f; @@ f(..., - NULL + 0 ) and // Options: --all-includes virtual patch virtual context @r@ identifier f, attrs; type t; @@ t f(..., struct dma_attrs *attrs); @@ identifier r.f; @@ f(..., - NULL + 0 ) Link: http://lkml.kernel.org/r/1468399300-5399-2-git-send-email-k.kozlowski@samsung.com Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com> Acked-by: Vineet Gupta <vgupta@synopsys.com> Acked-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Hans-Christian Noren Egtvedt <egtvedt@samfundet.no> Acked-by: Mark Salter <msalter@redhat.com> [c6x] Acked-by: Jesper Nilsson <jesper.nilsson@axis.com> [cris] Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> [drm] Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com> Acked-by: Joerg Roedel <jroedel@suse.de> [iommu] Acked-by: Fabien Dessenne <fabien.dessenne@st.com> [bdisp] Reviewed-by: Marek Szyprowski <m.szyprowski@samsung.com> [vb2-core] Acked-by: David Vrabel <david.vrabel@citrix.com> [xen] Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> [xen swiotlb] Acked-by: Joerg Roedel <jroedel@suse.de> [iommu] Acked-by: Richard Kuo <rkuo@codeaurora.org> [hexagon] Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> [m68k] Acked-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> [s390] Acked-by: Bjorn Andersson <bjorn.andersson@linaro.org> Acked-by: Hans-Christian Noren Egtvedt <egtvedt@samfundet.no> [avr32] Acked-by: Vineet Gupta <vgupta@synopsys.com> [arc] Acked-by: Robin Murphy <robin.murphy@arm.com> [arm64 and dma-iommu] Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-20ARC: dma: fix address translation in arc_dma_freeVladimir Kondratiev1-1/+2
page should be calculated using physical address. If platform uses non-trivial dma-to-phys memory translation, dma_handle should be converted to physicval address before calculation of page. Failing to do so results in struct page * pointing to wrong or non-existent memory. Fixes: f2e3d55397ff ("ARC: dma: reintroduce platform specific dma<->phys") Cc: stable@vger.kernel.org #4.6+ Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@intel.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-05-30Fix typosAndrea Gelmini1-1/+1
Signed-off-by: Andrea Gelmini <andrea.gelmini@gelma.net> Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-03-19ARC: dma: reintroduce platform specific dma<->physVineet Gupta1-5/+4
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-03-19ARC: dma: ioremap: use phys_addr_t consistenctly in code pathsVineet Gupta1-6/+6
To support dma in physical memory beyond 4GB with PAE40 Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-03-19ARC: dma: pass_phys() not sg_virt() to cache opsVineet Gupta1-2/+2
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-03-19ARC: dma: non-coherent pages need V-P mapping if in HIGHMEMVineet Gupta1-15/+32
Previously a non-coherent page (hardware IOC or simply driver needs) could be handled by cpu with paddr alone (kvaddr used to be needed for coherent mappings to enforce uncached semantics via a MMU mapping). Now however such a page might still require a V-P mapping if it was in physical address space > 32bits due to PAE40, which the CPU can't access directly with a paddr So decouple decision of kvaddr allocation from type of alloc request (coh/non-coh) Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-03-19ARC: dma: Use struct page based page allocator helpersVineet Gupta1-8/+17
vs. the ones which reutne void *, so that we can handle pages > 4GB in subsequent patches Also plug a potential page leak in case ioremap fails Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2016-01-21arc: convert to dma_map_opsChristoph Hellwig1-47/+105
[vgupta@synopsys.com: ARC: dma mapping fixes #2] Signed-off-by: Christoph Hellwig <hch@lst.de> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Christian Borntraeger <borntraeger@de.ibm.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Sebastian Ott <sebott@linux.vnet.ibm.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> Cc: Carlos Palminha <CARLOS.PALMINHA@synopsys.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-08-21ARC: Eliminate some ARCv2 specific code for ARCompact buildVineet Gupta1-2/+2
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2015-08-20ARCv2: Support IO Coherency and permutations involving L1 and L2 cachesAlexey Brodkin1-0/+18
In case of ARCv2 CPU there're could be following configurations that affect cache handling for data exchanged with peripherals via DMA: [1] Only L1 cache exists [2] Both L1 and L2 exist, but no IO coherency unit [3] L1, L2 caches and IO coherency unit exist Current implementation takes care of [1] and [2]. Moreover support of [2] is implemented with run-time check for SLC existence which is not super optimal. This patch introduces support of [3] and rework of DMA ops usage. Instead of doing run-time check every time a particular DMA op is executed we'll have 3 different implementations of DMA ops and select appropriate one during init. As for IOC support for it we need: [a] Implement empty DMA ops because IOC takes care of cache coherency with DMAed data [b] Route dma_alloc_coherent() via dma_alloc_noncoherent() This is required to make IOC work in first place and also serves as optimization as LD/ST to coherent buffers can be srviced from caches w/o going all the way to memory Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> [vgupta: -Added some comments about IOC gains -Marked dma ops as static, -Massaged changelog a bit] Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2015-07-06ARC: Don't memzero twice in dma_alloc_coherent for __GFP_ZEROVineet Gupta1-2/+2
alloc_pages_exact() get gfp flags and handle zero'ing already And while it, fix the case where ioremap fails: return rightaway. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2015-06-25ARCv2: SLC: Handle explcit flush for DMA ops (w/o IO-coherency)Vineet Gupta1-0/+12
L2 cache on ARCHS processors is called SLC (System Level Cache) For working DMA (in absence of hardware assisted IO Coherency) we need to manage SLC explicitly when buffers transition between cpu and controllers. Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2015-06-19ARC: remove the unused platform helpers from dma mapping APIVineet Gupta1-8/+4
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-02-15ARC: I/O and DMA MappingsVineet Gupta1-0/+94
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
generated by cgit v1.2.3 (git 2.39.0) at 2025-03-18 00:39:44 +0300