kernel/linux.git/drivers/dma/dw, branch v5.10.257

dmaengine: dw: Select only supported masters for ACPI devices

2025-01-09T12:25:03+00:00

[ Upstream commit f0e870a0e9c5521f2952ea9f3ea9d3d122631a89 ] The recently submitted fix-commit revealed a problem in the iDMA 32-bit platform code. Even though the controller supported only a single master the dw_dma_acpi_filter() method hard-coded two master interfaces with IDs 0 and 1. As a result the sanity check implemented in the commit b336268dde75 ("dmaengine: dw: Add peripheral bus width verification") got incorrect interface data width and thus prevented the client drivers from configuring the DMA-channel with the EINVAL error returned. E.g., the next error was printed for the PXA2xx SPI controller driver trying to configure the requested channels: > [ 164.525604] pxa2xx_spi_pci 0000:00:07.1: DMA slave config failed > [ 164.536105] pxa2xx_spi_pci 0000:00:07.1: failed to get DMA TX descriptor > [ 164.543213] spidev spi-SPT0001:00: SPI transfer failed: -16 The problem would have been spotted much earlier if the iDMA 32-bit controller supported more than one master interfaces. But since it supports just a single master and the iDMA 32-bit specific code just ignores the master IDs in the CTLLO preparation method, the issue has been gone unnoticed so far. Fix the problem by specifying the default master ID for both memory and peripheral devices in the driver data. Thus the issue noticed for the iDMA 32-bit controllers will be eliminated and the ACPI-probed DW DMA controllers will be configured with the correct master ID by default. Cc: stable@vger.kernel.org Fixes: b336268dde75 ("dmaengine: dw: Add peripheral bus width verification") Fixes: 199244d69458 ("dmaengine: dw: add support of iDMA 32-bit hardware") Reported-by: Ferry Toth Closes: https://lore.kernel.org/dmaengine/ZuXbCKUs1iOqFu51@black.fi.intel.com/ Reported-by: Andy Shevchenko Closes: https://lore.kernel.org/dmaengine/ZuXgI-VcHpMgbZ91@black.fi.intel.com/ Tested-by: Ferry Toth Signed-off-by: Andy Shevchenko Link: https://lore.kernel.org/r/20241104095142.157925-1-andriy.shevchenko@linux.intel.com Signed-off-by: Vinod Koul Signed-off-by: Sasha Levin

dmaengine: dw: Add memory bus width verification

2024-09-04T11:17:46+00:00

[ Upstream commit d04b21bfa1c50a2ade4816cab6fdc91827b346b1 ] Currently in case of the DEV_TO_MEM or MEM_TO_DEV DMA transfers the memory data width (single transfer width) is determined based on the buffer length, buffer base address or DMA master-channel max address width capability. It isn't enough in case of the channel disabling prior the block transfer is finished. Here is what DW AHB DMA IP-core databook says regarding the port suspension (DMA-transfer pause) implementation in the controller: "When CTLx.SRC_TR_WIDTH < CTLx.DST_TR_WIDTH and the CFGx.CH_SUSP bit is high, the CFGx.FIFO_EMPTY is asserted once the contents of the FIFO do not permit a single word of CTLx.DST_TR_WIDTH to be formed. However, there may still be data in the channel FIFO, but not enough to form a single transfer of CTLx.DST_TR_WIDTH. In this scenario, once the channel is disabled, the remaining data in the channel FIFO is not transferred to the destination peripheral." So in case if the port gets to be suspended and then disabled it's possible to have the data silently discarded even though the controller reported that FIFO is empty and the CTLx.BLOCK_TS indicated the dropped data already received from the source device. This looks as if the data somehow got lost on a way from the peripheral device to memory and causes problems for instance in the DW APB UART driver, which pauses and disables the DMA-transfer as soon as the recv data timeout happens. Here is the way it looks: Memory <------- DMA FIFO <------ UART FIFO <---------------- UART DST_TR_WIDTH -+--------| | | | | | | No more data Current lvl -+--------| |---------+- DMA-burst lvl | | |---------+- Leftover data | | |---------+- SRC_TR_WIDTH -+--------+-------+---------+ In the example above: no more data is getting received over the UART port and BLOCK_TS is not even close to be fully received; some data is left in the UART FIFO, but not enough to perform a bursted DMA-xfer to the DMA FIFO; some data is left in the DMA FIFO, but not enough to be passed further to the system memory in a single transfer. In this situation the 8250 UART driver catches the recv timeout interrupt, pauses the DMA-transfer and terminates it completely, after which the IRQ handler manually fetches the leftover data from the UART FIFO into the recv-buffer. But since the DMA-channel has been disabled with the data left in the DMA FIFO, that data will be just discarded and the recv-buffer will have a gap of the "current lvl" size in the recv-buffer at the tail of the lately received data portion. So the data will be lost just due to the misconfigured DMA transfer. Note this is only relevant for the case of the transfer suspension and _disabling_. No problem will happen if the transfer will be re-enabled afterwards or the block transfer is fully completed. In the later case the "FIFO flush mode" will be executed at the transfer final stage in order to push out the data left in the DMA FIFO. In order to fix the denoted problem the DW AHB DMA-engine driver needs to make sure that the _bursted_ source transfer width is greater or equal to the single destination transfer (note the HW databook describes more strict constraint than actually required). Since the peripheral-device side is prescribed by the client driver logic, the memory-side can be only used for that. The solution can be easily implemented for the DEV_TO_MEM transfers just by adjusting the memory-channel address width. Sadly it's not that easy for the MEM_TO_DEV transfers since the mem-to-dma burst size is normally dynamically determined by the controller. So the only thing that can be done is to make sure that memory-side address width is greater than the peripheral device address width. Fixes: a09820043c9e ("dw_dmac: autoconfigure data_width or get it via platform data") Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20240802075100.6475-3-fancer.lancer@gmail.com Signed-off-by: Vinod Koul Signed-off-by: Sasha Levin

dmaengine: dw: Add peripheral bus width verification

2024-09-04T11:17:46+00:00

[ Upstream commit b336268dde75cb09bd795cb24893d52152a9191f ] Currently the src_addr_width and dst_addr_width fields of the dma_slave_config structure are mapped to the CTLx.SRC_TR_WIDTH and CTLx.DST_TR_WIDTH fields of the peripheral bus side in order to have the properly aligned data passed to the target device. It's done just by converting the passed peripheral bus width to the encoded value using the __ffs() function. This implementation has several problematic sides: 1. __ffs() is undefined if no bit exist in the passed value. Thus if the specified addr-width is DMA_SLAVE_BUSWIDTH_UNDEFINED, __ffs() may return unexpected value depending on the platform-specific implementation. 2. DW AHB DMA-engine permits having the power-of-2 transfer width limited by the DMAH_Mk_HDATA_WIDTH IP-core synthesize parameter. Specifying bus-width out of that constraints scope will definitely cause unexpected result since the destination reg will be only partly touched than the client driver implied. Let's fix all of that by adding the peripheral bus width verification method and calling it in dwc_config() which is supposed to be executed before preparing any transfer. The new method will make sure that the passed source or destination address width is valid and if undefined then the driver will just fallback to the 1-byte width transfer. Fixes: 029a40e97d0d ("dmaengine: dw: provide DMA capabilities") Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20240802075100.6475-2-fancer.lancer@gmail.com Signed-off-by: Vinod Koul Signed-off-by: Sasha Levin

dmaengine: dw: Make it dependent to HAS_IOMEM

2021-04-21T11:00:50+00:00

[ Upstream commit 88cd1d6191b13689094310c2405394e4ce36d061 ] Some architectures do not provide devm_*() APIs. Hence make the driver dependent on HAVE_IOMEM. Fixes: dbde5c2934d1 ("dw_dmac: use devm_* functions to simplify code") Reported-by: kernel test robot Signed-off-by: Andy Shevchenko Acked-by: Viresh Kumar Link: https://lore.kernel.org/r/20210324141757.24710-1-andriy.shevchenko@linux.intel.com Signed-off-by: Vinod Koul Signed-off-by: Sasha Levin

dmaengine: dw: convert tasklets to use new tasklet_setup() API

2020-09-18T06:48:11+00:00

In preparation for unconditionally passing the struct tasklet_struct pointer to all tasklet callbacks, switch to using the new tasklet_setup() and from_tasklet() to pass the tasklet pointer explicitly. Signed-off-by: Romain Perier Signed-off-by: Allen Pais Link: https://lore.kernel.org/r/20200831103542.305571-6-allen.lkml@gmail.com Signed-off-by: Vinod Koul

dmaengine: dw: Add DMA-channels mask cell support

2020-08-17T06:28:31+00:00

DW DMA IP-core provides a way to synthesize the DMA controller with channels having different parameters like maximum burst-length, multi-block support, maximum data width, etc. Those parameters both explicitly and implicitly affect the channels performance. Since DMA slave devices might be very demanding to the DMA performance, let's provide a functionality for the slaves to be assigned with DW DMA channels, which performance according to the platform engineer fulfill their requirements. After this patch is applied it can be done by passing the mask of suitable DMA-channels either directly in the dw_dma_slave structure instance or as a fifth cell of the DMA DT-property. If mask is zero or not provided, then there is no limitation on the channels allocation. For instance Baikal-T1 SoC is equipped with a DW DMAC engine, which first two channels are synthesized with max burst length of 16, while the rest of the channels have been created with max-burst-len=4. It would seem that the first two channels must be faster than the others and should be more preferable for the time-critical DMA slave devices. In practice it turned out that the situation is quite the opposite. The channels with max-burst-len=4 demonstrated a better performance than the channels with max-burst-len=16 even when they both had been initialized with the same settings. The performance drop of the first two DMA-channels made them unsuitable for the DW APB SSI slave device. No matter what settings they are configured with, full-duplex SPI transfers occasionally experience the Rx FIFO overflow. It means that the DMA-engine doesn't keep up with incoming data pace even though the SPI-bus is enabled with speed of 25MHz while the DW DMA controller is clocked with 50MHz signal. There is no such problem has been noticed for the channels synthesized with max-burst-len=4. Signed-off-by: Serge Semin Link: https://lore.kernel.org/r/20200731200826.9292-6-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul

dmaengine: dw: Ignore burst setting for memory peripherals

2020-08-17T06:28:31+00:00

According to the DW DMA controller Databook 2.18b (page 40 "3.5 Memory Peripherals") memory peripherals don't have handshaking interface connected to the controller, therefore they can never be a flow controller. Since the CTLx.SRC_MSIZE and CTLx.DEST_MSIZE are properties valid only for peripherals with a handshaking interface, we can freely zero these fields out if the memory peripheral is selected to be the source or the destination of the DMA transfers. Note according to the databook, length of burst transfers to memory is always equal to the number of data items available in a channel FIFO or data items required to complete the block transfer, whichever is smaller; length of burst transfers from memory is always equal to the space available in a channel FIFO or number of data items required to complete the block transfer, whichever is smaller. Signed-off-by: Serge Semin Link: https://lore.kernel.org/r/20200731200826.9292-5-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul

dmaengine: dw: Discard dlen from the dev-to-mem xfer width calculation

2020-08-17T06:28:31+00:00

Indeed in case of the DMA_DEV_TO_MEM DMA transfers it's enough to take the destination memory address and the destination master data width into account to calculate the CTLx.DST_TR_WIDTH setting of the memory peripheral. According to the DW DMAC IP-core Databook 2.18b (page 66, Example 5) at the and of a DMA transfer when the DMA-channel internal FIFO is left with data less than for a single destination burst transaction, the destination peripheral will enter the Single Transaction Region where the DW DMA controller can complete a block transfer to the destination using single transactions (non-burst transaction of CTLx.DST_TR_WIDTH bytes). If there is no enough data in the DMA-channel internal FIFO for even a single non-burst transaction of CTLx.DST_TR_WIDTH bytes, then the channel enters "FIFO flush mode". That mode is activated to empty the FIFO and flush the leftovers out to the memory peripheral. The flushing procedure is simple. The data is sent to the memory by means of a set of single transaction of CTLx.SRC_TR_WIDTH bytes. To sum up it's redundant to use the LLPs length to find out the CTLx.DST_TR_WIDTH parameter value, since each DMA transfer will be completed with the CTLx.SRC_TR_WIDTH bytes transaction if it is required. We suggest to remove the LLP entry length from the statement which calculates the memory peripheral DMA transaction width since it's redundant due to the feature described above. By doing so we'll improve the memory bus utilization and speed up the DMA-channel performance for DMA_DEV_TO_MEM DMA-transfers. Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20200731200826.9292-4-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul

dmaengine: dw: Activate FIFO-mode for memory peripherals only

2020-08-17T06:28:31+00:00

CFGx.FIFO_MODE field controls a DMA-controller "FIFO readiness" criterion. In other words it determines when to start pushing data out of a DW DMAC channel FIFO to a destination peripheral or from a source peripheral to the DW DMAC channel FIFO. Currently FIFO-mode is set to one for all DW DMAC channels. It means they are tuned to flush data out of FIFO (to a memory peripheral or by accepting the burst transaction requests) when FIFO is at least half-full (except at the end of the block transfer, when FIFO-flush mode is activated) and are configured to get data to the FIFO when it's at least half-empty. Such configuration is a good choice when there is no slave device involved in the DMA transfers. In that case the number of bursts per block is less than when CFGx.FIFO_MODE = 0 and, hence, the bus utilization will improve. But the latency of DMA transfers may increase when CFGx.FIFO_MODE = 1, since DW DMAC will wait for the channel FIFO contents to be either half-full or half-empty depending on having the destination or the source transfers. Such latencies might be dangerous in case if the DMA transfers are expected to be performed from/to a slave device. Since normally peripheral devices keep data in internal FIFOs, any latency at some critical moment may cause one being overflown and consequently losing data. This especially concerns a case when either a peripheral device is relatively fast or the DW DMAC engine is relatively slow with respect to the incoming data pace. In order to solve problems, which might be caused by the latencies described above, let's enable the FIFO half-full/half-empty "FIFO readiness" criterion only for DMA transfers with no slave device involved. Thanks to the commit 99ba8b9b0d97 ("dmaengine: dw: Initialize channel before each transfer") we can freely do that in the generic dw_dma_initialize_chan() method. Signed-off-by: Serge Semin Reviewed-by: Andy Shevchenko Link: https://lore.kernel.org/r/20200731200826.9292-3-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul

Merge branch 'for-linus' into fixes

2020-08-05T13:32:07+00:00

Signed-off-by: Vinod Koul Conflicts: drivers/dma/idxd/sysfs.c