kernel/linux.git/drivers/dma/dw/core.c, branch v6.12.80

dmaengine: dw: Unify ret-val local variables naming

2024-08-05T16:37:47+00:00

Currently there are two names utilized in the driver to keep the functions call status: ret and err. For the sake of unification convert to using the first version only. Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20240802075100.6475-7-fancer.lancer@gmail.com Signed-off-by: Vinod Koul

dmaengine: dw: Simplify max-burst calculation procedure

2024-08-05T16:37:47+00:00

In order to have a more coherent DW AHB DMA slave configuration method - dwc_config() - let's simplify the source and destination channel max-burst calculation procedure: 1. Create the max-burst verification method as it has been just done for the memory and peripheral address widths. Thus the dwc_config() method will turn to a set of the verification methods execution. 2. Since both the generic DW AHB DMA and Intel iDMA 32-bit engines support the power-of-2 bursts only, then the specified by the client driver max-burst values can be converted to being power-of-2 right in the max-burst verification method. 3. Since max-burst encoded value is required on the CTL_LO fields calculation stage, the encode_maxburst() callback can be easily dropped from the dw_dma structure meanwhile the encoding procedure will be executed right in the CTL_LO register value calculation. Thus the update will provide the next positive effects: the internal DMA-slave config structure will contain only the real DMA-transfer config values, which will be encoded to the DMA-controller register fields only when it's required on the buffer mapping; the redundant encode_maxburst() callback will be dropped simplifying the internal HW-abstraction API; dwc_config() will look more readable executing the verification functions one-by-one. Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20240802075100.6475-6-fancer.lancer@gmail.com Signed-off-by: Vinod Koul

dmaengine: dw: Add memory bus width verification

2024-08-05T16:37:47+00:00

Currently in case of the DEV_TO_MEM or MEM_TO_DEV DMA transfers the memory data width (single transfer width) is determined based on the buffer length, buffer base address or DMA master-channel max address width capability. It isn't enough in case of the channel disabling prior the block transfer is finished. Here is what DW AHB DMA IP-core databook says regarding the port suspension (DMA-transfer pause) implementation in the controller: "When CTLx.SRC_TR_WIDTH < CTLx.DST_TR_WIDTH and the CFGx.CH_SUSP bit is high, the CFGx.FIFO_EMPTY is asserted once the contents of the FIFO do not permit a single word of CTLx.DST_TR_WIDTH to be formed. However, there may still be data in the channel FIFO, but not enough to form a single transfer of CTLx.DST_TR_WIDTH. In this scenario, once the channel is disabled, the remaining data in the channel FIFO is not transferred to the destination peripheral." So in case if the port gets to be suspended and then disabled it's possible to have the data silently discarded even though the controller reported that FIFO is empty and the CTLx.BLOCK_TS indicated the dropped data already received from the source device. This looks as if the data somehow got lost on a way from the peripheral device to memory and causes problems for instance in the DW APB UART driver, which pauses and disables the DMA-transfer as soon as the recv data timeout happens. Here is the way it looks: Memory <------- DMA FIFO <------ UART FIFO <---------------- UART DST_TR_WIDTH -+--------| | | | | | | No more data Current lvl -+--------| |---------+- DMA-burst lvl | | |---------+- Leftover data | | |---------+- SRC_TR_WIDTH -+--------+-------+---------+ In the example above: no more data is getting received over the UART port and BLOCK_TS is not even close to be fully received; some data is left in the UART FIFO, but not enough to perform a bursted DMA-xfer to the DMA FIFO; some data is left in the DMA FIFO, but not enough to be passed further to the system memory in a single transfer. In this situation the 8250 UART driver catches the recv timeout interrupt, pauses the DMA-transfer and terminates it completely, after which the IRQ handler manually fetches the leftover data from the UART FIFO into the recv-buffer. But since the DMA-channel has been disabled with the data left in the DMA FIFO, that data will be just discarded and the recv-buffer will have a gap of the "current lvl" size in the recv-buffer at the tail of the lately received data portion. So the data will be lost just due to the misconfigured DMA transfer. Note this is only relevant for the case of the transfer suspension and _disabling_. No problem will happen if the transfer will be re-enabled afterwards or the block transfer is fully completed. In the later case the "FIFO flush mode" will be executed at the transfer final stage in order to push out the data left in the DMA FIFO. In order to fix the denoted problem the DW AHB DMA-engine driver needs to make sure that the _bursted_ source transfer width is greater or equal to the single destination transfer (note the HW databook describes more strict constraint than actually required). Since the peripheral-device side is prescribed by the client driver logic, the memory-side can be only used for that. The solution can be easily implemented for the DEV_TO_MEM transfers just by adjusting the memory-channel address width. Sadly it's not that easy for the MEM_TO_DEV transfers since the mem-to-dma burst size is normally dynamically determined by the controller. So the only thing that can be done is to make sure that memory-side address width is greater than the peripheral device address width. Fixes: a09820043c9e ("dw_dmac: autoconfigure data_width or get it via platform data") Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20240802075100.6475-3-fancer.lancer@gmail.com Signed-off-by: Vinod Koul

dmaengine: dw: Add peripheral bus width verification

2024-08-05T16:37:47+00:00

Currently the src_addr_width and dst_addr_width fields of the dma_slave_config structure are mapped to the CTLx.SRC_TR_WIDTH and CTLx.DST_TR_WIDTH fields of the peripheral bus side in order to have the properly aligned data passed to the target device. It's done just by converting the passed peripheral bus width to the encoded value using the __ffs() function. This implementation has several problematic sides: 1. __ffs() is undefined if no bit exist in the passed value. Thus if the specified addr-width is DMA_SLAVE_BUSWIDTH_UNDEFINED, __ffs() may return unexpected value depending on the platform-specific implementation. 2. DW AHB DMA-engine permits having the power-of-2 transfer width limited by the DMAH_Mk_HDATA_WIDTH IP-core synthesize parameter. Specifying bus-width out of that constraints scope will definitely cause unexpected result since the destination reg will be only partly touched than the client driver implied. Let's fix all of that by adding the peripheral bus width verification method and calling it in dwc_config() which is supposed to be executed before preparing any transfer. The new method will make sure that the passed source or destination address width is valid and if undefined then the driver will just fallback to the 1-byte width transfer. Fixes: 029a40e97d0d ("dmaengine: dw: provide DMA capabilities") Signed-off-by: Serge Semin Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20240802075100.6475-2-fancer.lancer@gmail.com Signed-off-by: Vinod Koul

dmaengine: dw: Move check for paused channel to dwc_get_residue()

2023-02-16T13:15:48+00:00

Move check for paused channel to dwc_get_residue() and rename the latter to dwc_get_residue_and_status(). This improves data integrity as residue and DMA channel status are set in the same function under the same conditions. Signed-off-by: Andy Shevchenko Link: https://lore.kernel.org/r/20230130151747.20704-1-andriy.shevchenko@linux.intel.com Signed-off-by: Vinod Koul

dma:dw: remove reference to AVR32 architecture in core.c

2022-08-03T09:03:02+00:00

The AVR32 architecture does no longer exist in the Linux kernel, hence remove a reference to it in comments to avoid confusion. Signed-off-by: Hans-Christian Noren Egtvedt

dmaengine dw: Revert "dmaengine: dw: Enable runtime PM"

2021-02-08T12:06:12+00:00

This reverts commit 842067940a3e3fc008a60fee388e000219b32632. For some solutions e.g. sound/soc/intel/catpt, DW DMA is part of a compound device (in that very example, domains: ADSP, SSP0, SSP1, DMA0 and DMA1 are part of a single entity) rather than being a standalone one. Driver for said device may enlist DMA to transfer data during suspend or resume sequences. Manipulating RPM explicitly in dw's DMA request and release channel functions causes suspend() to also invoke resume() for the exact same device. Similar situation occurs for resume() sequence. Effectively renders device dysfunctional after first suspend() attempt. Revert the change to address the problem. Fixes: 842067940a3e ("dmaengine: dw: Enable runtime PM") Cc: Andy Shevchenko Signed-off-by: Cezary Rojewski Acked-by: Andy Shevchenko Link: https://lore.kernel.org/r/20210203191924.15706-1-cezary.rojewski@intel.com Signed-off-by: Vinod Koul

dmaengine: dw: Enable runtime PM

2020-11-09T11:49:20+00:00

When consumer requests channel power on the DMA controller device and otherwise on the freeing channel resources. Note, in some cases consumer acquires channel at the ->probe() stage and releases it at the ->remove() stage. It will mean that DMA controller device will be powered during all this time if there is no assist from hardware to idle it. The above mentioned cases should be investigated separately and individually. Signed-off-by: Andy Shevchenko Acked-by: Viresh Kumar Link: https://lore.kernel.org/r/20201103183938.64752-1-andriy.shevchenko@linux.intel.com Signed-off-by: Vinod Koul

dmaengine: dw: convert tasklets to use new tasklet_setup() API

2020-09-18T06:48:11+00:00

In preparation for unconditionally passing the struct tasklet_struct pointer to all tasklet callbacks, switch to using the new tasklet_setup() and from_tasklet() to pass the tasklet pointer explicitly. Signed-off-by: Romain Perier Signed-off-by: Allen Pais Link: https://lore.kernel.org/r/20200831103542.305571-6-allen.lkml@gmail.com Signed-off-by: Vinod Koul

dmaengine: dw: Add DMA-channels mask cell support

2020-08-17T06:28:31+00:00

DW DMA IP-core provides a way to synthesize the DMA controller with channels having different parameters like maximum burst-length, multi-block support, maximum data width, etc. Those parameters both explicitly and implicitly affect the channels performance. Since DMA slave devices might be very demanding to the DMA performance, let's provide a functionality for the slaves to be assigned with DW DMA channels, which performance according to the platform engineer fulfill their requirements. After this patch is applied it can be done by passing the mask of suitable DMA-channels either directly in the dw_dma_slave structure instance or as a fifth cell of the DMA DT-property. If mask is zero or not provided, then there is no limitation on the channels allocation. For instance Baikal-T1 SoC is equipped with a DW DMAC engine, which first two channels are synthesized with max burst length of 16, while the rest of the channels have been created with max-burst-len=4. It would seem that the first two channels must be faster than the others and should be more preferable for the time-critical DMA slave devices. In practice it turned out that the situation is quite the opposite. The channels with max-burst-len=4 demonstrated a better performance than the channels with max-burst-len=16 even when they both had been initialized with the same settings. The performance drop of the first two DMA-channels made them unsuitable for the DW APB SSI slave device. No matter what settings they are configured with, full-duplex SPI transfers occasionally experience the Rx FIFO overflow. It means that the DMA-engine doesn't keep up with incoming data pace even though the SPI-bus is enabled with speed of 25MHz while the DW DMA controller is clocked with 50MHz signal. There is no such problem has been noticed for the channels synthesized with max-burst-len=4. Signed-off-by: Serge Semin Link: https://lore.kernel.org/r/20200731200826.9292-6-Sergey.Semin@baikalelectronics.ru Signed-off-by: Vinod Koul