diff options
author | Dave Airlie <airlied@redhat.com> | 2019-02-01 02:51:23 +0300 |
---|---|---|
committer | Dave Airlie <airlied@redhat.com> | 2019-02-01 03:01:50 +0300 |
commit | 74b7d6a91311766ab6c94f6be21bd423021ca95e (patch) | |
tree | 7b4540d08d37894033bc148d87e0e6cca3a6002b /Documentation/gpu | |
parent | fb27a3cb9cbfedaa914c04bbae45861646d397b6 (diff) | |
parent | dcc9d76b6d834d06a317e27fa8242d7e009135ac (diff) | |
download | linux-74b7d6a91311766ab6c94f6be21bd423021ca95e.tar.xz |
Merge branch 'for-upstream/mali-dp' of git://linux-arm.org/linux-ld into drm-next
This pull includes the new Arm "komeda" DRM driver. It is currently hosted
in the same repo as the other "mali-dp" driver because it is the next
iteration of the IP.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Liviu Dudau <Liviu.Dudau@arm.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190131173600.GN25147@e110455-lin.cambridge.arm.com
Diffstat (limited to 'Documentation/gpu')
-rw-r--r-- | Documentation/gpu/afbc.rst | 235 | ||||
-rw-r--r-- | Documentation/gpu/drivers.rst | 2 | ||||
-rw-r--r-- | Documentation/gpu/komeda-kms.rst | 488 |
3 files changed, 725 insertions, 0 deletions
diff --git a/Documentation/gpu/afbc.rst b/Documentation/gpu/afbc.rst new file mode 100644 index 000000000000..4d38dc49d105 --- /dev/null +++ b/Documentation/gpu/afbc.rst @@ -0,0 +1,235 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +=================================== + Arm Framebuffer Compression (AFBC) +=================================== + +AFBC is a proprietary lossless image compression protocol and format. +It provides fine-grained random access and minimizes the amount of +data transferred between IP blocks. + +AFBC can be enabled on drivers which support it via use of the AFBC +format modifiers defined in drm_fourcc.h. See DRM_FORMAT_MOD_ARM_AFBC(*). + +All users of the AFBC modifiers must follow the usage guidelines laid +out in this document, to ensure compatibility across different AFBC +producers and consumers. + +Components and Ordering +======================= + +AFBC streams can contain several components - where a component +corresponds to a color channel (i.e. R, G, B, X, A, Y, Cb, Cr). +The assignment of input/output color channels must be consistent +between the encoder and the decoder for correct operation, otherwise +the consumer will interpret the decoded data incorrectly. + +Furthermore, when the lossless colorspace transform is used +(AFBC_FORMAT_MOD_YTR, which should be enabled for RGB buffers for +maximum compression efficiency), the component order must be: + + * Component 0: R + * Component 1: G + * Component 2: B + +The component ordering is communicated via the fourcc code in the +fourcc:modifier pair. In general, component '0' is considered to +reside in the least-significant bits of the corresponding linear +format. For example, COMP(bits): + + * DRM_FORMAT_ABGR8888 + + * Component 0: R(8) + * Component 1: G(8) + * Component 2: B(8) + * Component 3: A(8) + + * DRM_FORMAT_BGR888 + + * Component 0: R(8) + * Component 1: G(8) + * Component 2: B(8) + + * DRM_FORMAT_YUYV + + * Component 0: Y(8) + * Component 1: Cb(8, 2x1 subsampled) + * Component 2: Cr(8, 2x1 subsampled) + +In AFBC, 'X' components are not treated any differently from any other +component. Therefore, an AFBC buffer with fourcc DRM_FORMAT_XBGR8888 +encodes with 4 components, like so: + + * DRM_FORMAT_XBGR8888 + + * Component 0: R(8) + * Component 1: G(8) + * Component 2: B(8) + * Component 3: X(8) + +Please note, however, that the inclusion of a "wasted" 'X' channel is +bad for compression efficiency, and so it's recommended to avoid +formats containing 'X' bits. If a fourth component is +required/expected by the encoder/decoder, then it is recommended to +instead use an equivalent format with alpha, setting all alpha bits to +'1'. If there is no requirement for a fourth component, then a format +which doesn't include alpha can be used, e.g. DRM_FORMAT_BGR888. + +Number of Planes +================ + +Formats which are typically multi-planar in linear layouts (e.g. YUV +420), can be encoded into one, or multiple, AFBC planes. As with +component order, the encoder and decoder must agree about the number +of planes in order to correctly decode the buffer. The fourcc code is +used to determine the number of encoded planes in an AFBC buffer, +matching the number of planes for the linear (unmodified) format. +Within each plane, the component ordering also follows the fourcc +code: + +For example: + + * DRM_FORMAT_YUYV: nplanes = 1 + + * Plane 0: + + * Component 0: Y(8) + * Component 1: Cb(8, 2x1 subsampled) + * Component 2: Cr(8, 2x1 subsampled) + + * DRM_FORMAT_NV12: nplanes = 2 + + * Plane 0: + + * Component 0: Y(8) + + * Plane 1: + + * Component 0: Cb(8, 2x1 subsampled) + * Component 1: Cr(8, 2x1 subsampled) + +Cross-device interoperability +============================= + +For maximum compatibility across devices, the table below defines +canonical formats for use between AFBC-enabled devices. Formats which +are listed here must be used exactly as specified when using the AFBC +modifiers. Formats which are not listed should be avoided. + +.. flat-table:: AFBC formats + + * - Fourcc code + - Description + - Planes/Components + + * - DRM_FORMAT_ABGR2101010 + - 10-bit per component RGB, with 2-bit alpha + - Plane 0: 4 components + * Component 0: R(10) + * Component 1: G(10) + * Component 2: B(10) + * Component 3: A(2) + + * - DRM_FORMAT_ABGR8888 + - 8-bit per component RGB, with 8-bit alpha + - Plane 0: 4 components + * Component 0: R(8) + * Component 1: G(8) + * Component 2: B(8) + * Component 3: A(8) + + * - DRM_FORMAT_BGR888 + - 8-bit per component RGB + - Plane 0: 3 components + * Component 0: R(8) + * Component 1: G(8) + * Component 2: B(8) + + * - DRM_FORMAT_BGR565 + - 5/6-bit per component RGB + - Plane 0: 3 components + * Component 0: R(5) + * Component 1: G(6) + * Component 2: B(5) + + * - DRM_FORMAT_ABGR1555 + - 5-bit per component RGB, with 1-bit alpha + - Plane 0: 4 components + * Component 0: R(5) + * Component 1: G(5) + * Component 2: B(5) + * Component 3: A(1) + + * - DRM_FORMAT_VUY888 + - 8-bit per component YCbCr 444, single plane + - Plane 0: 3 components + * Component 0: Y(8) + * Component 1: Cb(8) + * Component 2: Cr(8) + + * - DRM_FORMAT_VUY101010 + - 10-bit per component YCbCr 444, single plane + - Plane 0: 3 components + * Component 0: Y(10) + * Component 1: Cb(10) + * Component 2: Cr(10) + + * - DRM_FORMAT_YUYV + - 8-bit per component YCbCr 422, single plane + - Plane 0: 3 components + * Component 0: Y(8) + * Component 1: Cb(8, 2x1 subsampled) + * Component 2: Cr(8, 2x1 subsampled) + + * - DRM_FORMAT_NV16 + - 8-bit per component YCbCr 422, two plane + - Plane 0: 1 component + * Component 0: Y(8) + Plane 1: 2 components + * Component 0: Cb(8, 2x1 subsampled) + * Component 1: Cr(8, 2x1 subsampled) + + * - DRM_FORMAT_Y210 + - 10-bit per component YCbCr 422, single plane + - Plane 0: 3 components + * Component 0: Y(10) + * Component 1: Cb(10, 2x1 subsampled) + * Component 2: Cr(10, 2x1 subsampled) + + * - DRM_FORMAT_P210 + - 10-bit per component YCbCr 422, two plane + - Plane 0: 1 component + * Component 0: Y(10) + Plane 1: 2 components + * Component 0: Cb(10, 2x1 subsampled) + * Component 1: Cr(10, 2x1 subsampled) + + * - DRM_FORMAT_YUV420_8BIT + - 8-bit per component YCbCr 420, single plane + - Plane 0: 3 components + * Component 0: Y(8) + * Component 1: Cb(8, 2x2 subsampled) + * Component 2: Cr(8, 2x2 subsampled) + + * - DRM_FORMAT_YUV420_10BIT + - 10-bit per component YCbCr 420, single plane + - Plane 0: 3 components + * Component 0: Y(10) + * Component 1: Cb(10, 2x2 subsampled) + * Component 2: Cr(10, 2x2 subsampled) + + * - DRM_FORMAT_NV12 + - 8-bit per component YCbCr 420, two plane + - Plane 0: 1 component + * Component 0: Y(8) + Plane 1: 2 components + * Component 0: Cb(8, 2x2 subsampled) + * Component 1: Cr(8, 2x2 subsampled) + + * - DRM_FORMAT_P010 + - 10-bit per component YCbCr 420, two plane + - Plane 0: 1 component + * Component 0: Y(10) + Plane 1: 2 components + * Component 0: Cb(10, 2x2 subsampled) + * Component 1: Cr(10, 2x2 subsampled) diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst index 7c1672118a73..044a7025477c 100644 --- a/Documentation/gpu/drivers.rst +++ b/Documentation/gpu/drivers.rst @@ -17,6 +17,8 @@ GPU Driver Documentation vkms bridge/dw-hdmi xen-front + afbc + komeda-kms .. only:: subproject and html diff --git a/Documentation/gpu/komeda-kms.rst b/Documentation/gpu/komeda-kms.rst new file mode 100644 index 000000000000..b08da1cffecc --- /dev/null +++ b/Documentation/gpu/komeda-kms.rst @@ -0,0 +1,488 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================== + drm/komeda Arm display driver +============================== + +The drm/komeda driver supports the Arm display processor D71 and later products, +this document gives a brief overview of driver design: how it works and why +design it like that. + +Overview of D71 like display IPs +================================ + +From D71, Arm display IP begins to adopt a flexible and modularized +architecture. A display pipeline is made up of multiple individual and +functional pipeline stages called components, and every component has some +specific capabilities that can give the flowed pipeline pixel data a +particular processing. + +Typical D71 components: + +Layer +----- +Layer is the first pipeline stage, which prepares the pixel data for the next +stage. It fetches the pixel from memory, decodes it if it's AFBC, rotates the +source image, unpacks or converts YUV pixels to the device internal RGB pixels, +then adjusts the color_space of pixels if needed. + +Scaler +------ +As its name suggests, scaler takes responsibility for scaling, and D71 also +supports image enhancements by scaler. +The usage of scaler is very flexible and can be connected to layer output +for layer scaling, or connected to compositor and scale the whole display +frame and then feed the output data into wb_layer which will then write it +into memory. + +Compositor (compiz) +------------------- +Compositor blends multiple layers or pixel data flows into one single display +frame. its output frame can be fed into post image processor for showing it on +the monitor or fed into wb_layer and written to memory at the same time. +user can also insert a scaler between compositor and wb_layer to down scale +the display frame first and and then write to memory. + +Writeback Layer (wb_layer) +-------------------------- +Writeback layer does the opposite things of Layer, which connects to compiz +and writes the composition result to memory. + +Post image processor (improc) +----------------------------- +Post image processor adjusts frame data like gamma and color space to fit the +requirements of the monitor. + +Timing controller (timing_ctrlr) +-------------------------------- +Final stage of display pipeline, Timing controller is not for the pixel +handling, but only for controlling the display timing. + +Merger +------ +D71 scaler mostly only has the half horizontal input/output capabilities +compared with Layer, like if Layer supports 4K input size, the scaler only can +support 2K input/output in the same time. To achieve the ful frame scaling, D71 +introduces Layer Split, which splits the whole image to two half parts and feeds +them to two Layers A and B, and does the scaling independently. After scaling +the result need to be fed to merger to merge two part images together, and then +output merged result to compiz. + +Splitter +-------- +Similar to Layer Split, but Splitter is used for writeback, which splits the +compiz result to two parts and then feed them to two scalers. + +Possible D71 Pipeline usage +=========================== + +Benefitting from the modularized architecture, D71 pipelines can be easily +adjusted to fit different usages. And D71 has two pipelines, which support two +types of working mode: + +- Dual display mode + Two pipelines work independently and separately to drive two display outputs. + +- Single display mode + Two pipelines work together to drive only one display output. + + On this mode, pipeline_B doesn't work indenpendently, but outputs its + composition result into pipeline_A, and its pixel timing also derived from + pipeline_A.timing_ctrlr. The pipeline_B works just like a "slave" of + pipeline_A(master) + +Single pipeline data flow +------------------------- + +.. kernel-render:: DOT + :alt: Single pipeline digraph + :caption: Single pipeline data flow + + digraph single_ppl { + rankdir=LR; + + subgraph { + "Memory"; + "Monitor"; + } + + subgraph cluster_pipeline { + style=dashed + node [shape=box] + { + node [bgcolor=grey style=dashed] + "Scaler-0"; + "Scaler-1"; + "Scaler-0/1" + } + + node [bgcolor=grey style=filled] + "Layer-0" -> "Scaler-0" + "Layer-1" -> "Scaler-0" + "Layer-2" -> "Scaler-1" + "Layer-3" -> "Scaler-1" + + "Layer-0" -> "Compiz" + "Layer-1" -> "Compiz" + "Layer-2" -> "Compiz" + "Layer-3" -> "Compiz" + "Scaler-0" -> "Compiz" + "Scaler-1" -> "Compiz" + + "Compiz" -> "Scaler-0/1" -> "Wb_layer" + "Compiz" -> "Improc" -> "Timing Controller" + } + + "Wb_layer" -> "Memory" + "Timing Controller" -> "Monitor" + } + +Dual pipeline with Slave enabled +-------------------------------- + +.. kernel-render:: DOT + :alt: Slave pipeline digraph + :caption: Slave pipeline enabled data flow + + digraph slave_ppl { + rankdir=LR; + + subgraph { + "Memory"; + "Monitor"; + } + node [shape=box] + subgraph cluster_pipeline_slave { + style=dashed + label="Slave Pipeline_B" + node [shape=box] + { + node [bgcolor=grey style=dashed] + "Slave.Scaler-0"; + "Slave.Scaler-1"; + } + + node [bgcolor=grey style=filled] + "Slave.Layer-0" -> "Slave.Scaler-0" + "Slave.Layer-1" -> "Slave.Scaler-0" + "Slave.Layer-2" -> "Slave.Scaler-1" + "Slave.Layer-3" -> "Slave.Scaler-1" + + "Slave.Layer-0" -> "Slave.Compiz" + "Slave.Layer-1" -> "Slave.Compiz" + "Slave.Layer-2" -> "Slave.Compiz" + "Slave.Layer-3" -> "Slave.Compiz" + "Slave.Scaler-0" -> "Slave.Compiz" + "Slave.Scaler-1" -> "Slave.Compiz" + } + + subgraph cluster_pipeline_master { + style=dashed + label="Master Pipeline_A" + node [shape=box] + { + node [bgcolor=grey style=dashed] + "Scaler-0"; + "Scaler-1"; + "Scaler-0/1" + } + + node [bgcolor=grey style=filled] + "Layer-0" -> "Scaler-0" + "Layer-1" -> "Scaler-0" + "Layer-2" -> "Scaler-1" + "Layer-3" -> "Scaler-1" + + "Slave.Compiz" -> "Compiz" + "Layer-0" -> "Compiz" + "Layer-1" -> "Compiz" + "Layer-2" -> "Compiz" + "Layer-3" -> "Compiz" + "Scaler-0" -> "Compiz" + "Scaler-1" -> "Compiz" + + "Compiz" -> "Scaler-0/1" -> "Wb_layer" + "Compiz" -> "Improc" -> "Timing Controller" + } + + "Wb_layer" -> "Memory" + "Timing Controller" -> "Monitor" + } + +Sub-pipelines for input and output +---------------------------------- + +A complete display pipeline can be easily divided into three sub-pipelines +according to the in/out usage. + +Layer(input) pipeline +~~~~~~~~~~~~~~~~~~~~~ + +.. kernel-render:: DOT + :alt: Layer data digraph + :caption: Layer (input) data flow + + digraph layer_data_flow { + rankdir=LR; + node [shape=box] + + { + node [bgcolor=grey style=dashed] + "Scaler-n"; + } + + "Layer-n" -> "Scaler-n" -> "Compiz" + } + +.. kernel-render:: DOT + :alt: Layer Split digraph + :caption: Layer Split pipeline + + digraph layer_data_flow { + rankdir=LR; + node [shape=box] + + "Layer-0/1" -> "Scaler-0" -> "Merger" + "Layer-2/3" -> "Scaler-1" -> "Merger" + "Merger" -> "Compiz" + } + +Writeback(output) pipeline +~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. kernel-render:: DOT + :alt: writeback digraph + :caption: Writeback(output) data flow + + digraph writeback_data_flow { + rankdir=LR; + node [shape=box] + + { + node [bgcolor=grey style=dashed] + "Scaler-n"; + } + + "Compiz" -> "Scaler-n" -> "Wb_layer" + } + +.. kernel-render:: DOT + :alt: split writeback digraph + :caption: Writeback(output) Split data flow + + digraph writeback_data_flow { + rankdir=LR; + node [shape=box] + + "Compiz" -> "Splitter" + "Splitter" -> "Scaler-0" -> "Merger" + "Splitter" -> "Scaler-1" -> "Merger" + "Merger" -> "Wb_layer" + } + +Display output pipeline +~~~~~~~~~~~~~~~~~~~~~~~ +.. kernel-render:: DOT + :alt: display digraph + :caption: display output data flow + + digraph single_ppl { + rankdir=LR; + node [shape=box] + + "Compiz" -> "Improc" -> "Timing Controller" + } + +In the following section we'll see these three sub-pipelines will be handled +by KMS-plane/wb_conn/crtc respectively. + +Komeda Resource abstraction +=========================== + +struct komeda_pipeline/component +-------------------------------- + +To fully utilize and easily access/configure the HW, the driver side also uses +a similar architecture: Pipeline/Component to describe the HW features and +capabilities, and a specific component includes two parts: + +- Data flow controlling. +- Specific component capabilities and features. + +So the driver defines a common header struct komeda_component to describe the +data flow control and all specific components are a subclass of this base +structure. + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h + :internal: + +Resource discovery and initialization +===================================== + +Pipeline and component are used to describe how to handle the pixel data. We +still need a @struct komeda_dev to describe the whole view of the device, and +the control-abilites of device. + +We have &komeda_dev, &komeda_pipeline, &komeda_component. Now fill devices with +pipelines. Since komeda is not for D71 only but also intended for later products, +of course we’d better share as much as possible between different products. To +achieve this, split the komeda device into two layers: CORE and CHIP. + +- CORE: for common features and capabilities handling. +- CHIP: for register programing and HW specific feature (limitation) handling. + +CORE can access CHIP by three chip function structures: + +- struct komeda_dev_funcs +- struct komeda_pipeline_funcs +- struct komeda_component_funcs + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_dev.h + :internal: + +Format handling +=============== + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h + :internal: +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.h + :internal: + +Attach komeda_dev to DRM-KMS +============================ + +Komeda abstracts resources by pipeline/component, but DRM-KMS uses +crtc/plane/connector. One KMS-obj cannot represent only one single component, +since the requirements of a single KMS object cannot simply be achieved by a +single component, usually that needs multiple components to fit the requirement. +Like set mode, gamma, ctm for KMS all target on CRTC-obj, but komeda needs +compiz, improc and timing_ctrlr to work together to fit these requirements. +And a KMS-Plane may require multiple komeda resources: layer/scaler/compiz. + +So, one KMS-Obj represents a sub-pipeline of komeda resources. + +- Plane: `Layer(input) pipeline`_ +- Wb_connector: `Writeback(output) pipeline`_ +- Crtc: `Display output pipeline`_ + +So, for komeda, we treat KMS crtc/plane/connector as users of pipeline and +component, and at any one time a pipeline/component only can be used by one +user. And pipeline/component will be treated as private object of DRM-KMS; the +state will be managed by drm_atomic_state as well. + +How to map plane to Layer(input) pipeline +----------------------------------------- + +Komeda has multiple Layer input pipelines, see: +- `Single pipeline data flow`_ +- `Dual pipeline with Slave enabled`_ + +The easiest way is binding a plane to a fixed Layer pipeline, but consider the +komeda capabilities: + +- Layer Split, See `Layer(input) pipeline`_ + + Layer_Split is quite complicated feature, which splits a big image into two + parts and handles it by two layers and two scalers individually. But it + imports an edge problem or effect in the middle of the image after the split. + To avoid such a problem, it needs a complicated Split calculation and some + special configurations to the layer and scaler. We'd better hide such HW + related complexity to user mode. + +- Slave pipeline, See `Dual pipeline with Slave enabled`_ + + Since the compiz component doesn't output alpha value, the slave pipeline + only can be used for bottom layers composition. The komeda driver wants to + hide this limitation to the user. The way to do this is to pick a suitable + Layer according to plane_state->zpos. + +So for komeda, the KMS-plane doesn't represent a fixed komeda layer pipeline, +but multiple Layers with same capabilities. Komeda will select one or more +Layers to fit the requirement of one KMS-plane. + +Make component/pipeline to be drm_private_obj +--------------------------------------------- + +Add :c:type:`drm_private_obj` to :c:type:`komeda_component`, :c:type:`komeda_pipeline` + +.. code-block:: c + + struct komeda_component { + struct drm_private_obj obj; + ... + } + + struct komeda_pipeline { + struct drm_private_obj obj; + ... + } + +Tracking component_state/pipeline_state by drm_atomic_state +----------------------------------------------------------- + +Add :c:type:`drm_private_state` and user to :c:type:`komeda_component_state`, +:c:type:`komeda_pipeline_state` + +.. code-block:: c + + struct komeda_component_state { + struct drm_private_state obj; + void *binding_user; + ... + } + + struct komeda_pipeline_state { + struct drm_private_state obj; + struct drm_crtc *crtc; + ... + } + +komeda component validation +--------------------------- + +Komeda has multiple types of components, but the process of validation are +similar, usually including the following steps: + +.. code-block:: c + + int komeda_xxxx_validate(struct komeda_component_xxx xxx_comp, + struct komeda_component_output *input_dflow, + struct drm_plane/crtc/connector *user, + struct drm_plane/crtc/connector_state, *user_state) + { + setup 1: check if component is needed, like the scaler is optional depending + on the user_state; if unneeded, just return, and the caller will + put the data flow into next stage. + Setup 2: check user_state with component features and capabilities to see + if requirements can be met; if not, return fail. + Setup 3: get component_state from drm_atomic_state, and try set to set + user to component; fail if component has been assigned to another + user already. + Setup 3: configure the component_state, like set its input component, + convert user_state to component specific state. + Setup 4: adjust the input_dflow and prepare it for the next stage. + } + +komeda_kms Abstraction +---------------------- + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_kms.h + :internal: + +komde_kms Functions +------------------- +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_crtc.c + :internal: +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_plane.c + :internal: + +Build komeda to be a Linux module driver +======================================== + +Now we have two level devices: + +- komeda_dev: describes the real display hardware. +- komeda_kms_dev: attachs or connects komeda_dev to DRM-KMS. + +All komeda operations are supplied or operated by komeda_dev or komeda_kms_dev, +the module driver is only a simple wrapper to pass the Linux command +(probe/remove/pm) into komeda_dev or komeda_kms_dev. |