kernel/linux.git/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c, branch v6.1.175

drm/amdgpu: zero-initialize GART table on allocation

2026-06-01T15:39:02+00:00

commit e6c2e6c2e1fa066968a16aca1cb66cd1bdde7741 upstream. GART TLB is flushed after unmapping but not after mapping. Since amdgpu_bo_create_kernel() does not zero-initialize the buffer, when a single PTE is written the TLB may speculatively load other uninitialized entries from the same cacheline. Those garbage entries can appear valid, and a subsequent write to another PTE in the same cacheline may cause the GPU to use a stale garbage PTE from the TLB. Fix this by calling memset_io() to zero-initialize the GART table with gart_pte_flags immediately after allocation. Using AMDGPU_GEM_CREATE_VRAM_CLEARED, SDMA-based clear will not work since SDMA needs GART to be initialized to work. Suggested-by: Felix Kuehling Signed-off-by: Philip Yang Reviewed-by: Christian König Signed-off-by: Alex Deucher (cherry picked from commit d9af8263b82b6eaa60c5718e0c6631c5037e4b24) Cc: stable@vger.kernel.org Signed-off-by: Greg Kroah-Hartman

drm/amdgpu: check the GART table before invalidating TLB

2022-02-07T23:01:16+00:00

Bypass group programming (utcl2_harvest) aims to forbid UTCL2 to send invalidation command to harvested SE/SA. Once invalidation command comes into harvested SE/SA, SE/SA has no response and system hang. This patch is to add checking if the GART table is already allocated before invalidating TLB. The new procedure is as following: 1. Calling amdgpu_gtt_mgr_init() in amdgpu_ttm_init(). After this step GTT BOs can be allocated, but GART mappings are still ignored. 2. Calling amdgpu_gart_table_vram_alloc() from the GMC code. This allocates the GART backing store. 3. Initializing the hardware, and programming the backing store into VMID0 for all VMHUBs. 4. Calling amdgpu_gtt_mgr_recover() to make sure the table is updated with the GTT allocations done before it was allocated. Signed-off-by: Christian König Signed-off-by: Aaron Liu Acked-by: Huang Rui Reviewed-by: Christian König Signed-off-by: Alex Deucher

drm/amdgpu: drop WARN_ON in amdgpu_gart_bind/unbind

2022-01-25T23:00:32+00:00

NULL pointer check has guarded it already. calltrace: amdgpu_ttm_gart_bind+0x49/0xa0 [amdgpu] amdgpu_ttm_alloc_gart+0x13f/0x180 [amdgpu] amdgpu_bo_create_reserved+0x139/0x2c0 [amdgpu] ? amdgpu_ttm_debugfs_init+0x120/0x120 [amdgpu] amdgpu_bo_create_kernel+0x17/0x80 [amdgpu] amdgpu_ttm_init+0x542/0x5e0 [amdgpu] Fixes: 1b08dfb889b2c5 ("drm/amdgpu: remove gart.ready flag") Signed-off-by: Guchun Chen Reviewed-by: Christian König Signed-off-by: Alex Deucher

drm/amdgpu: remove gart.ready flag

2022-01-20T03:32:47+00:00

That's just a leftover from old radeon days and was preventing CS and GART bindings before the hardware was initialized. But nowdays that is perfectly valid. The only thing we need to warn about are GART binding before the table is even allocated. Signed-off-by: Christian König Reviewed-by: Guchun Chen Signed-off-by: Alex Deucher

drm/amdgpu: recover gart table at resume

2022-01-11T20:44:26+00:00

Get rid off pin/unpin of gart BO at resume/suspend and instead pin only once and try to recover gart content at resume time. This is much more stable in case there is OOM situation at 2nd call to amdgpu_device_evict_resources() while evicting GART table. v3: remove gart recovery from other places v2: pin gart at amdgpu_gart_table_vram_alloc() Reviewed-by: Christian König Signed-off-by: Nirmoy Das Signed-off-by: Alex Deucher

drm/amdgpu: use adev_to_drm for consistency when accessing drm_device

2021-10-08T17:22:13+00:00

adev_to_drm is used everywhere, so improve recent changes when accessing drm_device pointer from amdgpu_device. Signed-off-by: Guchun Chen Acked-by: Christian König Signed-off-by: Alex Deucher

drm/amdgpu: Put drm_dev_enter/exit outside hot codepath

2021-09-23T19:17:29+00:00

We hit soft hang while doing memory pressure test on one numa system. After a qucik look, this is because kfd invalid/valid userptr memory frequently with process_info lock hold. Looks like update page table mapping use too much cpu time. perf top says below, 75.81% [kernel] [k] __srcu_read_unlock 6.19% [amdgpu] [k] amdgpu_gmc_set_pte_pde 3.56% [kernel] [k] __srcu_read_lock 2.20% [amdgpu] [k] amdgpu_vm_cpu_update 2.20% [kernel] [k] __sg_page_iter_dma_next 2.15% [drm] [k] drm_dev_enter 1.70% [drm] [k] drm_prime_sg_to_dma_addr_array 1.18% [kernel] [k] __sg_alloc_table_from_pages 1.09% [drm] [k] drm_dev_exit So move drm_dev_enter/exit outside gmc code, instead let caller do it. They are gart_unbind, gart_map, vm_clear_bo, vm_update_pdes and gmc_init_pdb0. vm_bo_update_mapping already calls it. Signed-off-by: xinhui pan Reviewed-and-tested-by: Andrey Grodzovsky Signed-off-by: Alex Deucher

drm/amdgpu: switch from 'pci_' to 'dma_' API

2021-08-24T19:35:41+00:00

The wrappers in include/linux/pci-dma-compat.h should go away. The patch has been generated with the coccinelle script below. It has been compile tested. @@ @@ - PCI_DMA_BIDIRECTIONAL + DMA_BIDIRECTIONAL @@ @@ - PCI_DMA_TODEVICE + DMA_TO_DEVICE @@ @@ - PCI_DMA_FROMDEVICE + DMA_FROM_DEVICE @@ @@ - PCI_DMA_NONE + DMA_NONE @@ expression e1, e2, e3; @@ - pci_alloc_consistent(e1, e2, e3) + dma_alloc_coherent(&e1->dev, e2, e3, GFP_) @@ expression e1, e2, e3; @@ - pci_zalloc_consistent(e1, e2, e3) + dma_alloc_coherent(&e1->dev, e2, e3, GFP_) @@ expression e1, e2, e3, e4; @@ - pci_free_consistent(e1, e2, e3, e4) + dma_free_coherent(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_map_single(e1, e2, e3, e4) + dma_map_single(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_unmap_single(e1, e2, e3, e4) + dma_unmap_single(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4, e5; @@ - pci_map_page(e1, e2, e3, e4, e5) + dma_map_page(&e1->dev, e2, e3, e4, e5) @@ expression e1, e2, e3, e4; @@ - pci_unmap_page(e1, e2, e3, e4) + dma_unmap_page(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_map_sg(e1, e2, e3, e4) + dma_map_sg(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_unmap_sg(e1, e2, e3, e4) + dma_unmap_sg(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_dma_sync_single_for_cpu(e1, e2, e3, e4) + dma_sync_single_for_cpu(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_dma_sync_single_for_device(e1, e2, e3, e4) + dma_sync_single_for_device(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_dma_sync_sg_for_cpu(e1, e2, e3, e4) + dma_sync_sg_for_cpu(&e1->dev, e2, e3, e4) @@ expression e1, e2, e3, e4; @@ - pci_dma_sync_sg_for_device(e1, e2, e3, e4) + dma_sync_sg_for_device(&e1->dev, e2, e3, e4) @@ expression e1, e2; @@ - pci_dma_mapping_error(e1, e2) + dma_mapping_error(&e1->dev, e2) @@ expression e1, e2; @@ - pci_set_dma_mask(e1, e2) + dma_set_mask(&e1->dev, e2) @@ expression e1, e2; @@ - pci_set_consistent_dma_mask(e1, e2) + dma_set_coherent_mask(&e1->dev, e2) Reviewed-by: Christian König Signed-off-by: Christophe JAILLET Signed-off-by: Alex Deucher

drm/amdgpu: remove unused parameter in amdgpu_gart_bind

2021-06-18T21:12:47+00:00

Pagelist is no long used in amdgpu_gart_bind. Remove it. Signed-off-by: Yifan Zhang Reviewed-by: Christian König Signed-off-by: Alex Deucher

drm/amdgpu: Don't flush/invalidate HDP for APUs and A+A

2021-06-04T20:02:38+00:00

Integrate two generic functions to determine if HDP flush is needed for all Asics. Signed-off-by: Eric Huang Reviewed-by: Christian König Signed-off-by: Alex Deucher