summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/amd/amdkfd
AgeCommit message (Collapse)AuthorFilesLines
2023-03-22drm/amdkfd: Fix an illegal memory accessQu Huang1-6/+3
[ Upstream commit 4fc8fff378b2f2039f2a666d9f8c570f4e58352c ] In the kfd_wait_on_events() function, the kfd_event_waiter structure is allocated by alloc_event_waiters(), but the event field of the waiter structure is not initialized; When copy_from_user() fails in the kfd_wait_on_events() function, it will enter exception handling to release the previously allocated memory of the waiter structure; Due to the event field of the waiters structure being accessed in the free_waiters() function, this results in illegal memory access and system crash, here is the crash log: localhost kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x185/0x1e0 localhost kernel: RSP: 0018:ffffaa53c362bd60 EFLAGS: 00010082 localhost kernel: RAX: ff3d3d6bff4007cb RBX: 0000000000000282 RCX: 00000000002c0000 localhost kernel: RDX: ffff9e855eeacb80 RSI: 000000000000279c RDI: ffffe7088f6a21d0 localhost kernel: RBP: ffffe7088f6a21d0 R08: 00000000002c0000 R09: ffffaa53c362be64 localhost kernel: R10: ffffaa53c362bbd8 R11: 0000000000000001 R12: 0000000000000002 localhost kernel: R13: ffff9e7ead15d600 R14: 0000000000000000 R15: ffff9e7ead15d698 localhost kernel: FS: 0000152a3d111700(0000) GS:ffff9e855ee80000(0000) knlGS:0000000000000000 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 localhost kernel: CR2: 0000152938000010 CR3: 000000044d7a4000 CR4: 00000000003506e0 localhost kernel: Call Trace: localhost kernel: _raw_spin_lock_irqsave+0x30/0x40 localhost kernel: remove_wait_queue+0x12/0x50 localhost kernel: kfd_wait_on_events+0x1b6/0x490 [hydcu] localhost kernel: ? ftrace_graph_caller+0xa0/0xa0 localhost kernel: kfd_ioctl+0x38c/0x4a0 [hydcu] localhost kernel: ? kfd_ioctl_set_trap_handler+0x70/0x70 [hydcu] localhost kernel: ? kfd_ioctl_create_queue+0x5a0/0x5a0 [hydcu] localhost kernel: ? ftrace_graph_caller+0xa0/0xa0 localhost kernel: __x64_sys_ioctl+0x8e/0xd0 localhost kernel: ? syscall_trace_enter.isra.18+0x143/0x1b0 localhost kernel: do_syscall_64+0x33/0x80 localhost kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 localhost kernel: RIP: 0033:0x152a4dff68d7 Allocate the structure with kcalloc, and remove redundant 0-initialization and a redundant loop condition check. Signed-off-by: Qu Huang <qu.huang@linux.dev> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-04-20drm/amdkfd: Check for potential null return of kmalloc_array()QintaoShen1-0/+2
[ Upstream commit ebbb7bb9e80305820dc2328a371c1b35679f2667 ] As the kmalloc_array() may return null, the 'event_waiters[i].wait' would lead to null-pointer dereference. Therefore, it is better to check the return value of kmalloc_array() to avoid this confusion. Signed-off-by: QintaoShen <unSimple1993@163.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-04-15drm/amdkfd: make CRAT table missing message informational onlyAlex Deucher1-1/+1
[ Upstream commit 9dff13f9edf755a15f6507874185a3290c1ae8bb ] The driver has a fallback so make the message informational rather than a warning. The driver has a fallback if the Component Resource Association Table (CRAT) is missing, so make this informational now. Bug: https://gitlab.freedesktop.org/drm/amd/-/issues/1906 Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-05-22drm/amdkfd: fix build error with AMD_IOMMU_V2=mFelix Kuehling2-2/+13
[ Upstream commit 1e87068570a2cc4db5f95a881686add71729e769 ] Using 'imply AMD_IOMMU_V2' does not guarantee that the driver can link against the exported functions. If the GPU driver is built-in but the IOMMU driver is a loadable module, the kfd_iommu.c file is indeed built but does not work: x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_bind_process_to_device': kfd_iommu.c:(.text+0x516): undefined reference to `amd_iommu_bind_pasid' x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_unbind_process': kfd_iommu.c:(.text+0x691): undefined reference to `amd_iommu_unbind_pasid' x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_suspend': kfd_iommu.c:(.text+0x966): undefined reference to `amd_iommu_set_invalidate_ctx_cb' x86_64-linux-ld: kfd_iommu.c:(.text+0x97f): undefined reference to `amd_iommu_set_invalid_ppr_cb' x86_64-linux-ld: kfd_iommu.c:(.text+0x9a4): undefined reference to `amd_iommu_free_device' x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_resume': kfd_iommu.c:(.text+0xa9a): undefined reference to `amd_iommu_init_device' x86_64-linux-ld: kfd_iommu.c:(.text+0xadc): undefined reference to `amd_iommu_set_invalidate_ctx_cb' x86_64-linux-ld: kfd_iommu.c:(.text+0xaff): undefined reference to `amd_iommu_set_invalid_ppr_cb' x86_64-linux-ld: kfd_iommu.c:(.text+0xc72): undefined reference to `amd_iommu_bind_pasid' x86_64-linux-ld: kfd_iommu.c:(.text+0xe08): undefined reference to `amd_iommu_set_invalidate_ctx_cb' x86_64-linux-ld: kfd_iommu.c:(.text+0xe26): undefined reference to `amd_iommu_set_invalid_ppr_cb' x86_64-linux-ld: kfd_iommu.c:(.text+0xe42): undefined reference to `amd_iommu_free_device' Use IS_REACHABLE to only build IOMMU-V2 support if the amd_iommu symbols are reachable by the amdkfd driver. Output a warning if they are not, because that may not be what the user was expecting. Fixes: 64d1c3a43a6f ("drm/amdkfd: Centralize IOMMUv2 code and make it conditional") Reported-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2020-10-01drm/amdkfd: fix a memory leak issueDennis Li1-0/+2
[ Upstream commit 087d764159996ae378b08c0fdd557537adfd6899 ] In the resume stage of GPU recovery, start_cpsch will call pm_init which set pm->allocated as false, cause the next pm_release_ib has no chance to release ib memory. Add pm_release_ib in stop_cpsch which will be called in the suspend stage of GPU recovery. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Dennis Li <Dennis.Li@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2020-09-03drm/amdkfd: Fix reference count leaks.Qiushi Wu1-5/+15
[ Upstream commit 20eca0123a35305e38b344d571cf32768854168c ] kobject_init_and_add() takes reference even when it fails. If this function returns an error, kobject_put() must be called to properly clean up the memory associated with the object. Signed-off-by: Qiushi Wu <wu000273@umn.edu> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2020-04-23drm/amdkfd: kfree the wrong pointerJack Zhang1-2/+2
[ Upstream commit 3148a6a0ef3cf93570f30a477292768f7eb5d3c3 ] Originally, it kfrees the wrong pointer for mem_obj. It would cause memory leak under stress test. Signed-off-by: Jack Zhang <Jack.Zhang1@amd.com> Acked-by: Nirmoy Das <nirmoy.das@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-12-31drm/amdkfd: fix a potential NULL pointer dereference (v2)Allen Pais1-0/+5
[ Upstream commit 81de29d842ccb776c0f77aa3e2b11b07fff0c0e2 ] alloc_workqueue is not checked for errors and as a result, a potential NULL dereference could occur. v2 (Felix Kuehling): * Fix compile error (kfifo_free instead of fifo_free) * Return proper error code Signed-off-by: Allen Pais <allen.pais@oracle.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-09-16drm/amdkfd: Add missing Polaris10 IDKent Russell1-0/+1
[ Upstream commit 0a5a9c276c335870a1cecc4f02b76d6d6f663c8b ] This was added to amdgpu but was missed in amdkfd Signed-off-by: Kent Russell <kent.russell@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Cc: stable@vger.kernel.rg Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-07-31drm/amdkfd: Fix sdma queue map issueOak Zeng1-9/+12
[ Upstream commit 065e4bdfa1f3ab2884c110394d8b7e7ebe3b988c ] Previous codes assumes there are two sdma engines. This is not true e.g., Raven only has 1 SDMA engine. Fix the issue by using sdma engine number info in device_info. Signed-off-by: Oak Zeng <Oak.Zeng@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-07-31drm/amdkfd: Fix a potential memory leakOak Zeng1-1/+4
[ Upstream commit e73390d181103a19e1111ec2f25559a0570e9fe0 ] Free mqd_mem_obj it GTT buffer allocation for MQD+control stack fails. Signed-off-by: Oak Zeng <ozeng@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-04-20drm/amdkfd: use init_mqd function to allocate object for hid_mqd (CI)Kevin Wang1-51/+1
[ Upstream commit cac734c2dbd2514f14c8c6a17caba1990d83bf1d ] if use the legacy method to allocate object, when mqd_hiq need to run uninit code, it will be cause WARNING call trace. eg: (s3 suspend test) [ 34.918944] Call Trace: [ 34.918948] [<ffffffff92961dc1>] dump_stack+0x19/0x1b [ 34.918950] [<ffffffff92297648>] __warn+0xd8/0x100 [ 34.918951] [<ffffffff9229778d>] warn_slowpath_null+0x1d/0x20 [ 34.918991] [<ffffffffc03ce1fe>] uninit_mqd_hiq_sdma+0x4e/0x50 [amdgpu] [ 34.919028] [<ffffffffc03d0ef7>] uninitialize+0x37/0xe0 [amdgpu] [ 34.919064] [<ffffffffc03d15a6>] kernel_queue_uninit+0x16/0x30 [amdgpu] [ 34.919086] [<ffffffffc03d26c2>] pm_uninit+0x12/0x20 [amdgpu] [ 34.919107] [<ffffffffc03d4915>] stop_nocpsch+0x15/0x20 [amdgpu] [ 34.919129] [<ffffffffc03c1dce>] kgd2kfd_suspend.part.4+0x2e/0x50 [amdgpu] [ 34.919150] [<ffffffffc03c2667>] kgd2kfd_suspend+0x17/0x20 [amdgpu] [ 34.919171] [<ffffffffc03c103a>] amdgpu_amdkfd_suspend+0x1a/0x20 [amdgpu] [ 34.919187] [<ffffffffc02ec428>] amdgpu_device_suspend+0x88/0x3a0 [amdgpu] [ 34.919189] [<ffffffff922e22cf>] ? enqueue_entity+0x2ef/0xbe0 [ 34.919205] [<ffffffffc02e8220>] amdgpu_pmops_suspend+0x20/0x30 [amdgpu] [ 34.919207] [<ffffffff925c56ff>] pci_pm_suspend+0x6f/0x150 [ 34.919208] [<ffffffff925c5690>] ? pci_pm_freeze+0xf0/0xf0 [ 34.919210] [<ffffffff926b45c6>] dpm_run_callback+0x46/0x90 [ 34.919212] [<ffffffff926b49db>] __device_suspend+0xfb/0x2a0 [ 34.919213] [<ffffffff926b4b9f>] async_suspend+0x1f/0xa0 [ 34.919214] [<ffffffff922c918f>] async_run_entry_fn+0x3f/0x130 [ 34.919216] [<ffffffff922b9d4f>] process_one_work+0x17f/0x440 [ 34.919217] [<ffffffff922bade6>] worker_thread+0x126/0x3c0 [ 34.919218] [<ffffffff922bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0 [ 34.919220] [<ffffffff922c1c31>] kthread+0xd1/0xe0 [ 34.919221] [<ffffffff922c1b60>] ? insert_kthread_work+0x40/0x40 [ 34.919222] [<ffffffff92974c1d>] ret_from_fork_nospec_begin+0x7/0x21 [ 34.919224] [<ffffffff922c1b60>] ? insert_kthread_work+0x40/0x40 [ 34.919224] ---[ end trace 38cd9f65c963adad ]--- Signed-off-by: Kevin Wang <kevin1.wang@amd.com> Reviewed-by: Oak Zeng <Oak.Zeng@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-02-27drm/amdkfd: Don't assign dGPUs to APU topology devicesFelix Kuehling1-3/+9
[ Upstream commit bbdf514fe5648566b0754476cbcb92ac3422dde2 ] dGPUs need their own topology devices. Don't assign them to APU topology devices with CPU cores. Bug: https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/issues/66 Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Tested-by: Elias Konstantinidis <ekondis@gmail.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2019-01-26drm/amdkfd: fix interrupt spin lockChristian König1-2/+3
[ Upstream commit 2383a767c0ca06f96534456d8313909017c6c8d0 ] Vega10 has multiple interrupt rings, so this can be called from multiple calles at the same time resulting in: [ 71.779334] ================================ [ 71.779406] WARNING: inconsistent lock state [ 71.779478] 4.19.0-rc1+ #44 Tainted: G W [ 71.779565] -------------------------------- [ 71.779637] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage. [ 71.779740] kworker/6:1/120 [HC0[0]:SC0[0]:HE1:SE1] takes: [ 71.779832] 00000000ad761971 (&(&kfd->interrupt_lock)->rlock){?...}, at: kgd2kfd_interrupt+0x75/0x100 [amdgpu] [ 71.780058] {IN-HARDIRQ-W} state was registered at: [ 71.780115] _raw_spin_lock+0x2c/0x40 [ 71.780180] kgd2kfd_interrupt+0x75/0x100 [amdgpu] [ 71.780248] amdgpu_irq_callback+0x6c/0x150 [amdgpu] [ 71.780315] amdgpu_ih_process+0x88/0x100 [amdgpu] [ 71.780380] amdgpu_irq_handler+0x20/0x40 [amdgpu] [ 71.780409] __handle_irq_event_percpu+0x49/0x2a0 [ 71.780436] handle_irq_event_percpu+0x30/0x70 [ 71.780461] handle_irq_event+0x37/0x60 [ 71.780484] handle_edge_irq+0x83/0x1b0 [ 71.780506] handle_irq+0x1f/0x30 [ 71.780526] do_IRQ+0x53/0x110 [ 71.780544] ret_from_intr+0x0/0x22 [ 71.780566] cpuidle_enter_state+0xaa/0x330 [ 71.780591] do_idle+0x203/0x280 [ 71.780610] cpu_startup_entry+0x6f/0x80 [ 71.780634] start_secondary+0x1b0/0x200 [ 71.780657] secondary_startup_64+0xa4/0xb0 Fix this by always using irq save spin locks. Signed-off-by: Christian König <christian.koenig@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2018-12-19drm/amdkfd: add new vega10 pci idsAlex Deucher1-0/+6
commit 756e16bf79f2815e7c83a04881b5545b55a99fd3 upstream. New vega10 ids. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Cc: stable@vger.kernel.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-10-04drm/amdkfd: Fix incorrect use of process->mmFelix Kuehling1-8/+29
This mm_struct pointer should never be dereferenced. If running in a user thread, just use current->mm. If running in a kernel worker use get_task_mm to get a safe reference to the mm_struct. Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-20drm/amdkfd: Fix ATS capablity was not reported correctly on some APUsYong Zhao3-6/+29
Because CRAT_CU_FLAGS_IOMMU_PRESENT was not set in some BIOS crat, we need to workaround this. For future compatibility, we also overwrite the bit in capability according to the value of needs_iommu_device. Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-20drm/amdkfd: Change the control stack MTYPE from UC to NC on GFX9Yong Zhao2-2/+3
CWSR fails on Raven if the control stack is MTYPE_UC, which is used for regular GART mappings. As a workaround we map it using MTYPE_NC. The MEC firmware expects the control stack at one page offset from the start of the MQD so it is part of the MQD allocation on GFXv9. AMDGPU added a memory allocation flag just for this purpose. Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-17drm/amdkfd: Call kfd2kgd.set_compute_idleFelix Kuehling1-2/+5
User mode queue submissions don't go through KFD. Therefore we don't know exactly when compute is idle or not idle. We use the existence of user mode queues on a device as an approximation. register_process is called when the first queue of a process is created. Conversely unregister_process is called when the last queue is destroyed. The first process that is registered takes compute out of idle. The last process that is unregisters sets compute back to idle. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Eric Huang <JinHuiEric.Huang@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-15drm/amdkfd: Add CU-masking ioctl to KFDFelix Kuehling9-1/+219
CU-masking allows a KFD client to control the set of CUs used by a user mode queue for executing compute dispatches. This can be used for optimizing the partitioning of the GPU and minimize conflicts between concurrent tasks. Signed-off-by: Flora Cui <flora.cui@amd.com> Signed-off-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Eric Huang <JinHuiEric.Huang@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-13drm/amdkfd: Enable Raven for KFDYong Zhao1-0/+16
Add DID and kfd_device_info for Raven. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-13drm/amdkfd: Optimize out some duplicated code in kfd_signal_iommu_event()Yong Zhao1-15/+11
memory_exception_data is already initialized for not-present faults. It only needs to be overridden for permission faults. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-13drm/amdkfd: Workaround to accommodate Raven too many PPR issueYong Zhao1-5/+16
On Raven multiple PPRs can be queued up by the hardware. When the first of those requests is handled by the IOMMU driver, the memory access succeeds. After that the application may be done with the memory and unmap it. At that point the page table entries are invalidated, but there are still outstanding duplicate PPRs for those addresses. When the IOMMU driver processes those duplicate requests, it finds invalid page table entries and triggers an invalid PPR fault. As a workaround, don't signal invalid PPR faults on Raven to avoid segfaulting applications that haven't done anything wrong. As a side effect, real GPU memory access faults may go unnoticed by the application. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-13drm/amdkfd: Avoid flooding dmesg on Raven due to IOMMU issuesYong Zhao1-1/+1
On Raven Invalid PPRs (peripheral page requests) can be reported because multiple PPRs can be still queued when memory is freed. Apply a rate limit to avoid flooding the log in this case. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-13drm/amdkfd: Make SDMA engine number an ASIC-dependent variableYong Zhao5-15/+36
On Raven there is only one SDMA engine instead of previously assumed two, so we need to adapt our code to this new scenario. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-13drm/amdkfd: Consolidate duplicate memory banks info in topologyYong Zhao1-11/+46
If there are several memory banks that has the same properties in CRAT, we aggregate them into one memory bank. This cleans up memory banks on APUs (e.g. Raven) where the CRAT reports each memory channel as a separate bank. This only confuses user mode, which only deals with virtual memory. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Clean up reference of radeonYong Zhao5-6/+41
Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Replace mqd with mqd_mgr as the variable name for mqd_managerYong Zhao5-66/+68
This will make reading code much easier. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Use module parameters noretry as the internal variable nameYong Zhao3-8/+10
This makes all module parameters use the same form. Meanwhile clean up the surrounding code. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Introduce KFD module parameter halt_if_hws_hangYong Zhao3-0/+16
This avoids triggering a GPU reset or otherwise changing the HW state. Instead KFD will hang, which allows HW debugging tools to analyze the problem. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Add debugfs interface to trigger HWS hangShaoyun Liu5-0/+113
Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Fix kernel queue 64 bit doorbell offset calculationShaoyun Liu1-4/+5
The bitmap index calculation should reverse the logic used on allocation so it will clear the same bit used on allocation Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Implement hang detection in KFD and call amdgpuShaoyun Liu2-1/+24
The reset will be performed in a new hw_exception work thread to handle HWS hang without blocking the thread that detected the hang. Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Implement GPU reset handlers in KFDShaoyun Liu5-3/+75
Lock KFD and evict existing queues on reset. Notify user mode by signaling hw_exception events. Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Add gpu reset interface and place holderShaoyun Liu3-0/+16
Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: fix zero reading of VMID and PASID for HawaiiLan Xiao5-9/+53
Upon VM Fault, the VMID and PASID written by HW are zeros in Hawaii. Instead of reading from ih_ring_entry, read directly from the registers. This workaround fix the soft hang issues caused by mishandled VM Fault in Hawaii. Signed-off-by: Lan Xiao <Lan.Xiao@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Handle VM faults in KFDshaoyunl6-6/+97
1. Pre-GFX9 the amdgpu ISR saves the vm-fault status and address per per-vmid. amdkfd needs to get the information from amdgpu through the new get_vm_fault_info interface. On GFX9 and later, all the required information is in the IH ring 2. amdkfd unmaps all queues from the faulting process and create new run-list without the guilty process 3. amdkfd notifies the runtime of the vm fault trap via EVENT_TYPE_MEMORY Signed-off-by: shaoyun liu <shaoyun.liu@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: send SIGSEGV to process upon KFD_EVENT_TYPE_MEMORYMoses Reuben1-0/+7
Signed-off-by: Moses Reuben <moses.reuben@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Fix error codes in kfd_get_processWei Lu1-0/+2
Return ERR_PTR(-EINVAL) if kfd_get_process fails to find the process. This fixes kernel oopses when a child process calls KFD ioctls with a file descriptor inherited from the parent process. Signed-off-by: Wei Lu <wei.lu2@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Fix race between scheduler and context restoreJay Cornwall3-230/+262
The scheduler may raise SQ_WAVE_STATUS.SPI_PRIO via SQ_CMD before context restore has completed. Restoring SPI_PRIO=0 after this point may cause context save to fail as the lower priority wavefronts are not selected for execution among spin-waiting wavefronts. Leave SPI_PRIO at its SPI-initialized or scheduler-raised value. v2: Also fix race with exception handler Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Stop using GFP_NOIO explicitlyFelix Kuehling4-6/+6
This is no longer needed with the memalloc_nofs_save/restore in dqm_lock/unlock. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-12drm/amdkfd: Reliably prevent reclaim-FS while holding DQM lockFelix Kuehling2-43/+62
This is needed to prevent deadlocks when MMU notifiers run in reclaim-FS context and take the DQM lock for userptr evictions. Previously this was done by making all memory allocations under DQM locks GFP_NOIO. This is error prone. Using memalloc_nofs_save/restore will reliably affect all memory allocations anywhere in the kernel while the DQM lock is held. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-07-11drm/admkfd use modern ktime accessorsArnd Bergmann1-6/+2
getrawmonotonic64() and get_monotonic_boottime64() are deprecated because of the nonstandard naming. The replacement functions ktime_get_raw_ns() and ktime_get_boot_ns() also simplify the callers. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>. Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-05-15Merge tag 'drm-amdkfd-next-2018-05-14' of ↵Dave Airlie36-750/+4409
git://people.freedesktop.org/~gabbayo/linux into drm-next This is amdkfd pull for 4.18. The major new features are: - Add support for GFXv9 dGPUs (VEGA) - Add support for userptr memory mapping In addition, there are a couple of small fixes and improvements, such as: - Fix lock handling - Fix rollback packet in kernel kfd_queue - Optimize kfd signal handling - Fix CP hang in APU Signed-off-by: Dave Airlie <airlied@redhat.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180514070126.GA1827@odedg-x270
2018-04-24drm/amdkfd: fix build, select MMU_NOTIFIERRandy Dunlap1-0/+1
When CONFIG_MMU_NOTIFIER is not enabled, struct mmu_notifier has an incomplete type definition, which causes build errors. ../drivers/gpu/drm/amd/amdkfd/kfd_priv.h:607:22: error: field 'mmu_notifier' has incomplete type ../include/linux/kernel.h:979:32: error: dereferencing pointer to incomplete type ../include/linux/kernel.h:980:18: error: dereferencing pointer to incomplete type ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:434:2: error: implicit declaration of function 'mmu_notifier_unregister_no_release' [-Werror=implicit-function-declaration] ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:435:2: error: implicit declaration of function 'mmu_notifier_call_srcu' [-Werror=implicit-function-declaration] ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:438:21: error: variable 'kfd_process_mmu_notifier_ops' has initializer but incomplete type ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:439:2: error: unknown field 'release' specified in initializer ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:439:2: warning: excess elements in struct initializer [enabled by default] ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:439:2: warning: (near initialization for 'kfd_process_mmu_notifier_ops') [enabled by default] ../drivers/gpu/drm/amd/amdkfd/kfd_process.c:534:2: error: implicit declaration of function 'mmu_notifier_register' [-Werror=implicit-function-declaration] Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Tested-by: Anders Roxell <anders.roxell@linaro.org> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-04-24drm/amdkfd: fix clock counter retrieval for node without GPUAndres Rodriguez1-6/+7
Currently if a user requests clock counters for a node without a GPU resource we will always return EINVAL. Instead if no GPU resource is attached, fill the gpu_clock_counter argument with zeroes so that we may proceed and return valid CPU counters. Signed-off-by: Andres Rodriguez <andres.rodriguez@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-04-24drm/amdkfd: Fix the error return code in kfd_ioctl_unmap_memory_from_gpu()Wei Yongjun1-1/+1
Passing NULL pointer to PTR_ERR will result in return value of 0 indicating success which is clearly not what it is intended here. This patch returns -EINVAL instead. v2: change ret code to -ENODEV Fixes: 5ec7e02854b3 ("drm/amdkfd: Add ioctls for GPUVM memory management") Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-04-24drm/amdkfd: kfd_dev_is_large_bar() can be statickbuild test robot1-1/+1
Fixes: 5ec7e02854b3 ("drm/amdkfd: Add ioctls for GPUVM memory management") Signed-off-by: Fengguang Wu <fengguang.wu@intel.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-04-14drm/amdkfd: Remove vlaLaura Abbott2-3/+7
There's an ongoing effort to remove VLAs[1] from the kernel to eventually turn on -Wvla. Switch to a constant value that covers all hardware. [1] https://lkml.org/lkml/2018/3/7/621 Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Laura Abbott <labbott@redhat.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2018-05-02drm/amdkfd: Add sanity checks in IRQ handlersFelix Kuehling2-21/+39
Only accept interrupts from KFD VMIDs. Just checking for a PASID may not be enough because amdgpu started using PASIDs to map VM faults to processes. Warn if an IRQ doesn't have a valid PASID (indicating a firmware bug). Suggested-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Suggested-by: Oak Zeng <Oak.Zeng@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
generated by cgit v1.2.3 (git 2.39.0) at 2025-09-27 15:36:03 +0300