summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/xe/xe_gt.c
AgeCommit message (Collapse)AuthorFilesLines
2026-03-23drm/xe/vf: Improve getting clean NULL contextMichal Wajdeczko1-3/+6
There is a small risk that when fetching a NULL context image the VF may get a tweaked context image prepared by another VF that was previously running on the engine before the GuC scheduler switched the VFs. To avoid that risk, without forcing GuC scheduler to trigger costly engine reset on every VF switch, use a watchdog mechanism that when configured with impossible condition, triggers an interrupt, which GuC will handle by doing an engine reset. Also adjust job size to account for additional dwords with watchdog setup. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com> Link: https://patch.msgid.link/20260303201354.17948-4-michal.wajdeczko@intel.com
2026-03-11drm/xe/nvlp: Implement Wa_14026539277Gustavo Sousa1-0/+39
Implement the KMD part of Wa_14026539277, which applies to NVL-P A0. The KMD implementation is just one component of the workaround, which also depends on Pcode to implement its part in order to be complete. v2: - Add FUNC(xe_rtp_match_not_sriov_vf) to skip applying the workaround to SRIOV VFs. (Matt) v3: - Make Wa_14026539277 a device workaround instead of a GT workaround. (Matt) v4: - Drop FUNC(xe_rtp_match_not_sriov_vf) and use a direct check with IS_SRIOV_VF() in the workaround implementation. (Matt) Reviewed-by: Matt Roper <matthew.d.roper@intel.com> # v3 Link: https://patch.msgid.link/20260309-extra-nvl-p-enabling-patches-v5-5-be9c902ee34e@intel.com Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
2026-03-06drm/xe: Don't disable GuCRC in suspend pathVinay Belgaumkar1-0/+2
GuCRC should not be disabled in xe_guc_stop_prepare() as C6 is a prerequisite for s0ix and s2idle. This is a regression caused by the patch below. Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/7510 Fixes: 40a684f91d26 ("drm/xe: Decouple GuC RC code from xe_guc_pc") Cc: Riana Tauro <riana.tauro@intel.com> Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com> Reviewed-by: Riana Tauro <riana.tauro@intel.com> Link: https://patch.msgid.link/20260303181416.3880937-1-vinay.belgaumkar@intel.com
2026-02-19drm/xe/wa: Steer RMW of MCR registers while building default LRCMatt Roper1-12/+54
When generating the default LRC, if a register is not masked, we apply any save-restore programming necessary via a read-modify-write sequence that will ensure we only update the relevant bits/fields without clobbering the rest of the register. However some of the registers that need to be updated might be MCR registers which require steering to a non-terminated instance to ensure we can read back a valid, non-zero value. The steering of reads originating from a command streamer is controlled by register CS_MMIO_GROUP_INSTANCE_SELECT. Emit additional MI_LRI commands to update the steering before any RMW of an MCR register to ensure the reads are performed properly. Note that needing to perform a RMW of an MCR register while building the default LRC is pretty rare. Most of the MCR registers that are part of an engine's LRCs are also masked registers, so no MCR is necessary. Fixes: f2f90989ccff ("drm/xe: Avoid reading RMW registers in emit_wa_job") Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com> Link: https://patch.msgid.link/20260206223058.387014-2-matthew.d.roper@intel.com Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
2026-02-18drm/xe: Convert GT stats to per-cpu countersMatthew Brost1-0/+5
Current GT statistics use atomic64_t counters. Atomic operations incur a global coherency penalty. Transition to dynamic per-cpu counters using alloc_percpu(). This allows stats to be incremented via this_cpu_add(), which compiles to a single non-locking instruction. This approach keeps the hot-path updates local to the CPU, avoiding expensive cross-core cache invalidation traffic. Use for_each_possible_cpu() during aggregation and clear operations to ensure data consistency across CPU hotplug events. Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Stuart Summers <stuart.summers@intel.com> Link: https://patch.msgid.link/20260217200552.596718-1-matthew.brost@intel.com
2026-02-06drm/xe: Decouple GuC RC code from xe_guc_pcVinay Belgaumkar1-1/+0
Move enable/disable GuC RC logic into the new file. This will allow us to independently enable/disable GuC RC and not rely on SLPC related functions. GuC already provides separate H2G interfaces to setup GuC RC and SLPC. Cc: Riana Tauro <riana.tauro@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Riana Tauro <riana.tauro@intel.com> Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com> Link: https://patch.msgid.link/20260204014234.2867763-2-vinay.belgaumkar@intel.com
2026-01-29drm/xe/gt: Use CLASS() for forcewake in xe_gt_enable_comp_1wcohShuicheng Lin1-5/+2
Adopt the scoped forcewake management using CLASS(xe_force_wake, ...) to simplify the code and ensure proper resource release. Cc: Xin Wang <x.wang@intel.com> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com> Reviewed-by: Xin Wang <x.wang@intel.com> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com> Link: https://patch.msgid.link/20260123180425.3262944-2-shuicheng.lin@intel.com
2026-01-15drm/xe: Cleanup unused header includesMatt Roper1-2/+0
clangd reports many "unused header" warnings throughout the Xe driver. Start working to clean this up by removing unnecessary includes in our .c files and/or replacing them with explicit includes of other headers that were previously being included indirectly. By far the most common offender here was unnecessary inclusion of xe_gt.h. That likely originates from the early days of xe.ko when xe_mmio did not exist and all register accesses, including those unrelated to GTs, were done with GT functions. There's still a lot of additional #include cleanup that can be done in the headers themselves; that will come as a followup series. v2: - Squash the 79-patch series down to a single patch. (MattB) Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/20260115032803.4067824-2-matthew.d.roper@intel.com Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
2026-01-10drm/xe: Allow compressible surfaces to be 1-way coherentXin Wang1-0/+32
Previously, compressible surfaces were required to be non-coherent (allocated as WC) because compression and coherency were mutually exclusive. Starting with Xe3, hardware supports combining compression with 1-way coherency, allowing compressible surfaces to be allocated as WB memory. This provides applications with more efficient memory allocation by avoiding WC allocation overhead that can cause system stuttering and memory management challenges. The implementation adds support for compressed+coherent PAT entry for the xe3_lpg devices and updates the driver logic to handle the new compression capabilities. v2: (Matthew Auld) - Improved error handling with XE_IOCTL_DBG() - Enhanced documentation and comments - Fixed xe_bo_needs_ccs_pages() outdated compression assumptions v3: - Improve WB compression support detection by checking PAT table instead of version check v4: - Add XE_CACHE_WB_COMPRESSION, which simplifies the logic. v5: - Use U16_MAX for the invalid PAT index. (Matthew Auld) Bspec: 71582, 59361, 59399 Cc: Matthew Auld <matthew.auld@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Signed-off-by: Xin Wang <x.wang@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patch.msgid.link/20260109093007.546784-1-x.wang@intel.com Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
2025-12-13drm/xe: Restore engine registers before restarting schedulers after GT resetJan Maslak1-3/+4
During GT reset recovery in do_gt_restart(), xe_uc_start() was called before xe_reg_sr_apply_mmio() restored engine-specific registers. This created a race window where the scheduler could run jobs before hardware state was fully restored. This caused failures in eudebug tests (xe_exec_sip_eudebug@breakpoint- waitsip-*) where TD_CTL register (containing TD_CTL_GLOBAL_DEBUG_ENABLE) wasn't restored before jobs started executing. Breakpoints would fail to trigger SIP entry because the debug enable bit wasn't set yet. Fix by moving xe_uc_start() after all MMIO register restoration, including engine registers and CCS mode configuration, ensuring all hardware state is fully restored before any jobs can be scheduled. Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs") Signed-off-by: Jan Maslak <jan.maslak@intel.com> Reviewed-by: Jonathan Cavitt <jonathan.cavitt@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/20251210145618.169625-2-jan.maslak@intel.com
2025-12-03Merge drm/drm-next into drm-xe-nextThomas Hellström1-1/+0
Backmerging to bring in a needed dependency for the Xe VFIO driver variant. This should ideally have been done before we commited that, so we now have a small window in drm-xe-next where that driver doesn't compile. Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202512030331.I8CveRre-lkp@intel.com/ Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
2025-12-02drm/xe/gt: Use scope-based forcewakeRaag Jadav1-22/+10
Switch runtime PM code to use scope-based forcewake for consistency with other parts of the driver. Signed-off-by: Raag Jadav <raag.jadav@intel.com> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> Link: https://patch.msgid.link/20251128082212.294592-1-raag.jadav@intel.com Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
2025-11-27drm/xe/gt: Introduce runtime suspend/resumeRaag Jadav1-0/+60
If power state is retained between suspend/resume cycle, we don't need to perform full GT re-initialization. Introduce runtime helpers for GT which greatly reduce suspend/resume delay. v2: Drop redundant xe_gt_sanitize() and xe_guc_ct_stop() (Daniele) Use runtime naming for guc helpers (Daniele) v3: Drop redundant logging, add kernel doc (Michal) Use runtime naming for ct helpers (Michal) v4: Fix tags (Rodrigo) v5: Include host_l2_vram workaround (Daniele) Reuse xe_guc_submit_enable/disable() helpers (Daniele) Co-developed-by: Riana Tauro <riana.tauro@intel.com> Signed-off-by: Riana Tauro <riana.tauro@intel.com> Signed-off-by: Raag Jadav <raag.jadav@intel.com> Acked-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com> Link: https://patch.msgid.link/20251030122357.128825-5-raag.jadav@intel.com
2025-11-21Merge tag 'v6.18-rc6' into drm-nextDave Airlie1-1/+0
Linux 6.18-rc6 Backmerge in order to merge msm next Signed-off-by: Dave Airlie <airlied@redhat.com>
2025-11-19drm/xe/gt: Use scope-based cleanupMatt Roper1-89/+41
Using scope-based cleanup for forcewake and runtime PM allows us to reduce or eliminate some of the goto-based error handling and simplify several functions. v2: - Drop changes to do_gt_restart(). This function still has goto-based logic, making scope-based cleanup unsafe for now. (Gustavo) Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com> Link: https://patch.msgid.link/20251118164338.3572146-32-matthew.d.roper@intel.com Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
2025-11-04drm/xe: Add xe_guc_pagefault layerMatthew Brost1-6/+0
Add xe_guc_pagefault layer (producer) which parses G2H fault messages messages into struct xe_pagefault, forwards them to the page fault layer (consumer) for servicing, and provides a vfunc to acknowledge faults to the GuC upon completion. Replace the old (and incorrect) GT page fault layer with this new layer throughout the driver. As part of this change, the ACC handling code has been removed, as it is dead code that is currently unused. v2: - Include engine instance (Stuart) Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Tested-by: Francois Dugast <francois.dugast@intel.com> Link: https://patch.msgid.link/20251031165416.2871503-7-matthew.brost@intel.com
2025-11-04drm/xe: Implement xe_pagefault_resetMatthew Brost1-0/+2
Squash any pending faults on the GT being reset by setting the GT field in struct xe_pagefault to NULL. v4: - Only do reset it page faults queues initialized (CI) Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Tested-by: Francois Dugast <francois.dugast@intel.com> Link: https://patch.msgid.link/20251031165416.2871503-4-matthew.brost@intel.com
2025-11-03drm/xe/gt: Synchronize GT reset with device unbindBalasubramani Vivekanandan1-0/+7
When unbinding wait for any GT reset in progress to complete. Unbinding will release the mmio mapping but mmio operations are performed during GT reset causing Kernel panic. Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://patch.msgid.link/20251103123144.3231829-5-balasubramani.vivekanandan@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-11-03drm/xe: Inline gt_reset in the workerLucas De Marchi1-18/+11
gt_reset() doesn't make sense by itself: it can only be called as part of the worker. Inline it there to avoid it being called from elsewhere and clarify the gt_reset() vs do_gt_reset() paths. Note that the error return from gt_reset() was just being ignored. Also add a comment to the xe_pm_runtime_put() to make sure the get()/put() pair is clear. Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/20251031222244.37735-2-lucas.demarchi@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-10-29drm/xe: Do not wake device during a GT resetMatthew Brost1-7/+12
Waking the device during a GT reset can lead to unintended memory allocation, which is not allowed since GT resets occur in the reclaim path. Prevent this by holding a PM reference while a reset is in flight. Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs") Cc: stable@vger.kernel.org Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://lore.kernel.org/r/20251022005538.828980-3-matthew.brost@intel.com (cherry picked from commit 480b358e7d8ef69fd8f1b0cad6e07c7d70a36ee4) Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-10-22drm/xe: Do not wake device during a GT resetMatthew Brost1-7/+12
Waking the device during a GT reset can lead to unintended memory allocation, which is not allowed since GT resets occur in the reclaim path. Prevent this by holding a PM reference while a reset is in flight. Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs") Cc: stable@vger.kernel.org Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://lore.kernel.org/r/20251022005538.828980-3-matthew.brost@intel.com
2025-10-09drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VFMatthew Brost1-4/+14
VF CCS restore is a primary GT operation on which the media GT depends. Therefore, it doesn't make much sense to run these operations in parallel. To address this, point the media GT's ordered work queue to the primary GT's ordered work queue on platforms that require (PTL VFs) CCS restore as part of VF post-migration recovery. v7: - Remove bool from xe_gt_alloc (Lucas) v9: - Fix typo (Lucas) Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20251008214532.3442967-32-matthew.brost@intel.com
2025-10-09drm/xe/vf: Don't allow GT reset to be queued during VF post migration recoveryMatthew Brost1-9/+0
With well-behaved software, a GT reset should never occur, nor should it happen during VF post-migration recovery. If it does, trigger a warning but suppress the GT reset, as VF post-migration recovery is expected to bring the VF back to a working state. v3: - Better commit message (Tomasz) v5: - Use xe_gt_WARN_ON (Michal) Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Tomasz Lis <tomasz.lis@intel.com> Link: https://lore.kernel.org/r/20251008214532.3442967-17-matthew.brost@intel.com
2025-10-09drm/xe/vf: Teardown VF post migration worker on driver unloadMatthew Brost1-0/+6
Be cautious and ensure the VF post-migration worker is not running during driver unload. v3: - More teardown later in driver init, use devm (Tomasz) Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Tomasz Lis <tomasz.lis@intel.com> Link: https://lore.kernel.org/r/20251008214532.3442967-16-matthew.brost@intel.com
2025-10-09drm/xe/vf: Make VF recovery run on per-GT workerMatthew Brost1-0/+6
VF recovery is a per-GT operation, so it makes sense to isolate it to a per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state. v2: - Fix lockdep splat (Adam) - Use xe_sriov_vf_migration_supported helper v3: - Drop xe_gt_sriov_ prefix for private functions (Michal) - Drop message in xe_gt_sriov_vf_migration_init_early (Michal) - Logic rework in vf_post_migration_notify_resfix_done (Michal) - Rework init sequence layering (Michal) Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Tomasz Lis <tomasz.lis@intel.com> Link: https://lore.kernel.org/r/20251008214532.3442967-10-matthew.brost@intel.com
2025-10-06drm/xe/pf: Make the late-initialization really lateMichal Wajdeczko1-3/+1
While the late PF per-GT initialization is done quite late in the single GT initialization flow, in case of multi-GT platforms, it may still be done before other GT early initialization. That leads to some issues during unwind, when there are cross-GT dependencies, like resource cleanup that is shared by both GTs, but the other GT may already be sanitized or disabled. The following errors could be observed when trying to unload the PF driver with some LMEM/VRAM already provisioned for few VFs: [ ] xe 0000:03:00.0: DEVRES REL ffff88814708f240 fini_config (16 bytes) [ ] xe 0000:03:00.0: [drm:lmtt_write_pte [xe]] PF: LMTT: WRITE level=2 index=1 pte=0x0 [ ] xe 0000:03:00.0: [drm:lmtt_invalidate_hw [xe]] PF: LMTT: num_fences=2 err=-19 [ ] xe 0000:03:00.0: [drm:lmtt_pt_free [xe]] PF: LMTT: level=0 addr=53a470000 [ ] xe 0000:03:00.0: [drm:lmtt_pt_free [xe]] PF: LMTT: level=1 addr=53a4b0000 [ ] xe 0000:03:00.0: [drm:lmtt_invalidate_hw [xe]] PF: LMTT: num_fences=2 err=-19 [ ] xe 0000:03:00.0: [drm] PF: LMTT0 invalidation failed (-ENODEV) [ ] xe 0000:03:00.0: [drm:lmtt_write_pte [xe]] PF: LMTT: WRITE level=2 index=2 pte=0x0 [ ] xe 0000:03:00.0: [drm:lmtt_invalidate_hw [xe]] PF: LMTT: num_fences=2 err=-19 [ ] xe 0000:03:00.0: [drm:lmtt_pt_free [xe]] PF: LMTT: level=0 addr=539b70000 [ ] xe 0000:03:00.0: [drm:lmtt_pt_free [xe]] PF: LMTT: level=1 addr=539bf0000 [ ] xe 0000:03:00.0: [drm:lmtt_invalidate_hw [xe]] PF: LMTT: num_fences=2 err=-19 [ ] xe 0000:03:00.0: [drm] PF: LMTT0 invalidation failed (-ENODEV) Move all PF per-GT late initialization to the already defined late SR-IOV initialization function to allow proper order of the cleanup actions. While around, format all PF function stubs as one-liners, like many other stubs are defined in the Xe driver. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Jonathan Cavitt <jonathan.cavitt@intel.com> Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com> Link: https://lore.kernel.org/r/20251004162008.1782-1-michal.wajdeczko@intel.com
2025-10-02drm/xe: Detect GT workqueue allocation failureMichal Wajdeczko1-15/+5
The allocation of the per-GT workqueue may fail and we shouldn't ignore that. While around use drm managed allocation function to drop our custom fini action. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20251001144051.202040-1-michal.wajdeczko@intel.com
2025-09-11drm/xe/guc: Set RCS/CCS yield policyDaniele Ceraolo Spurio1-1/+1
All recent platforms (including all the ones officially supported by the Xe driver) do not allow concurrent execution of RCS and CCS workloads from different address spaces, with the HW blocking the context switch when it detects such a scenario. The DUAL_QUEUE flag helps with this, by causing the GuC to not submit a context it knows will not be able to execute. This, however, causes a new problem: if RCS and CCS queues have pending workloads from different address spaces, the GuC needs to choose from which of the 2 queues to pick the next workload to execute. By default, the GuC prioritizes RCS submissions over CCS ones, which can lead to CCS workloads being significantly (or completely) starved of execution time. The driver can tune this by setting a dedicated scheduling policy KLV; this KLV allows the driver to specify a quantum (in ms) and a ratio (percentage value between 0 and 100), and the GuC will prioritize the CCS for that percentage of each quantum. Given that we want to guarantee enough RCS throughput to avoid missing frames, we set the yield policy to 20% of each 80ms interval. v2: updated quantum and ratio, improved comment, use xe_guc_submit_disable in gt_sanitize Fixes: d9a1ae0d17bd ("drm/xe/guc: Enable WA_DUAL_QUEUE for newer platforms") Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: John Harrison <John.C.Harrison@Intel.com> Cc: Vinay Belgaumkar <vinay.belgaumkar@intel.com> Reviewed-by: John Harrison <John.C.Harrison@Intel.com> Tested-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com> Link: https://lore.kernel.org/r/20250905235632.3333247-2-daniele.ceraolospurio@intel.com
2025-08-27drm/xe: Split TLB invalidation code in frontend and backendMatthew Brost1-2/+0
The frontend exposes an API to the driver to send invalidations, handles sequence number assignment, synchronization (fences), and provides a timeout mechanism. The backend issues the actual invalidation to the hardware (or firmware). The new layering easily allows issuing TLB invalidations to different hardware or firmware interfaces. Normalize some naming while here too. Signed-off-by: Stuart Summers <stuart.summers@intel.com> Reviewed-by: Stuart Summers <stuart.summers@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250826182911.392550-10-stuart.summers@intel.com
2025-08-27drm/xe: Decouple TLB invalidations from GTMatthew Brost1-3/+3
Decouple TLB invalidations from the GT by updating the TLB invalidation layer to accept a `struct xe_tlb_inval` instead of a `struct xe_gt`. Also, rename *gt_tlb* to *tlb*. The internals of the TLB invalidation code still operate on a GT, but this is now hidden from the rest of the driver. Signed-off-by: Stuart Summers <stuart.summers@intel.com> Reviewed-by: Stuart Summers <stuart.summers@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250826182911.392550-7-stuart.summers@intel.com
2025-08-27drm/xe: s/tlb_invalidation/tlb_invalMatthew Brost1-5/+5
tlb_invalidation is a bit verbose leading to ugly wraps in the code, shorten to tlb_inval. Signed-off-by: Stuart Summers <stuart.summers@intel.com> Reviewed-by: Stuart Summers <stuart.summers@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250826182911.392550-4-stuart.summers@intel.com
2025-08-27drm/xe: Cancel pending TLB inval workers on teardownStuart Summers1-0/+2
Add a new _fini() routine on the GT TLB invalidation side to handle this worker cleanup on driver teardown. v2: Move the TLB teardown to the gt fini() routine called during gt_init rather than in gt_alloc. This way the GT structure stays alive for while we reset the TLB state. Signed-off-by: Stuart Summers <stuart.summers@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250826182911.392550-3-stuart.summers@intel.com
2025-08-08drm/xe: Update function names for GT specific workaroundsMatt Atwood1-2/+2
Now that there distinctly different OOB functions, update the names to reflect the IPs they interact with. Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://lore.kernel.org/r/20250807214224.32728-2-matthew.s.atwood@intel.com Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2025-08-08drm/xe: rename XE_WA to XE_GT_WAMatt Atwood1-3/+3
Now that there are two types of wa tables and infrastructure, be more concise in the naming of GT wa macros. v2: update the documentation Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://lore.kernel.org/r/20250807214224.32728-1-matthew.s.atwood@intel.com Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2025-08-04drm/xe: Block reset while recovering from VF migrationTomasz Lis1-0/+10
Resetting GuC during recovery could interfere with the recovery process. Such reset might be also triggered without justification, due to migration taking time, rather than due to the workload not progressing. Doing GuC reset during the recovery would cause exit of RESFIX state, and therefore continuation of GuC work while fixups are still being applied. To avoid that, reset needs to be blocked during the recovery. This patch blocks the reset during recovery. Reset request in that time range will be stalled, and unblocked only after GuC goes out of RESFIX state. In case a reset procedure already started while the recovery is triggered, there isn't much we can do - we cannot wait for it to finish as it involves waiting for hardware, and we can't be sure at which exact point of the reset procedure the GPU got switched. Therefore, the rare cases where migration happens while reset is in progress, are still dangerous. Resets are not a part of the standard flow, and cause unfinished workloads - that will happen during the reset interrupted by migration as well, so it doesn't diverge that much from what normally happens during such resets. v2: Introduce a new atomic for reset blocking, as we cannot reuse `stopped` atomic (that could lead to losing a workload). v3: Switched atomic functs to ones which include proper barriers Signed-off-by: Tomasz Lis <tomasz.lis@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: Michal Winiarski <michal.winiarski@intel.com> Reviewed-by: Michal Winiarski <michal.winiarski@intel.com> Link: https://lore.kernel.org/r/20250802031045.1127138-4-tomasz.lis@intel.com Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
2025-07-30drm/xe/vf: Fix VM crash during VF driver releaseSatyanarayana K V P1-4/+2
The VF CCS save/restore series (patchwork #149108) has a dependency on the migration framework. A recent migration update in commit d65ff1ec8535 ("drm/xe: Split xe_migrate allocation from initialization") caused a VM crash during XE driver release for iGPU devices. Oops: general protection fault, probably for non-canonical address 0x6b6b6b6b6b6b6b83: 0000 [#1] SMP NOPTI RIP: 0010:xe_lrc_ring_head+0x12/0xb0 [xe] Call Trace: xe_sriov_vf_ccs_fini+0x1e/0x40 [xe] devm_action_release+0x12/0x30 release_nodes+0x3a/0x120 devres_release_all+0x96/0xd0 device_unbind_cleanup+0x12/0x80 device_release_driver_internal+0x23a/0x280 device_release_driver+0x12/0x20 pci_stop_bus_device+0x69/0x90 pci_stop_and_remove_bus_device+0x12/0x30 pci_iov_remove_virtfn+0xbd/0x130 sriov_disable+0x42/0x100 pci_disable_sriov+0x34/0x50 xe_pci_sriov_configure+0xf71/0x1020 [xe] Update the VF CCS migration initialization sequence to align with the new migration framework changes, resolving the release-time crash. Fixes: f3009272ff2e ("drm/xe/vf: Create contexts for CCS read write") Signed-off-by: Satyanarayana K V P <satyanarayana.k.v.p@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Piotr Piórkowski <piotr.piorkowski@intel.com> Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250729120720.13990-1-satyanarayana.k.v.p@intel.com
2025-07-15drm/xe/pf: Prepare to stop SR-IOV support prior GT resetMichal Wajdeczko1-0/+3
As part of the resume or GT reset, the PF driver schedules work which is then used to complete restarting of the SR-IOV support, including resending to the GuC configurations of provisioned VFs. However, in case of short delay between those two actions, which could be seen by triggering a GT reset on the suspened device: $ echo 1 > /sys/kernel/debug/dri/0000:00:02.0/gt0/force_reset this PF worker might be still busy, which lead to errors due to just stopped or disabled GuC CTB communication: [ ] xe 0000:00:02.0: [drm:xe_gt_resume [xe]] GT0: resumed [ ] xe 0000:00:02.0: [drm] GT0: trying reset from force_reset_show [xe] [ ] xe 0000:00:02.0: [drm] GT0: reset queued [ ] xe 0000:00:02.0: [drm] GT0: reset started [ ] xe 0000:00:02.0: [drm:guc_ct_change_state [xe]] GT0: GuC CT communication channel stopped [ ] xe 0000:00:02.0: [drm:guc_ct_send_recv [xe]] GT0: H2G request 0x5503 canceled! [ ] xe 0000:00:02.0: [drm] GT0: PF: Failed to push VF1 12 config KLVs (-ECANCELED) [ ] xe 0000:00:02.0: [drm] GT0: PF: Failed to push VF1 configuration (-ECANCELED) [ ] xe 0000:00:02.0: [drm:guc_ct_change_state [xe]] GT0: GuC CT communication channel disabled [ ] xe 0000:00:02.0: [drm] GT0: PF: Failed to push VF2 12 config KLVs (-ENODEV) [ ] xe 0000:00:02.0: [drm] GT0: PF: Failed to push VF2 configuration (-ENODEV) [ ] xe 0000:00:02.0: [drm] GT0: PF: Failed to push 2 of 2 VFs configurations [ ] xe 0000:00:02.0: [drm:pf_worker_restart_func [xe]] GT0: PF: restart completed While this VFs reprovisioning will be successful during next spin of the worker, to avoid those errors, make sure to cancel restart worker if we are about to trigger next reset. Fixes: 411220808cee ("drm/xe/pf: Restart VFs provisioning after GT reset") Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com> Link: https://lore.kernel.org/r/20250711193316.1920-2-michal.wajdeczko@intel.com
2025-07-14drm/xe: Waste fewer instructions in emit_wa_job()Tvrtko Ursulin1-35/+41
I was debugging some unrelated issue and noticed the current code was very verbose. We can improve it easily by using the more common batch buffer building pattern. Before: bb->cs[bb->len++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO; c4d: 41 8b 56 10 mov 0x10(%r14),%edx c51: 49 8b 4e 08 mov 0x8(%r14),%rcx c55: 8d 72 01 lea 0x1(%rdx),%esi c58: 41 89 76 10 mov %esi,0x10(%r14) c5c: c7 04 91 01 00 08 15 movl $0x15080001,(%rcx,%rdx,4) bb->cs[bb->len++] = entry->reg.addr; c63: 8b 08 mov (%rax),%ecx c65: 41 8b 56 10 mov 0x10(%r14),%edx c69: 49 8b 76 08 mov 0x8(%r14),%rsi c6d: 81 e1 ff ff 3f 00 and $0x3fffff,%ecx c73: 8d 7a 01 lea 0x1(%rdx),%edi c76: 41 89 7e 10 mov %edi,0x10(%r14) c7a: 89 0c 96 mov %ecx,(%rsi,%rdx,4) ..etc.. After: *cs++ = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO; c52: 41 c7 04 24 01 00 08 movl $0x15080001,(%r12) c59: 15 *cs++ = entry->reg.addr; c5a: 8b 10 mov (%rax),%edx ..etc.. Resulting in the following binary change: add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-348 (-348) Function old new delta xe_gt_record_default_lrcs.cold 304 296 -8 xe_gt_record_default_lrcs 2200 1860 -340 Total: Before=13554, After=13206, chg -2.57% Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250710-lrc-refactors-v2-7-a5e2ca03f6bd@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-07-14drm/xe/gt: Drop third submission for default contextLucas De Marchi1-8/+0
There's no need to submit the nop job again on the first queue. Any state needed is already saved when the first LRC is switched out. The comment is a little misleading regarding indirect W/A: first of all there's still no indirect W/A enabled and secondly, even after they are, there's no need to submit this job again for having their state propagated: the indirect W/A will actually run on every LRC switch. Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250710-lrc-refactors-v2-6-a5e2ca03f6bd@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-07-14drm/xe/gt: Extract emit_job_sync()Lucas De Marchi1-32/+22
Both the nop and wa jobs are going through the same boiler plate calls to emit the job with a timeout and handling error for both bb and job. Extract emit_job_sync() so those functions create the bb, handling possible errors and delegate the part about really emitting the job and waiting for its completion. Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com> Link: https://lore.kernel.org/r/20250710-lrc-refactors-v2-3-a5e2ca03f6bd@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-07-14drm/xe: Count dwords before allocatingLucas De Marchi1-14/+24
The bb allocation in emit_wa_job() is wrong in 2 ways: first it's allocating enough space for the 3DSTATE or hardcoding 4k depending on the engine. In the first case it doesn't account for the WAs and in the former it may not be sufficient. Secondly it's using the size instead of number of dwords, causing the buffer to be 4x bigger than needed: xe_bb_new() receives number of dwords as parameter and its declaration was also not following its implementation. Lastly, reword the debug message since it's not only about the LRC WAs anymore as it also include the 3DSTATE for render. While it's unlikely this is causing any real issue, let's calculate the needed space and allocate just enough. Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com> Link: https://lore.kernel.org/r/20250710-lrc-refactors-v2-2-a5e2ca03f6bd@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-07-14drm/xe: Introduce xe_gt_is_main_type helperMichal Wajdeczko1-6/+6
Instead of checking for not being a media type GT provide a small helper to explicitly express our intentions. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com> Link: https://lore.kernel.org/r/20250713103625.1964-5-michal.wajdeczko@intel.com
2025-07-11drm/xe: Move page fault init after topology initMatthew Brost1-3/+3
We need the topology to determine GT page fault queue size, move page fault init after topology init. Cc: stable@vger.kernel.org Fixes: 3338e4f90c14 ("drm/xe: Use topology to determine page fault queue size") Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Jonathan Cavitt <jonathan.cavitt@intel.com> Reviewed-by: Stuart Summers <stuart.summers@intel.com> Link: https://lore.kernel.org/r/20250710191208.1040215-1-matthew.brost@intel.com
2025-06-26drm/xe: Rename xe_uc_init_hw to xe_uc_load_hwMaarten Lankhorst1-3/+3
It feels to me like load is closer to the intention than init_hw. It makes the init calls slightly less confusing to me. :) Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20250619104858.418440-24-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
2025-06-26drm/xe: Remove xe_uc_init_hwconfig()Maarten Lankhorst1-4/+0
This function is called immediately after xe_uc_init(), so just put it there instead. Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20250619104858.418440-22-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
2025-06-26drm/xe: Split init of xe_gt_init_hwconfig to xe_gt_init and *_earlyMaarten Lankhorst1-34/+26
Now that we added the separate step of initialising GUC in xe_gt_init_early, it should be ok to initialise the minimum during early init, and the rest after allocations are allowed. Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20250619104858.418440-20-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
2025-06-26drm/xe: Rename gt_init sub-functionsMaarten Lankhorst1-4/+4
s/gt_fw_domain_init/gt_init_with_gt_forcewake()/ s/all_fw_domain_init/gt_init_with_all_forcewake()/ Clarify that the functions are the part of gt_init() that are called with the respective power domains held. all_domain() of course only works after discovering and initialisation of force_wake on all engines, that's why the split is needed in the first place. Suggested-by: Lucas De Marchi <lucas.demarchi@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20250619104858.418440-19-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
2025-06-26drm/xe: Only dump PAT when xe_hw_engines_init_early failsMaarten Lankhorst1-2/+3
After discussion with Lucas De Marchi, it turns out that is the specific caller requiring a dump. This allows us to cleanup gt_init in a bit. Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20250619104858.418440-18-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
2025-06-26drm/xe: Simplify GuC early initializationMaarten Lankhorst1-1/+1
Add a 2-stage GuC init. An early one for everything that is needed for VF, and a full one that loads GuC and is allowed to do allocations. Link: https://lore.kernel.org/r/20250619104858.418440-16-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
2025-06-26drm/xe/sriov: Move VF bootstrap and query_config to vf_guc_initMaarten Lankhorst1-0/+7
We want to split up GUC init to an alloc and noalloc part to keep the init path the same for VF and !VF as much as possible. Everything in vf_guc_init should be done as early as possible, otherwise VRAM probing becomes impossible. Also move xe_gt_mmio_init to the end of xe_gt_init_early(), cleaning up the init in xe_device slightly. Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20250619104858.418440-15-dev@lankhorst.se Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
generated by cgit v1.2.3 (git 2.39.0) at 2026-04-22 05:08:34 +0300