kernel/linux.git/drivers/pci/controller/vmd.c, branch v6.6.133

PCI: vmd: Disable MSI remapping bypass under Xen

2025-06-04T12:41:56+00:00

[ Upstream commit 6c4d5aadf5df31ea0ac025980670eee9beaf466b ] MSI remapping bypass (directly configuring MSI entries for devices on the VMD bus) won't work under Xen, as Xen is not aware of devices in such bus, and hence cannot configure the entries using the pIRQ interface in the PV case, and in the PVH case traps won't be setup for MSI entries for such devices. Until Xen is aware of devices in the VMD bus prevent the VMD_FEAT_CAN_BYPASS_MSI_REMAP capability from being used when running as any kind of Xen guest. The MSI remapping bypass is an optional feature of VMD bridges, and hence when running under Xen it will be masked and devices will be forced to redirect its interrupts from the VMD bridge. That mode of operation must always be supported by VMD bridges and works when Xen is not aware of devices behind the VMD bridge. Signed-off-by: Roger Pau Monné Acked-by: Bjorn Helgaas Message-ID: <20250219092059.90850-3-roger.pau@citrix.com> Signed-off-by: Juergen Gross Signed-off-by: Sasha Levin

PCI: vmd: Make vmd_dev::cfg_lock a raw_spinlock_t type

2025-04-25T08:45:19+00:00

[ Upstream commit 18056a48669a040bef491e63b25896561ee14d90 ] The access to the PCI config space via pci_ops::read and pci_ops::write is a low-level hardware access. The functions can be accessed with disabled interrupts even on PREEMPT_RT. The pci_lock is a raw_spinlock_t for this purpose. A spinlock_t becomes a sleeping lock on PREEMPT_RT, so it cannot be acquired with disabled interrupts. The vmd_dev::cfg_lock is accessed in the same context as the pci_lock. Make vmd_dev::cfg_lock a raw_spinlock_t type so it can be used with interrupts disabled. This was reported as: BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 Call Trace: rt_spin_lock+0x4e/0x130 vmd_pci_read+0x8d/0x100 [vmd] pci_user_read_config_byte+0x6f/0xe0 pci_read_config+0xfe/0x290 sysfs_kf_bin_read+0x68/0x90 Signed-off-by: Ryo Takakura Tested-by: Luis Claudio R. Goncalves Acked-by: Luis Claudio R. Goncalves [bigeasy: reword commit message] Signed-off-by: Sebastian Andrzej Siewior Tested-off-by: Luis Claudio R. Goncalves Link: https://lore.kernel.org/r/20250218080830.ufw3IgyX@linutronix.de [kwilczynski: commit log] Signed-off-by: Krzysztof Wilczyński [bhelgaas: add back report info from https://lore.kernel.org/lkml/20241218115951.83062-1-ryotkkr98@gmail.com/] Signed-off-by: Bjorn Helgaas Signed-off-by: Sasha Levin

PCI: vmd: Create domain symlink before pci_bus_add_devices()

2024-12-27T12:58:41+00:00

[ Upstream commit f24c9bfcd423e2b2bb0d198456412f614ec2030a ] The vmd driver creates a "domain" symlink in sysfs for each VMD bridge. Previously this symlink was created after pci_bus_add_devices() added devices below the VMD bridge and emitted udev events to announce them to userspace. This led to a race between userspace consumers of the udev events and the kernel creation of the symlink. One such consumer is mdadm, which assembles block devices into a RAID array, and for devices below a VMD bridge, mdadm depends on the "domain" symlink. If mdadm loses the race, it may be unable to assemble a RAID array, which may cause a boot failure or other issues, with complaints like this: (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: Unable to get real path for '/sys/bus/pci/drivers/vmd/0000:c7:00.5/domain/device'' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: /dev/nvme1n1 is not attached to Intel(R) RAID controller.' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: No OROM/EFI properties for /dev/nvme1n1' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: no RAID superblock on /dev/nvme1n1.' (udev-worker)[2149]: nvme1n1: Process '/sbin/mdadm -I /dev/nvme1n1' failed with exit code 1. This symptom prevents the OS from booting successfully. After a NVMe disk is probed/added by the nvme driver, udevd invokes mdadm to detect if there is a mdraid associated with this NVMe disk, and mdadm determines if a NVMe device is connected to a particular VMD domain by checking the "domain" symlink. For example: Thread A Thread B Thread mdadm vmd_enable_domain pci_bus_add_devices __driver_probe_device ... work_on_cpu schedule_work_on : wakeup Thread B nvme_probe : wakeup scan_work to scan nvme disk and add nvme disk then wakeup udevd : udevd executes mdadm command flush_work main : wait for nvme_probe done ... __driver_probe_device find_driver_devices : probe next nvme device : 1) Detect domain symlink ... 2) Find domain symlink ... from vmd sysfs ... 3) Domain symlink not ... created yet; failed sysfs_create_link : create domain symlink Create the VMD "domain" symlink before invoking pci_bus_add_devices() to avoid this race. Suggested-by: Adrian Huang Link: https://lore.kernel.org/linux-pci/20240605124844.24293-1-sjiwei@163.com Signed-off-by: Jiwei Sun Signed-off-by: Krzysztof Wilczyński [bhelgaas: commit log] Signed-off-by: Bjorn Helgaas Reviewed-by: Nirmal Patel Signed-off-by: Sasha Levin

PCI: vmd: Set devices to D0 before enabling PM L1 Substates

2024-12-14T19:00:14+00:00

[ Upstream commit d66041063192497a4a97d21dbf86b79a03a7f4fb ] The remapped PCIe Root Port and the child device have PM L1 Substates capability, but they are disabled originally. Here is a failed example on ASUS B1400CEAE: Capabilities: [900 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ PortCommonModeRestoreTime=32us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=101376ns L1SubCtl2: T_PwrOn=50us Enable PCI-PM L1 PM Substates for devices below VMD while they are in D0 (see PCIe r6.0, sec 5.5.4). Link: https://lore.kernel.org/r/20241001083438.10070-4-jhp@endlessos.org Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 Signed-off-by: Jian-Hong Pan Signed-off-by: Krzysztof Wilczyński Signed-off-by: Bjorn Helgaas Reviewed-by: Kuppuswamy Sathyanarayanan Signed-off-by: Sasha Levin

PCI: vmd: Add DID 8086:B06F and 8086:B60B for Intel client SKUs

2024-12-14T19:00:14+00:00

[ Upstream commit b727484cace4be22be9321cc0bc9487648ba447b ] Add support for this VMD device which supports the bus restriction mode. The feature that turns off vector 0 for MSI-X remapping is also enabled. Link: https://lore.kernel.org/r/20241011175657.249948-1-nirmal.patel@linux.intel.com Signed-off-by: Nirmal Patel Signed-off-by: Krzysztof Wilczyński Reviewed-by: Manivannan Sadhasivam Signed-off-by: Sasha Levin

PCI: vmd: Fix potential deadlock when enabling ASPM

2023-12-20T16:01:54+00:00

commit 49de0dc87965079a8e2803ee4b39f9d946259423 upstream. The vmd_pm_enable_quirk() helper is called from pci_walk_bus() during probe to enable ASPM for controllers with VMD_FEAT_BIOS_PM_QUIRK set. Since pci_walk_bus() already holds a pci_bus_sem read lock, use pci_enable_link_state_locked() to enable link states in order to avoid a potential deadlock (e.g. in case someone takes a write lock before reacquiring the read lock). Fixes: f492edb40b54 ("PCI: vmd: Add quirk to configure PCIe ASPM and LTR") Link: https://lore.kernel.org/r/20231128081512.19387-3-johan+linaro@kernel.org Signed-off-by: Johan Hovold [bhelgaas: add "potential" in subject since the deadlock has only been reported by lockdep, include helper name in commit log] Signed-off-by: Bjorn Helgaas Reviewed-by: Manivannan Sadhasivam Cc: # 6.3 Cc: Michael Bottini Cc: David E. Box Signed-off-by: Greg Kroah-Hartman

PCI: vmd: Correct PCI Header Type Register's multi-function check

2023-11-20T10:59:20+00:00

[ Upstream commit 5827e17d0555b566c32044b0632b46f9f95054fa ] vmd_domain_reset() attempts to find whether the device may contain multiple functions by checking 0x80 (Multi-Function Device), however, the hdr_type variable has already been masked with PCI_HEADER_TYPE_MASK so the check can never true. To fix the issue, don't mask the read with PCI_HEADER_TYPE_MASK. Fixes: 6aab5622296b ("PCI: vmd: Clean up domain before enumeration") Link: https://lore.kernel.org/r/20231003125300.5541-2-ilpo.jarvinen@linux.intel.com Signed-off-by: Ilpo Järvinen Signed-off-by: Bjorn Helgaas Cc: Nirmal Patel Signed-off-by: Sasha Levin

PCI: vmd: Disable bridge window for domain reset

2023-08-22T09:09:42+00:00

During domain reset process vmd_domain_reset() clears PCI configuration space of VMD root ports. But certain platform has observed following errors and failed to boot. ... DMAR: VT-d detected Invalidation Queue Error: Reason f DMAR: VT-d detected Invalidation Time-out Error: SID ffff DMAR: VT-d detected Invalidation Completion Error: SID ffff DMAR: QI HEAD: UNKNOWN qw0 = 0x0, qw1 = 0x0 DMAR: QI PRIOR: UNKNOWN qw0 = 0x0, qw1 = 0x0 DMAR: Invalidation Time-out Error (ITE) cleared The root cause is that memset_io() clears prefetchable memory base/limit registers and prefetchable base/limit 32 bits registers sequentially. This seems to be enabling prefetchable memory if the device disabled prefetchable memory originally. Here is an example (before memset_io()): PCI configuration space for 10000:00:00.0: 86 80 30 20 06 00 10 00 04 00 04 06 00 00 01 00 00 00 00 00 00 00 00 00 00 01 01 00 00 00 00 20 00 00 00 00 01 00 01 00 ff ff ff ff 75 05 00 00 ... So, prefetchable memory is ffffffff00000000-575000fffff, which is disabled. When memset_io() clears prefetchable base 32 bits register, the prefetchable memory becomes 0000000000000000-575000fffff, which is enabled and incorrect. Here is the quote from section 7.5.1.3.9 of PCI Express Base 6.0 spec: The Prefetchable Memory Limit register must be programmed to a smaller value than the Prefetchable Memory Base register if there is no prefetchable memory on the secondary side of the bridge. This is believed to be the reason for the failure and in addition the sequence of operation in vmd_domain_reset() is not following the PCIe specs. Disable the bridge window by executing a sequence of operations borrowed from pci_disable_bridge_window() and pci_setup_bridge_io(), that comply with the PCI specifications. Link: https://lore.kernel.org/r/20230810215029.1177379-1-nirmal.patel@linux.intel.com Signed-off-by: Nirmal Patel Signed-off-by: Lorenzo Pieralisi

PCI: vmd: Fix uninitialized variable usage in vmd_enable_domain()

2023-06-24T16:09:16+00:00

The ret variable in the vmd_enable_domain() function was used uninitialized when printing a warning message upon failure of the pci_reset_bus() function. Thus, fix the issue by assigning ret with the value returned from pci_reset_bus() before referencing it in the warning message. This was detected by Smatch: drivers/pci/controller/vmd.c:931 vmd_enable_domain() error: uninitialized symbol 'ret'. [kwilczynski: drop the second patch from the series, add missing reported by tag, commit log] Fixes: 0a584655ef89 ("PCI: vmd: Fix secondary bus reset for Intel bridges") Link: https://lore.kernel.org/all/202305270219.B96IiIfv-lkp@intel.com Link: https://lore.kernel.org/linux-pci/20230420094332.1507900-2-korantwork@gmail.com Reported-by: kernel test robot Reported-by: Dan Carpenter Signed-off-by: Xinghui Li Signed-off-by: Krzysztof Wilczyński Reviewed-by: Nirmal Patel

PCI: vmd: Reset VMD config register between soft reboots

2023-05-08T07:17:13+00:00

VMD driver can disable or enable MSI remapping by changing VMCONFIG_MSI_REMAP register. This register needs to be set to the default value during soft reboots. Drives failed to enumerate when Windows boots after performing a soft reboot from Linux. Windows doesn't support MSI remapping disable feature and stale register value hinders Windows VMD driver initialization process. Adding vmd_shutdown function to make sure to set the VMCONFIG register to the default value. Link: https://lore.kernel.org/r/20230224202811.644370-1-nirmal.patel@linux.intel.com Fixes: ee81ee84f873 ("PCI: vmd: Disable MSI-X remapping when possible") Signed-off-by: Nirmal Patel Signed-off-by: Lorenzo Pieralisi Reviewed-by: Jon Derrick