Repository navigation
[26.04_linux-nvidia-7.3-next] CXL Type-2 device support - cxl reset and cxl type2 passthrough - #714
JiandiAnNVIDIA wants to merge 57 commits into
Conversation
CXL 2.0 added 16-way memory interleaving, making the comment that the specification supports only 8-way interleaving stale. Explain that 16 is the largest interleave a decoder can express, without changing the limit. Suggested-by: Jonathan Cameron <[email protected]> Link: https://lore.kernel.org/linux-cxl/20260912003057.3db9335c@jic23-hlaptop/ Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Jonathan Cameron <[email protected]> Reviewed-by: Dave Jiang <[email protected]> Reviewed-by: Richard Cheng <[email protected]> Reviewed-by: Li Ming <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
read_cdat_data() resolves a CXL port's upstream device to its backing PCI device. HDM cache updates need the same mapping. Factor the lookup into cxl_port_get_uport_pci_dev() and return a referenced PCI device to make caller ownership explicit. Use automatic reference cleanup in read_cdat_data(), as suggested by Li Ming, so early returns release the PCI device reference. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Jonathan Cameron <[email protected]> Reviewed-by: Dave Jiang <[email protected]> Reviewed-by: Richard Cheng <[email protected]> Reviewed-by: Li Ming <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Move the common decoder declarations to include/cxl/cxl.h for use by shared decoder settings. Preserve their layout and documentation. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Jonathan Cameron <[email protected]> Reviewed-by: Dave Jiang <[email protected]> Reviewed-by: Richard Cheng <[email protected]> Reviewed-by: Li Ming <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…ture Define cxl_decoder_config independently and embed it in cxl_decoder. Update decoder field accesses without changing behavior, so reset snapshots can reuse the configuration type without struct_group_tagged(). Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Richard Cheng <[email protected]> Reviewed-by: Dave Jiang <[email protected]> Reviewed-by: Li Ming <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) [jan: reapplied the cxl_decoder config sub-struct conversion onto the multi-level interleave selector/distance logic from 5dd563b] Signed-off-by: Jiandi An <[email protected]>
Keep common decoder configuration in cxl_decoder_config and represent endpoint DPA skip in cxl_endpoint_decoder_settings. Share range programming, but keep endpoint skip and switch target-list writes in their respective commit paths. Separate commit initiation from completion waiting so reset restoration can reuse register programming without changing normal DPA-lock policy. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Dave Jiang <[email protected]> Reviewed-by: Li Ming <[email protected]> Assisted-by: LLM (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
The PCI core will restore HDM decoders after a reset, possibly while no CXL driver is loaded, so it cannot call into cxl_core. Move the HDM register programming helpers from hdm.c to hdm_regs.c and build that file into the kernel whenever CXL_BUS is enabled, including when CXL_BUS is modular. Export only the symbols that cxl_core uses. Keep the DPA lock assertion in the normal decoder programming caller so the register helpers do not depend on cxl_rwsem. Leave cxl_rwsem in cxl_core until built-in cache management needs it. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Dave Jiang <[email protected]> Reviewed-by: Li Ming <[email protected]> Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Factor common HDM register parsing into cxl_hdm_unpack_decoder(). Validate range and interleave parameters in locals before publishing the decoder configuration, including enable and lock flags. Reject nonzero ranges whose end overflows the address space while preserving the zero-size range representation. Read decoder base and size registers low then high through a shared helper used by enumeration and the reset snapshot. Read switch target lists and endpoint DPA skip in their respective branches of init_hdm_decoder(). Preserve the decoder ID allocated by cxl_decoder_init() when publishing the parsed configuration. Signed-off-by: Srirangan Madhavan <[email protected]> Assisted-by: LLM Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Keep the region and DPA locks available to built-in CXL state management when cxl_core is modular. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Early PCI discovery creates the HDM cache, while later CXL enumeration and decoder operations provide updated programming state. Refresh the PCI snapshot when decoders are enumerated, committed, or reset so reset recovery need not walk the CXL topology. Keep cache access and locking in the built-in HDM state helpers. Ignore updates when no cache exists and reject decoder-count mismatches. Use CXL_RESET for the PCI HDM cache. Shared HDM register helpers do not require it. Skip cache checks and updates for DVSEC-emulated decoders, which do not use HDM registers. Keep cached global control when MMIO reads return all ones. Non-PCI mock ports have no upstream PCI device, so the cache helpers ignore them. This avoids test wrappers and a circular dependency between cxl_core and cxl_mock. Signed-off-by: Srirangan Madhavan <[email protected]> Assisted-by: LLM Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Build register discovery with the built-in HDM helpers so PCI enumeration can use it even when cxl_core is modular. Export the interfaces used by cxl_core and preserve the BAR and offset in the register map. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Cache HDM state in pci_dev->hdm after BAR resource setup and before driver binding, so reset can restore it without a bound CXL driver. Limit caching to devices that advertise CXL.cache, CXL.mem, CXL Reset, and Reset Memory Clear. Save their HDM register location, decoder settings, and Device DVSEC state. Validate capability reads and decoder counts, and restore PCI_COMMAND after temporary MMIO access, reporting restoration failures. Publish the cache under the PCI device lock and protect later updates with the DPA lock. Signed-off-by: Srirangan Madhavan <[email protected]> Assisted-by: LLM (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Implement CXL Device Reset for eligible Type 2 devices: disable CXL.cache, perform supported writeback, drain transactions, prepare the IOMMU, initiate reset with Memory Clear enabled, and wait for completion. Observe the 100 ms post-reset quiet period before polling Status2, and treat an all-ones read as not ready. Wait for the timeout the device advertises, up to 100 seconds, and access only Status2 until reset completion, error or expiry, as CXL 4.0 section 9.7.3 requires. Control2 cleanup runs only afterwards. Treat Control2 command bits as transient state, restore cache policy on exit, preserve the primary error, and abort when IOMMU preparation fails. Reuse Device DVSEC discovery from cache initialization and keep the reset entry point declaration private to PCI core. Signed-off-by: Srirangan Madhavan <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Boro watcher review skippedThe PR watcher skips automatic boro reviews for changes with more than 50 commits. This change currently has 56 commits. To run the review anyway, ask Head: This comment is maintained by BaseOS Reviewer. It is updated when the watcher sees a newer head. |
PR Validation ReportPatchscan
|
0b9da42 to
41c6eca
Compare
|
Consolidated findings from Codex + Claude:
No earlier upstream report of items 3 or 5 was found in the reviewed material. Reviewed draft head: Validation:
No physical CXL hardware reproduction was performed. |
| }; | ||
| MODULE_DEVICE_TABLE(arm_smccc, smccc_cache_id_table); | ||
|
|
||
| static struct arm_smccc_driver smccc_cache_driver = { |
There was a problem hiding this comment.
P3 — SMCCC cache instances survive driver unbind/rebind.
Successful probe allocates and registers cache, but this driver has neither a remove callback nor stored driver data. The SMCCC bus exposes normal unbind/rebind. Unbind therefore leaves the allocated instance on the cache-maintenance list; each rebind adds another instance and another firmware operation per invalidation.
cache_coherency_ops_instance_register() only adds the instance to the list; unregister and put are separate operations. A focused cleanup pattern is:
/* After successful registration in smccc_cache_probe(): */
dev_set_drvdata(&sdev->dev, cache);static void smccc_cache_remove(struct arm_smccc_device *sdev)
{
struct smccc_cache *cache = dev_get_drvdata(&sdev->dev);
cache_coherency_ops_instance_unregister(&cache->cci);
mutex_destroy(&cache->lock);
cache_coherency_ops_instance_put(&cache->cci);
}Add .remove = smccc_cache_remove to this driver initializer. Preserve the existing registration-failure cleanup. Unregister takes the framework write lock, which waits for the read-locked cache callbacks before destroying/freeing the instance.
This is a static ownership finding, not a reproduced hardware failure. The pattern has not been compiled or executed; validation should cover unbind/rebind, concurrent invalidation, and registration-failure cleanup. It fixes this leak, not the separate framework behavior when the last provider disappears.
|
41c6eca to
926ec29
Compare
|
Consolidated findings from Codex + Claude:
The provider-state use-after-free and readiness findings are fixed. Matt's reset-cleanup and annotation findings, and the SMCCC-unbind cleanup, are also addressed. Reviewed head Validation:
|
926ec29 to
fd7aa2c
Compare
Defer this for now. On VR this is not exposed. Srirangan is making changes to his v18 for the fix. The CXL reset/hotplug lock cycle Dave Jiang reported remains unfixed in this backport, by choice. pci_cxl_reset_prepare() holds the global region quiesce across the whole reset, giving cxl_rwsem.region → slot reset lock → pci_rescan_remove_lock → cxl_rwsem.region. The lock is not incidental: the HDM address range is already protected by the request_mem_region() the quiesce takes, so what the rwsem adds is blocking teardown of a committed region over the endpoint. Narrowing it needs a per-endpoint reset marker honoured by the region teardown paths — and Srirangan has now said he plans to solve it in v18 with reference counting rather than lock re-ordering, which touches the same prepare/restore interface this backport carries. A local fix would therefore have to be unpicked at re-port, and the contained alternatives are all worse: taking pci_lock_rescan_remove() first only reduces the cycle to two parties, a trylock doesn't help because the hazard is holding the lock across the SBR, and dropping it across the SBR lets a region be torn down underneath, after which the restore re-commits a stale decoder — corruption instead of a hang. Exposure here is bounded and measured: all four GPU root ports report Root Port (Slot-), so pciehp cannot bind in their hierarchy and the hotplug-driven form cannot occur; what remains needs a deliberate removal or unbind of a CXL device during an in-flight CXL bus reset. |
|
Consolidated findings from Codex + Claude: Fixed. Both previous loader findings are resolved: the async-probe deadlock (Sashiko, via Alex Williamson) and the deferred bind that could stay pending after provider loading. Deferred. Deferring Dave Jiang's hotplug lock cycle is fine for this backport, given the reported non-hotplug VR topology and the planned v18 fix. Remaining:
Reviewed head |
|
Four additional findings from my re-review, but IMO these can be pursued as follow-ups.
|
…o handlers Currently, if there are no registered handlers, cpu_cache_invalidate_memregion() returns 0. Callers are expected to check cpu_cache_has_invalidate_memregion() and return -ENXIO themselves, instead of calling this function. If a caller forgets to do this, they may erroneously think the operation succeeded. This is in contrast to x86, where cpu_cache_invalidate_memregion() itself checks if the operation is supported, and returns -ENXIO if it is not. Reviewed-by: Jonathan Cameron <[email protected]> Signed-off-by: Bence Csokas <[email protected]> Signed-off-by: Conor Dooley <[email protected]> (cherry picked from commit b2f3a17) Signed-off-by: Jiandi An <[email protected]>
Refuse reset unless enabled system-physical HDM ranges can be reserved exclusively and CPU-cache invalidation is available. Invalidate before reset and again before ending IOMMU exclusion, holding range reservations until the second invalidation completes. A later patch places state restoration before the second invalidation. Userspace is expected to offline CXL.mem users before reset. The reservation enforces this for claimed ranges: the reset fails while a conflicting resource is claimed, and no one can claim the range during the reset. It cannot see accesses by code that uses the range without claiming it, which the offline expectation also covers. Reject normalized-addressing decoders because their cached ranges are not system physical addresses. Record Normalized Addressing in the cache when region discovery sets it, since the cache is filled earlier. A reset that takes its snapshot between endpoint probe and that update can still miss it. Ignore zero-size decoders because they map no address range. Refuse reset when the device decodes memory through its DVSEC ranges instead of HDM decoders, since the cache holds no range to reserve for that memory. Hold the global cxl_rwsem.region write lock across snapshot, reset and restoration to keep the decoder state and reserved ranges stable. This also blocks region and decoder sysfs, region probe, poison operations, sanitize and region-aware event tracing across CXL devices. The Memory Clear wait lasts up to the timeout the device advertises, at most 100 seconds; cache flushes and other reset work extend the hold. Allow the existing CXL_REGION_INVALIDATION_TEST option to bypass missing CPU cache invalidation for test environments without a provider. Signed-off-by: Srirangan Madhavan <[email protected]> Assisted-by: LLM (backported from https://lore.kernel.org/linux-cxl/[email protected]/) [jan: propagate the invalidation result instead of discarding it, so an absent cache-maintenance provider fails the quiesce rather than reporting a writeback that never happened; upstream b2f3a17 makes that case return -ENXIO (Matt Ochs)] Signed-off-by: Jiandi An <[email protected]>
The per-device PCI reset interface must not reset sibling functions. Reject cxl_reset and cxl_bus for VFs, multifunction devices, and PFs with enabled VFs. Suggested-by: Alex Williamson <[email protected]> Link: https://lore.kernel.org/linux-cxl/[email protected]/ Link: https://lore.kernel.org/linux-cxl/[email protected]/ Signed-off-by: Srirangan Madhavan <[email protected]> Acked-by: Bjorn Helgaas <[email protected]> Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Restore the PCI configuration needed for HDM access, followed by cached HDM decoder and CXL Device DVSEC state. Set Lock On Commit before recommitting a cleared locked decoder; leave surviving locked state unchanged. Stop at the first restoration failure and preserve a disabled PCI command snapshot. Keep IOMMU exclusion active through restoration and post-reset cache invalidation. Program the decoders from built-in CXL code that takes only a mapped register block, and keep PCI state handling in drivers/pci/cxl.c. The restore helper structure and diagnostics follow Dave Jiang's reference implementation on his cxl-type2-reset branch. Signed-off-by: Srirangan Madhavan <[email protected]> Assisted-by: LLM (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Register cxl_reset with the PCI reset-method interface and place it before FLR, which does not reset CXL.cache or CXL.mem protocol state. Return -ENOTTY when device capabilities, function scope, cached HDM state, or required cache synchronization do not permit a safe reset. Signed-off-by: Srirangan Madhavan <[email protected]> Reviewed-by: Richard Cheng <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…BR masked A CXL port ignores the Secondary Bus Reset bit in Bridge Control unless Unmask SBR is set in its CXL Port DVSEC (CXL r4.0 sec 8.1.5.2). pci_reset_bus_function() already returns -ENOTTY in that case. pci_reset_bus(), pci_probe_reset_bus() and the slot variants do not, so they report success for a reset that reached no device below the port. vfio-pci uses these for VFIO_DEVICE_GET_PCI_HOT_RESET_INFO and VFIO_DEVICE_PCI_HOT_RESET. A user is then told a hot reset happened while a CXL device below a masked port kept its state. Return -ENOTTY from pci_probe_reset_slot() and pci_probe_reset_bus() when the bridge above the bus is a CXL port with SBR masked. pci_reset_bus() and pci_try_reset_bridge() probe before they reset, so they refuse too. Users see this in three places: - the vfio hot reset ioctls fail (vfio reports -ENODEV when neither probe succeeds), - the reset_subordinate sysfs attribute returns -ENOTTY, - vfio-pci groups a device in a hotplug slot below such a port by bus instead of by slot. Every other pci_reset_bus() caller, including the vfio bus reset at close, is refused the same way. With one device per CXL port both groupings have the same members. Leave pci_bus_error_reset() as it is. AER and host bridge error recovery call it without saving or restoring state and only map its result to recovered or disconnect. Failing it below a masked port would turn recovery of every CXL device there, memory expanders included, into a disconnect; that is a CXL error handling decision, not part of this fix. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
vfio-pci-core must stay free of any CXL header dependency, so CXL Type-2 handling lives in a separate vfio-cxl module that plugs in a set of callbacks. Add the registration interface: - vfio-cxl registers a single struct vfio_cxl_ops at module_init - vfio-pci-core stores it under an rwsem. A second registration is refused with -EBUSY. The owner field will be used to pin vfio-cxl for the lifetime of each bound CXL device. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
… on demand
A CXL device needs the vfio-cxl callbacks, but pulling vfio-cxl and the
CXL core in unconditionally would bloat every vfio-pci setup. Detect a
CXL Type-2 device at bind and request_module("vfio-cxl") only then, then
hand the device to the registered ops.
Each bound CXL device pins the provider through
vfio_pci_get_cxl_ops() (try_module_get) and drops it with
vfio_pci_put_cxl_ops() at release, so vfio-cxl can unload once no CXL
device is bound.
If the provider is absent the device is bound without the CXL
extensions. A built-in provider whose initcall has not run yet is waited
for with -EPROBE_DEFER; the deferred-probe machinery will retry the bind
once the provider registers.
The provider .init() can return -EPROBE_DEFER when the device is not
ready yet, for example when its CXL port has not enumerated. Propagate
it so the bind is retried later. Any other init failure is non-fatal: a
CXL Type-2 device is also a valid vfio-pci device, so log the error, drop
the provider reference, leave cxl_ops NULL and continue without CXL
support.
Assisted-by: LLM
Signed-off-by: Manish Honap <[email protected]>
(backported from https://lore.kernel.org/linux-cxl/[email protected]/)
[jan: never load the provider synchronously from probe, and carry a softdep on vfio-cxl so userspace loads it first; pci_call_probe() can run probe on a NUMA locality worker it waits for under an async probe, which current_is_async() cannot detect, and modprobe would then block in async_synchronize_full(). Fall back to binding without CXL instead of deferring, because registering the provider binds no device and so retriggers no deferred probe (Sashiko via Alex Williamson, Jamie Nguyen)]
[jan: release the provider from vfio_pci_core_unregister_device() and make the release idempotent; the vfio_device release op can run after unbind has already freed the devres-owned provider state (Sashiko, via Alex Williamson)]
[jan: also release the provider from the vfio_pci_core_register_device() error path; a registration failure after the cdev is published can leave an open in flight that delays the final release past PCI cleanup (Matt Ochs)]
Signed-off-by: Jiandi An <[email protected]>
A provider such as vfio-cxl needs to keep sub-ranges of a BAR off the direct guest path and serve them through its own trap. vfio-pci-core already has one such window, the MSI-X table, open-coded in vfio_pci_bar_rw(). Add vfio_pci_core_add_excluded_range(). Each BAR has its own list, sorted by start, and an insert that overlaps a range on the same BAR or ends past the BAR fails with -EINVAL. Ranges are added only while the device is being opened and are freed in vfio_pci_core_disable(), so readers on an open device take no lock. The MSI-X table is virtualized in vfio_pci_bar_rw() by an open-coded x_start/x_end window that fills reads with -1 and drops writes. Register the MSI-X table as a read and write excluded range when the MSI-X capability is parsed in vfio_pci_core_enable(), and walk the BAR's ranges in order in vfio_pci_bar_rw(), so an access that spans several ranges is handled in one pass. Only the part of the table inside its BAR is registered, so a device whose MSI-X capability points outside the BAR still opens, as before. The ROM BAR keeps its trailing -1 window, outside the list. No behavioral change for an MSI-X table in a standard BAR: an access to it still reads -1 and drops writes. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
A range excluded from mmap must not be reachable through an mmap of its BAR. Refuse an mmap that overlaps such a range, and advertise the BAR with a sparse mmap capability that lists the areas between the excluded ranges, so a VMM maps those and traps the rest. mmap is page granular, so each range is rounded out to whole pages, and ranges that meet after rounding form one hole. The sorted list gives the areas in two walks: the first counts them and the second fills them. A BAR without mmap-excluded ranges gets no new capability. The MSI-X table is excluded from read and write only, so the MSI-X BAR is unchanged. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…xcluded ranges vfio_pci_bar_rw() keeps the excluded ranges of a BAR off the device, but two other paths reach BAR MMIO without it. An ioeventfd writes the BAR directly when it fires, through vfio_pci_core_iowriteN(). Refuse an ioeventfd whose range overlaps a write-excluded range, as one over the MSI-X table is already refused. A dma-buf export of a BAR range lets an importer map it for DMA, which reaches a range that the BAR keeps off mmap. Refuse an export whose ranges overlap an mmap-excluded range, so the dma-buf follows the same rule as mmap. Suggested-by: Alex Williamson <[email protected]> Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
A CXL Type-2 device is reprogrammable through its CXL DVSEC: a guest could set Config Lock, toggle CXL.cache and CXL.mem enable, or rewrite the HDM range registers that govern host memory decode. Virtualize the DVSEC so the guest sees a shadow it cannot use to reprogram the hardware. Build a per-device cxl_perm permission map, modeled on msi_perm. When the device is bound through the CXL provider (e.g. vfio-cxl), the whole CXL DVSEC is served from the vconfig shadow; only Control and Control2 are guest programmable, while Capability, Status, Lock and the Range registers keep their firmware snapshot. A vendor DVSEC on the same device is unaffected: the map is selected by the CXL DVSEC's range, since every DVSEC has the same ID in the config map and two DVSECs can be adjacent. A Type-2 function bound without the provider (the provider is absent, its init failed, or the extensions are disabled) gets a guard map instead. The DVSEC reads from hardware, Control and Control2 stay writable and the RW1C status bits stay clearable, but writes to Lock and the Range registers are dropped, since they outlive the user. Initiate CXL Reset uses the same doorbell; a cache write-back goes to the device. Control2 carries the CXL reset and cache write-back-invalidate initiate bits as self-clearing doorbells. With the provider, vfio never forwards them to hardware, so a custom writefn synthesizes their completion in the shadow: the initiate bit self-clears and the matching Status2 bit (Cache Invalid or Reset Done) is set, so a guest that writes back its cache (CXL r4.0 sec 9.6) and then runs CXL Reset (sec 9.7.2) progresses instead of timing out. Cache Invalid is reported only while Disable Caching is set, as the spec requires. The host performs the real cache write-back and reset at the vfio reset points. Until a later patch in this series runs the real reset from the doorbell, Initiate CXL Reset completes in the shadow only. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…ice use vfio-pci-core frees device regions and excluded ranges on close and on a failed open, so a CXL provider must set them up at each open. Add open and close to the CXL ops and call them from the common enable and close paths. A failed open unwinds the enable in reverse order: release the device regions, unmap the BARs, drop the excluded ranges, then free the virtual config. Factor the device region release and the BAR unmap out of vfio_pci_core_disable() into helpers so the enable failure path and the disable path share them. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
A CXL Type-2 device must quiesce its coherent link before any reset and restore its HDM decoder state afterward, or a reset can leave the device mastering the bus over decoders that are no longer valid. This has to happen around every reset the PCI core drives for the device, not only the ones VFIO issues directly, and around AER and DPC recovery, which can reset the link without the reset callbacks. Add: - reset_prepare - reset_done - an optional error_resume to the CXL ops Export: - vfio_pci_core_reset_prepare() - vfio_pci_core_reset_done() - vfio_pci_core_error_detected() - vfio_pci_core_error_resume() which take the vfio device. vfio-pci calls them from its own pci_error_handlers, which resolve the driver data it owns. A variant driver that binds CXL devices calls them from its own handlers the same way; a driver without reset_prepare, reset_done, error_detected and resume callbacks gets no CXL support. The provider hooks run with memory_lock held for write: vfio-pci-core already holds it across its own hot reset, and vfio_pci_core_reset_prepare() takes it for a reset started anywhere else until vfio_pci_core_reset_done(). The hot reset records its task, so a reset that another task starts meanwhile waits for memory_lock instead of skipping it. pci_dev_lock() blocks user config access, which a PCI_COMMAND write can be waiting for while it holds memory_lock, so reset_prepare lets config access through while it waits. reset_done returns an error when the provider could not restore its device, and VFIO_DEVICE_PCI_HOT_RESET returns that error; for any other reset the provider logs it. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) [jan: added an optional reset_flush provider callback, invoked with memory_lock held after every device in the set has had its mappings and dma-bufs revoked and before pci_reset_bus(), so the provider can complete the cache maintenance a raw bus reset skips and refuse the reset if it fails (Jamie Nguyen)] Signed-off-by: Jiandi An <[email protected]>
…ions Let an admin or a variant driver keep the CXL Type-2 extensions out of the path and bind the device without them. Add a vfio_pci.disable_cxl module parameter, and let a variant driver set vdev->disable_cxl in its own init callback for per-device control. The parameter is built only when the CXL provider is configured. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…helper A provider such as vfio-cxl must reset its device itself: a CXL Type-2 function must not take an FLR, and its reset changes what the provider lets the fault path map. vfio-pci-core resets a function at five points: enable, close, VFIO_DEVICE_RESET, a virtualized PCIe or AF FLR, and the guest's CXL Reset doorbell in the virtualized CXL DVSEC. Add a mandatory reset op to the provider ops, and refuse to register ops without it. Add vfio_pci_try_reset_function(), which keeps the trylock of pci_try_reset_function(): it returns -EAGAIN when the device lock is contended, and otherwise runs the provider's reset with the device lock and memory_lock held. The close path keeps its bridge and device trylock and calls a locked variant. vfio_pci_try_reset_function() saves the PCI state first, as pci_try_reset_function() does, for the provider's reset to restore. The locked variant does not save: on close the reset must restore the open-time state that vfio_pci_core_disable() has just loaded. A failed save is returned and the reset is not attempted. The enable path still fails only on -EAGAIN. A provider checks at bind that it can reset the device, so reset_works is set for a provider device whatever the result of the first reset. The CXL Reset doorbell now runs the reset and reports its result in Status2: Reset Error on failure, Reset Done on success. Suggested-by: Alex Williamson <[email protected]> Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
The BARs of a vfio PCI device follow the memory enable bit: clearing it zaps their mmaps and revokes their dma-bufs, and setting it re-arms the dma-bufs. A device-specific region does not have to follow that bit. The HDM memory region of a CXL Type-2 device follows the committed HDM decoder, which changes across reset, bus reset and low power, not with Memory Space Enable. Other device-specific regions, such as nvgrace coherent memory or the IGD OpRegion, do not depend on it either. Record the region index of each dma-buf at export. vfio_pci_dma_buf_move() now moves only the BAR dma-bufs, and vfio_pci_dma_buf_cleanup() still drains every dma-buf of the device. Export vfio_pci_core_revoke_region(), which zaps the user mmaps of one device-specific region and revokes its dma-bufs, and vfio_pci_core_rearm_region(), which re-arms them. A provider calls them under memory_lock when the state its region depends on changes. Also export __vfio_pci_vmf_insert_pfn(), the pfn insert of vfio_pci_vmf_insert_pfn() without its memory enable and runtime PM checks, for the fault handler of such a region. Export vfio_pci_core_clear_memory_enable() for a provider whose failed operation left Memory Space off in the device: it zaps the BAR mmaps, revokes the BAR dma-bufs and clears vfio's record of the bit. Suggested-by: Alex Williamson <[email protected]> Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…ansitions A provider may serve state that does not survive low power. A CXL Type-2 device loses its committed HDM decoder in D3hot or D3cold, and the HDM memory region must stay out of use until the host restores the decoder. Without a hook the provider can only keep the device in D0. Add optional power_down and power_up ops. vfio-pci-core calls power_down with memory_lock held for write before the device leaves D0: in vfio_pci_set_power_state() for a lower state, and in runtime suspend before the PCI core moves the device to low power. It calls power_up with memory_lock held after the device is back in D0 with its config space restored: after a guest PM control write returns the device to D0, and in runtime resume. If a request for a lower state leaves the device in D0, because the transition failed or a quirk keeps it there, vfio-pci-core calls power_up at once. The other paths that return a device to D0 either reset it afterwards (close, VFIO_DEVICE_RESET and the hot reset ioctl), where the provider restores its state, or tear it down (unregister). Suggested-by: Alex Williamson <[email protected]> Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
Add the vfio-cxl module that vfio-pci-core loads on demand for a CXL
Type-2 device, registering its callbacks at module_init. Its reset op
runs the PCI core's reset methods until vfio-cxl serves the device
memory.
vfio-cxl is a feature of vfio-pci-core rather than a variant driver, so
its Kconfig sources above the variant drivers and its objects link right
after vfio-pci-core while still being loaded on demand:
request_module("vfio-cxl") finds it by module name.
Assisted-by: LLM
Signed-off-by: Manish Honap <[email protected]>
(backported from https://lore.kernel.org/linux-cxl/[email protected]/)
Signed-off-by: Jiandi An <[email protected]>
… bind At bind, build the CXL memory device for the passed-through Type-2 accelerator so it joins the CXL topology and its HDM region resolves to a host physical range. A Type-2 device has no mailbox, so there is no media-ready register to poll: set media ready directly once the component registers validate (mirroring drivers/net/ethernet/sfc/efx_cxl.c) As per current vfio-cxl support, reject a device with: - more than one HDM decoder - interleaving enabled - whose reset the host cannot service, including the CPU cache invalidation that the reset needs The CXL-core allocations are grouped with devres, in a helper that returns on the first error, so a failed bind unwinds them: a failed init leaves the device bound without CXL support, so devm would otherwise hold them until unbind. The device is not kept in D0. Low power loses the committed HDM decoder, and the host restores the decoder configuration when the device returns to D0. The contents of device memory across low power are the user's to keep. Take the bind-time copy of the cached HDM state with pci_cxl_hdm_snapshot(), which needs no device lock: probe may run in a worker on the device's NUMA node that does not own the lock. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
devm_cxl_probe_mem() maps the component registers through the CXL core, which by default claims each HDM and RAS sub-block. That collides with the full-BAR request vfio-pci-core makes when the guest opens the device. After the component registers are discovered, declare the containing BAR owned with cxl_reg_map_set_owned_resource(). The core then ioremaps the sub-blocks without requesting them, so vfio-pci-core owns the whole BAR and the requests do not collide. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
The CXL reset series restores HDM state through a prepared sequence: pci_cxl_reset_prepare() takes the region quiesce and hands out cache state that cxl_restore_state_after_pci_reset() must consume, releasing the quiesce on every exit. A driver that owns the HDM range can lose the decoder outside that sequence. vfio-cxl loses it twice over: a low power transition clears the committed decoder, and VFIO_DEVICE_PCI_HOT_RESET drives a plain secondary bus reset rather than cxl_reset_bus_function(). Neither path can hold the quiesce across its window. The quiesce is an exclusive rwsem, and a bus reset runs reset_prepare() for every device on the bus before resetting any of them, so a second CXL endpoint behind the same bridge would block on the first. A low power caller would hold it for as long as the guest keeps the device suspended. Add cxl_recover_state_after_pci_reset(), which takes and releases the quiesce within the call. Refresh the decoder settings from the cache first: the caller's snapshot comes from pci_cxl_hdm_snapshot() at bind, and a region commit or teardown since then has updated the cache. Apply the PCI command directly rather than return it, as these callers have not stopped IOMMU translation and have nothing to defer it for. Signed-off-by: Jiandi An <[email protected]>
A CXL Type-2 guest maps the device HDM memory to use its coherent CXL.mem. The HDM memory is a host physical range with no struct page, so the guest and KVM need a write-back mapping of it. Register the range as a region under VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id rather than a bespoke region type, per open because vfio-pci-core tears down all dynamic regions on close and on a failed open. This patch serves read and write on the region fd; mmap follows. Own the resolved host physical range exclusively (IORESOURCE_EXCLUSIVE) so nothing, /dev/mem included, can map a conflicting cacheable alias that would fault the host once the range is mapped write-back. Release it in the provider .release op rather than through a devm action on the PCI device, which could run after vfio-cxl has unloaded. vfio-cxl now holds the range, so its reset op runs CXL Reset with pci_cxl_reset_owned(). Host access to the range is safe only while the committed decoder is in place. Track that in hdm_valid, set at bind once the decoder is verified committed. Low power loses the decoder, so close the gate in power_down and restore the decoder with cxl_restore_state_after_pci_reset() in power_up, reopening the gate on success. Device memory can stay unavailable for seconds after the decoder is back: the device clears Memory_Active in DVSEC CXL Range 1 Size Low until the memory is usable (CXL r4.0 sec 8.1.3.8.2). At bind, wait up to the Memory_Active_Timeout the device reports for Memory_Active, since a reset just before the bind also clears it. A device whose memory then reads active brings it up by itself, and power_up waits for Memory_Active, up to the same timeout, before it reopens the gate. A device whose memory stays inactive relies on its driver and is not waited for. CXL.mem decode follows the committed decoder and Mem_Enable, not Memory Space Enable (CXL r4.0 sec 8.2.4.20.2, 8.2.4.20.7 and 8.1.3.2), so the gate does not check the Command register. Read and write copy through a page-sized kernel buffer, so memory_lock is never held across a user copy. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) [jan: rebased onto v17: cxl_restore_state_after_pci_reset() became the completion half of v17's prepared reset pair and does nothing without a cache from pci_cxl_reset_prepare(); switched the vfio_cxl_power_up() decoder restore to cxl_recover_state_after_pci_reset(), which keeps the self-contained semantics this path needs because a low power caller cannot hold the region quiesce for as long as the guest stays suspended] [jan: decide the post-reset Memory_Active wait from Mem_HwInit_Mode in the CXL DVSEC capability instead of polling at bind; the poll classified a hardware-init timeout or a config-read -ENODEV as a driver-initialised device, which skipped the wait on later resets and could reopen HDM that is not there, and it stalled every driver-initialised bind for the full Memory_Active_Timeout (Jamie Nguyen)] Signed-off-by: Jiandi An <[email protected]>
A CXL Type-2 guest maps the device HDM memory to use its coherent CXL.mem, so the guest and KVM need a write-back mapping of the HDM memory region. Make the region mmap-able. The range has no struct page, so the mapping is VM_PFNMAP and its fault handler inserts the pfn, at huge-page order where the alignment allows. The fault handler inserts a pfn only while the decoder is known good and the media is ready, and returns SIGBUS otherwise. It inserts with __vfio_pci_vmf_insert_pfn(), so a fault does not depend on memory enable; low power already closes the gate through power_down. When the gate closes, in low power and later in reset and bus reset, revoke the region with vfio_pci_core_revoke_region() so no live mapping outlives the decoder, and re-arm it once the decoder is restored. The BARs keep following the memory enable bit; this region does not. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
The HDM memory region is mmap-able, so an fd holder can fault its struct-page-less device memory in from the host CPU. A memory error on that range cannot be routed through the normal struct-page path and would otherwise escalate to a fatal host SError. Register the range with memory_failure() via register_pfn_address_space() so such an error is instead contained to a SIGBUS to each process that maps the poisoned page. Provide a pfn-to-vma-offset callback so memory_failure() can map a poisoned pfn back to the file offset of each user mapping; the region is a single linear range at hpa_range.start, so recover the offset the same way the fault handler derived the pfn. register_pfn_address_space() returns -EOPNOTSUPP when CONFIG_MEMORY_FAILURE is off, in which case expose the region without containment. Unregister the range on close. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…o the guest A CXL Type-2 guest reads the HDM decoder registers to learn the HDM region it was handed. Those registers live in the component BAR that vfio-pci owns. Map the decoder block at bind: its location comes from the pdev->hdm enumeration cache, and vfio-pci owns the BAR, so map it without claiming the block and expose it as a second, read-only region under VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id, registered per open like the HDM memory region. Serve reads live from the mapped block and absorb writes without forwarding them to hardware. The block is BAR MMIO, so it decodes only with Memory Space enabled: serve reads under memory_lock and fail them with -EIO while memory is disabled, as vfio_pci_bar_rw() does. A failed decoder restore leaves Memory Space off; clear vfio's record to match. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…t BAR access The HDM decoder register block lives in the component BAR that vfio-pci owns. The guest reads the decoder only through the trapped, read-only decoder region; it must not reach the physical registers through the BAR directly, and a write through the BAR could uncommit or move an unlocked decoder that the host relies on. Add the block to the generic excluded-range list with VFIO_PCI_EXCLUDE_MMAP | VFIO_PCI_EXCLUDE_READ | VFIO_PCI_EXCLUDE_WRITE similar to the MSI-X exclusion. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
The HDM memory fault inserts a PFN only when the decoder is in a known-good state (cxl->hdm_valid), so the host CPU never faults the coherent range into a disabled decoder. VFIO_DEVICE_PCI_HOT_RESET drives a plain secondary bus reset rather than the CXL reset sequence. A secondary bus reset only reaches a CXL endpoint when the upstream port has SBR unmasked (CXL r4.0 sec 8.1.5.2). With SBR masked the PCI core now refuses the bus reset, so the hot reset fails instead of doing nothing. With SBR unmasked the reset is a Hot Reset, which returns the decoder's RWL fields, Commit included, to their defaults and decommits it (CXL r4.0 sec 8.2.4.20.7, Table 8-1). Take the HDM memory region out of use in reset_prepare. In reset_done, when the port has SBR unmasked and the decoder is no longer committed, restore it with cxl_restore_state_after_pci_reset(), the helper the PCI core's cxl_bus reset method uses, and put the region back. A decoder that is still committed, after an FLR or a cxl_bus reset that restored it itself, is left alone. If the restore fails, reset_done returns the error, the hot reset ioctl fails with it, and the region stays out of use until a reset succeeds. reset_done also runs after a reset method that failed and after resets that do not reach the decoder; then put the region back only if it was in use when reset_prepare ran. Before putting the region back, reset_done waits for Memory_Active as power_up does, and returns -ETIMEDOUT if the memory does not come back. AER and DPC recovery can reset the link without reset_done. The HDM region is taken out of use when the error is reported and put back when recovery resumes the device only if it was in use before and the decoder is still committed; recovery does not restore config space, so otherwise the region stays out of use until the user resets the device. The resume path also waits for Memory_Active before it puts the region back. vfio_cxl_sbr_unmasked() mirrors the cxl_sbr_masked() check the PCI core uses (drivers/pci/pci.c) for its own CXL bus reset. power_up now also skips the restore when the decoder is still committed, as after a D3 request that left the device in D0. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) [jan: rebased onto v17: switched the vfio_cxl_reset_done() decoder restore to cxl_recover_state_after_pci_reset() for the same reason as the power_up path, and reworded the comment that claimed the cxl_bus reset method uses the same helper; it now uses the prepared pair, which this path cannot enter because a bus reset runs reset_prepare() for every device on the bus before resetting any of them] [jan: write back and invalidate the host CPU caches over the HDM range before a raw bus reset, in a reset_flush provider callback that runs under memory_lock after the mappings are revoked and refuses the reset if the maintenance fails, since vfio_pci_zap_bars() does not reach the device- specific HDM region and a revoke alone leaves dirty lines behind a decoder the reset removes (Jamie Nguyen, Matt Ochs)] Signed-off-by: Jiandi An <[email protected]>
… to userspace Advertise the assignment with a device-info flag so a VMM can tell a CXL Type-2 device apart from a device bound without the CXL extensions. On the trapped decoder register region, add a capability carrying the BAR and offset of the HDM decoder block so the VMM can place the trapped window where the guest expects to find it. The size of the guest's memory window comes from the HDM memory region. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
A Type-2 accelerator issues ATS-translated DMA to addresses inside its own HDM window, so that coherent host range must be present in the guest's IOAS (the iommufd IOAS backing the nested SMMU stage-2). iommufd maps a struct-page-less range only by fd, via IOMMU_IOAS_MAP_FILE over a dma-buf; a userspace-VA IOMMU_IOAS_MAP of the HDM mmap is rejected because the VMA is VM_IO | VM_PFNMAP. Without a dma-buf the range could only be mapped through an out-of-tree PFNMAP work-around. vfio-pci already exports BAR memory as a P2P dma-buf, but the exporter is BAR-only: vfio_pci_core_feature_dma_buf() rejects any region index at or above the ROM index, and vfio_pci_core_get_dmabuf_phys() resolves the physical range from a PCI BAR. The HDM memory region is a dynamic device-specific region, not a BAR. Let a device-specific region reach the device's get_dmabuf_phys(): a region index at or above VFIO_PCI_NUM_REGIONS skips the BAR-resource check and is validated by the driver instead, bounded to the regions that exist. Install a CXL-aware get_dmabuf_phys() in the vfio-cxl provider that returns cxl->hpa_range for the HDM memory region and delegates real BARs to the core, keeping the BAR path unchanged and the core free of CXL knowledge. The HDM window is coherent host memory with no p2pdma provider of its own, so borrow BAR 0's, matching nvgrace-gpu's handling of its non-BAR device memory. The iommufd importer does not consume the provider; the scatterlist map path (real peer DMA) is left to a follow-up once upstream grows a negotiated interconnect for coherent CXL memory. Mark a device-specific dma-buf so vfio_pci_dma_buf_map() refuses that path, which would translate the range with the borrowed provider's bus offset. The HDM dma-buf belongs to its region, not to the BARs: a memory enable change leaves it alone, and vfio-cxl revokes and re-arms it with the region when the decoder state changes. A new dma-buf of a device-specific region starts in the state its region reports through a new optional revoked() region op, so an HDM export made with memory disabled starts armed, and one made after a failed reset starts revoked. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
A CXL Type-2 function must not take an FLR in place of CXL Reset: an FLR leaves CXL.cache and CXL.mem state, the CXL DVSEC and the HDM contents untouched (CXL r4.0 sec 9.5), so device memory and cache would carry one user's data to the next. CXL Reset resets that state as well (sec 9.7). The PCI core orders cxl_reset ahead of flr in pci_reset_fn_methods[], but falls through to FLR when cxl_reset returns -ENOTTY. vfio-pci-core routes its reset points through the provider reset op. In vfio-cxl that op runs pci_cxl_reset_owned() with the HDM range claim. The reset resets the function, clearing or randomizing volatile device memory, and restores the HDM decoder and PCI config state, so it is a complete replacement for pci_try_reset_function() on a CXL device. The op runs with memory_lock and the PCI device lock held. Take the HDM memory region out of use for the duration of the reset, so a fault cannot insert a PFN into a decoder that is being torn down, and put it back only once the reset has restored the decoder and the device reports its memory active again. After CXL Reset with Memory Clear the device can keep Memory_Active clear for seconds (about 3 s on the device this was tested on), and a host access to the HDM range in that window raised an SError. A reset whose memory does not come back within the Memory_Active_Timeout fails like any other. A failed reset returns its error and leaves the region out of use until a later CXL Reset succeeds. Neither a decoder restore after low power nor a hot reset reopens it in between, since device memory may still hold data the reset was meant to clear. If a failed reset left CXL.cache or CXL.mem disabled in the DVSEC Control register, which makes cxl_reset refuse the device, put back the Control value taken at bind before the next reset. A failed reset leaves Memory Space disabled; clear vfio-pci-core's record of it to match. AER recovery does not reopen HDM after a failed CXL Reset either. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…ssthrough Describe the vfio-cxl provider module: - Address model, - Two regions, - Live decoder region and the guest commit, - DVSEC virtualization, - The dma-buf export for iommufd stage-2 mapping, - The reset contract, including the hot reset, - Low power so the supported topology and the kernel/VMM split are on record. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) [jan: documented the vfio-pci soft dependency on vfio-cxl, the request_module_nowait() fallback that binds without CXL support when the provider is not loaded, and why the provider cannot be loaded synchronously from probe (Jamie Nguyen)] Signed-off-by: Jiandi An <[email protected]>
Exercise the vfio-cxl contract on a bound Type-2 device: the CXL device-info flag and the two vendor-type regions, the HDM memory mmap (including a 2 MB huge fault), the read-only trapped decoder region against the live host-committed decoder, and the guest's DVSEC reset doorbell and manual HDM discovery. Check that the decoder register region reads fail with EIO while Memory Space is disabled, and that the decoder reads back committed and HDM access works after low power entry and exit. Map the HDM memory the way a VMM does: export it as a dma-buf with VFIO_DEVICE_FEATURE_DMA_BUF and map the fd into the IOAS with IOMMU_IOAS_MAP_FILE. The struct-page-less range cannot be pinned through a userspace VA, so the by-fd path is the one that works. Skip when the kernel has no CXL dma-buf exporter, so the test becomes a real pass once that support is present. Check that a dma-buf exported with Memory Space disabled maps once it is enabled, that read and write on the HDM region span several pages and may target a mapping of the region itself, and that a multi-dword read of the decoder registers matches single-dword reads. Also check that close restores the config state saved at open, and that a D3hot to D0 cycle restores the decoder, also when the device stays in D0. The sparse mmap test accepts zero areas when the page-rounded decoder hole covers the whole BAR. The multi-chunk read and write test is sized in pages of the running kernel. Add a partial-mmap path to the vfio selftest library so the trapped component BAR maps its sparse ranges, and an __iommu_map_file() helper for the dma-buf mapping. Assisted-by: LLM Signed-off-by: Manish Honap <[email protected]> (backported from https://lore.kernel.org/linux-cxl/[email protected]/) Signed-off-by: Jiandi An <[email protected]>
…rough Build the CXL stack in and annotate the Kconfig symbols introduced by the CXL reset v17, SMCCC cache clean/invalidate v5 and vfio CXL Type-2 passthrough v6 series. CONFIG_FWCTL, CONFIG_RUST_FWCTL_ABSTRACTIONS and CONFIG_ACPI_APEI_EINJ follow from CONFIG_CXL_BUS=y rather than being needed directly. Based on the annotations from the Ubuntu nvidia-7.3-next branch: commit 984a88b ("NVIDIA: VR: SAUCE: [Config] Add a CXL config for CXL type 3 devices") commit 4280ec6 ("NVIDIA: VR: SAUCE: [Config] Enable CXL DAX and KMEM built-in for CXL memory access") commit f8c6367 ("NVIDIA: VR: SAUCE: [Config] CXL config annotations for Type-2 device and RAS support") Entries that debian.master already satisfies are dropped, as is CONFIG_PCIEAER_CXL which no longer exists. Per Matt Ochs's review, the notes are written as "m to y" rather than "m->y": the packaging annotations parser treats the first '>' after note< as the end of the note, so an arrow truncates the text when updateconfigs regenerates the file. The file is serialized with debian/scripts/misc/annotations so the entries sit in canonical order and the tool rewrites it unchanged. Signed-off-by: Jiandi An <[email protected]>
fd7aa2c to
50b001d
Compare
Description
LP: https://bugs.launchpad.net/ubuntu/+source/linux-nvidia-bos/+bug/2174630
Enables CXL Type-2 device support for Vera-Rubin GPUs in CXL mode: CXL Reset as a PCI reset method, the arm64 cache-maintenance provider it needs, and VFIO passthrough of a CXL Type-2 device to a QEMU/KVM guest.
Series backported
Upstream prerequisites
All three are 7.4-merge-window commits, cherry-picked unmodified, and are the only non-
NVIDIA: SAUCE:commits in the branch. Series 2 does not build without the two SMCCC commits, which are applied in upstream order ahead of it; the cache-maintenance fix is applied ahead of its first user instead.fe7f2a805443— firmware: smccc: Add an Arm SMCCC busProvides
include/linux/arm-smccc-bus.h,drivers/firmware/smccc/bus.cand the modalias plumbing.d8263a131bb0— firmware: hwrng: arm_smccc_trng: Register as an SMCCC deviceConverts
smccc.ctoarm_smccc_device_register(), which the cache provider also uses.b2f3a170ae96— cache: Make cpu_cache_invalidate_memregion() error out if there are no handlersMakes the generic framework return
-ENXIOwhen no cache-maintenance provider is registered, instead of reporting success for a writeback that never happened. Needed by the CXL reset paths, which must not drop a decoder while the CPU may still hold dirty lines for its range.Adaptation: v6 was written against CXL reset v15
Series 3 calls
cxl_restore_state_after_pci_reset(pdev, snapshot), which in v15 was a self-contained "restore the HDM decoder" helper that took the region quiesce itself. v17 reused the name for the completion half of a prepared reset pair; it takes six arguments and is a no-op when thecacheargument is NULL.vfio cannot use v17's prepared pair in either of its two recovery paths:
cxl_region_quiesce_lock()isdown_write()on a global rwsem.pci_dev_save_and_disable()runsreset_prepare()for every device on the bus before resetting any of them, so a second CXL endpoint behind the same bridge would block on the first.Both sites are "unplanned recovery" in v17's own terms, so they need the self-contained form. Added as one new patch:
NVIDIA: SAUCE: PCI/CXL: Recover HDM state after an unprepared resetdrivers/pci/cxl.c(+48),include/cxl/pci.h(+9)Adds
cxl_recover_state_after_pci_reset(), exported to thevfio-cxlmodule. It is v15's semantics rebuilt from v17's own internals — no new behaviour — and takes/releases the quiesce within the call. It also revivescxl_hdm_cache_copy_decoders(), which v17 left with no callers (see "Feedback for the series authors").Modified patches
Five commits deviate from the posted mails. Each carries a
[jan: ...]note under its(backported from ...)line.drivers/cxl/core/region.ccxld->configsub-struct conversion on top of the existing multi-level interleave logicdrivers/firmware/smccc/smccc.carm-rsismccc_devices[]entry carried as contextdrivers/pci/cxl.c,include/cxl/pci.h@ownedthroughcxl_reset_cache_prepare(), dropped the unused@devparameter, kept v17's cache-aware bodies and VF/multifunction checkinclude/cxl/pci.hand exports it tovfio-cxl; v17 reshaped everything it touchesdrivers/vfio/pci/cxl/vfio_cxl_core.cvfio_cxl_power_up()now callscxl_recover_state_after_pci_reset()drivers/vfio/pci/cxl/vfio_cxl_core.cvfio_cxl_reset_done(), plus corrected a comment claiming this path uses the same helper as thecxl_busreset methodcxl_busto the prepared pair, which this path cannot enterConfiguration
One
[Config]patch,debian.nvidia-7.3/config/annotations, 18 entries. Symbols already satisfied bydebian.masterwere deliberately left out.select VFIO_PCI_COREpreventsydebian.masterm→y;ACPI_APEI_EINJ_CXLdepends onCXL_BUS <= ACPI_APEI_EINJ