feat(usb): enumerate devices on xHCI root hub ports by nokosaaan · Pull Request #739 · tier4/awkernel

nokosaaan · 2026-10-20T09:20:04Z

Description

Enumerate devices attached to the root hub ports of the xHCI controller brought up in #737. For each connected device, the driver now issues Enable Slot and Address Device, then reads its Device Descriptor and logs the VID/PID and class. No class driver is attached yet; that comes in the PL2303 / CDC-ACM follow-ups.

What this PR does:

Port power and discovery: sets PORTSC.PP on unpowered ports (xHCI §4.19.1.1), waits 500 ms for VBUS to settle, and snapshots the ports with CCS = 1 before touching any of them.
Per port:
1. Port reset: set PR and wait for PRC, with the RW1CS change bits handled per spec.
2. Enable Slot, validating the returned slot ID.
3. Address Device (BSR = 0), with an Input Context for the Slot and EP0. The EP0 max packet size is chosen from the port speed.
4. GET_DESCRIPTOR(Device) over an EP0 control transfer (Setup / Data / Status TRBs).
Control transfers: a generic HcCore::control() builds the TRBs from a typed SetupPacket and tolerates a Short Packet on the data stage.
Robustness: a failure on one port is logged, stale events are drained, and enumeration continues with the next port.

Changed files: pcie/usb/xhci/mod.rs, regs.rs (port register and TRB definitions), ring.rs (Transfer Ring alias and slot_id()).

How was this PR tested?

scripts/ci.sh passes on my machine.
make qemu-x86_64 (controller only, as added in feat(usb): xHCI host controller bring-up (polled) #737): the No-Op succeeds, then no device found at boot, and the kernel boots to the shell.

QEMU with devices attached, using make qemu-x86_64's arguments plus -device usb-kbd,bus=xhci.0 -device usb-tablet,bus=xhci.0 -netdev user,id=un0 -device usb-net,netdev=un0,bus=xhci.0. All three devices are enumerated:

xHCI: 0000:00:02.0 xHCI: device detected on port 5
xHCI: 0000:00:02.0 xHCI: slot 1 assigned for port 5
xHCI: 0000:00:02.0 xHCI: slot 1 addressed
xHCI: 0000:00:02.0 xHCI: slot 1 — USB 2.00 VID=0x0627 PID=0x0001 class=0x00/0x00/0x00 bcdDevice=0x0000
...
xHCI: 0000:00:02.0 xHCI: slot 3 — USB 2.00 VID=0x0525 PID=0xa4a2 class=0x02/0x00/0x00 bcdDevice=0x0000
All CPUs are ready.

Without -device qemu-xhci, there is no xHCI output and boot is unchanged.

Notes for reviewers

DMA lifetime. Once a slot is addressed, the controller holds pointers to its Device Context and EP0 ring. Freeing them would let the controller DMA into reused memory.
- UsbDevice is therefore always kept in ManuallyDrop, which leaks about 12 KiB per device at boot, once.
- The transient Input Context is freed only if its command did not time out.
- Control transfers use a persistent per-device data buffer, so a late completion after a timeout cannot write into freed memory.
Untrusted input. Descriptor fields from the device are read through fixed-size array patterns; nothing is indexed with device-provided lengths.
Bounded waits. Port reset, each command and each transfer have explicit polling budgets, and a timeout is returned as an error.
Scope.
- Root hub ports only; devices behind a USB hub are not enumerated.
- Enumeration happens once at boot; hot-plug is not handled.
QEMU arguments. I can also add -device usb-kbd,bus=xhci.0 to QEMU_X86_ARGS, so that make qemu-x86_64 exercises enumeration by default. Let me know if you prefer that.

Power the root hub ports, reset every connected port, then Enable Slot, Address Device and read the Device Descriptor of each device over EP0 control transfers, logging its VID/PID and class. Device Contexts and EP0 rings stay allocated for the controller's lifetime because the controller keeps pointers to them; the control transfer data buffer is per-device and persistent so a timed-out transfer can never DMA into freed memory. Co-Authored-By: Claude Opus 5.5 <[email protected]>

Signed-off-by: nokosaaan <[email protected]>

ytakano · 2026-10-21T01:19:58Z

@nokosaaan

Could you make the timeout recovery safe before continuing to the next port?

drain_events() only removes events that are already available. A timed-out command can still complete afterward, while command() accepts the next Command Completion event without checking its command TRB pointer.

For example, if Address Device for port A times out and completes while port B is waiting for Enable Slot, B could accept A’s completion and use the wrong slot ID. The subsequent Address Device call could then consume B’s Enable Slot completion and release its Input Context before the controller has finished using it. wait_transfer() has a similar issue because it does not match the completion to the expected slot, endpoint, or TRB.

Please match completions to their submitted operations and keep DMA buffers alive until those operations are known to be complete. Alternatively, stop enumeration after a timeout and safely quiesce the controller before releasing the associated resources or proceeding.

Could you also add a regression test with two connected ports and a completion delayed beyond the polling timeout, verifying that the late event cannot complete the next operation or cause premature buffer release?

This is a concern from code inspection, not a runtime-reproduced failure.

…i/3-enumerate

…timeout

nokosaaan · 2026-10-23T14:23:12Z

@ytakano

Thank you for catching this. In commit bb0357fa, I implemented a two-layer fix and verified the before/after behavior in QEMU by reproducing the delayed-completion scenario.

1. Summary of Changes (`bb0357fa`)

Match completion events against the originating TRB physical address (Fix A)
- ProducerRing::enqueue() now returns the physical address of the enqueued TRB slot (Result<u64, XhciErr>) in ring.rs.
- HcCore::command() only accepts a CMD_COMPLETION event whose ev.param (Command TRB Pointer, xHCI §6.4.2.2) matches cmd_phys. Any stale CMD_COMPLETION event for an earlier TRB is logged with log::warn! and discarded in mod.rs.
- HcCore::wait_transfer() (EP0 Control Transfer) similarly verifies ev.slot_id() == slot && ev.endpoint_id() == dci and checks that ev.param matches setup_phys, data_phys, or status_phys of the current control TD.
Abort/resynchronize the Command Ring on timeout and clean up failed slots (Fix B)
- Simply calling drain_events() after a command timeout still leaves the timed-out TRB pending on the Command Ring. abort_command_ring() now writes CRCR.CA (Command Abort), polls CRCR.CRR == 0 while draining the Event Ring, resets the Command Ring (cmd_ring.reset()), and reprograms CRCR to the page-aligned ring base.
- If address_device() or read_device_descriptor() fails after ENABLE_SLOT, cleanup_failed_slot() issues DISABLE_SLOT and only clears DCBAA[slot] = 0 and drops UsbDevice (Device Context / EP0 ring) once DISABLE_SLOT succeeds (keeping the DMA pages leaked via ManuallyDrop if DISABLE_SLOT itself fails).
- Added a PORTSC.PED != 0 check and a 10 ms recovery wait (PORT_RESET_RECOVERY_USEC, USB 2.0 TRSTRCY) in reset_port().

2. Verification in QEMU (Delayed Completion Scenario)

Because QEMU's qemu-xhci processes commands synchronously inside the ring_doorbell(0, 0) MMIO handler, I tested with two attached USB devices (usb-kbd on port 5 and usb-mouse on port 6) and injected a fault that withholds the doorbell for port 5's ADDRESS_DEVICE TRB (0x57a8020) and returns Err(XhciErr::Timeout), so that both TRBs (0x57a8020 and 0x57a8030) are processed back-to-back when port 6 rings the doorbell for ENABLE_SLOT.

# Build
make x86_64 RELEASE=1

# Run QEMU with two USB devices (usb-kbd + usb-mouse)
qemu-system-x86_64 \
  -machine q35,accel=tcg -cpu Broadwell -smp 4 -m 4G \
  -chardev stdio,id=char0,mux=on,logfile=qemu_serial.log,signal=off \
  -serial chardev:char0 -mon chardev=char0,mode=readline \
  -device qemu-xhci,id=xhci \
  -device usb-kbd,bus=xhci.0 \
  -device usb-mouse,bus=xhci.0 \
  -drive format=raw,file=x86_64_uefi.img \
  -display none -no-reboot

① Before fix (`62642004`): Off-by-one completion mismatch and premature `Input Context` deallocation

[  2716 INFO] xHCI: cmd TRB 0x57a8010 (type 9) <- completion for TRB 0x57a8010 (code 1, slot 1)
[  2717 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 assigned for port 5
[  2719 WARN] xHCI: FAULT INJECTION: Address Device TRB 0x57a8020 enqueued, doorbell withheld -> report Timeout
[  2720 INFO] xHCI: slot 1 port 5: Input Context 0x57b6000 kept (timeout)
[  2720 ERROR] xHCI: 0000:00:03.0 xHCI: port 5 enumeration failed: Timeout
[  2721 INFO] xHCI: 0000:00:03.0 xHCI: device detected on port 6
[  2722 INFO] xHCI: cmd TRB 0x57a8030 (type 9) <- completion for TRB 0x57a8020 (code 1, slot 1)  ** MISMATCH **
[  2723 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 assigned for port 6
[  2723 INFO] xHCI: cmd TRB 0x57a8040 (type 11) <- completion for TRB 0x57a8030 (code 1, slot 2)  ** MISMATCH **
[  2724 INFO] xHCI: slot 1 port 6: Input Context 0x57be000 FREED
[  2725 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 addressed

As pointed out, port 6's ENABLE_SLOT (0x57a8030) consumed the delayed completion for port 5's ADDRESS_DEVICE (0x57a8020) and misidentified its slot as slot 1. Port 6's ADDRESS_DEVICE (0x57a8040) then consumed the completion for 0x57a8030 (slot 2) and immediately freed its Input Context (0x57be000).

② After fix (Fix A only: TRB physical address matching discards stale completions)

[  2724 INFO] xHCI: 0000:00:03.0 xHCI: device detected on port 6
[  2724 WARN] xHCI: discarded stale completion for command TRB 0x57a8020 (code 1, slot 1) while waiting for 0x57a8030
[  2725 INFO] xHCI: cmd TRB 0x57a8030 (type 9) <- completion for TRB 0x57a8030 (code 1, slot 2)
[  2726 INFO] xHCI: 0000:00:03.0 xHCI: slot 2 assigned for port 6
[  2726 INFO] xHCI: cmd TRB 0x57a8040 (type 11) <- completion for TRB 0x57a8040 (code 1, slot 2)
[  2727 INFO] xHCI: slot 2 port 6: Input Context 0x57be000 FREED
[  2728 INFO] xHCI: 0000:00:03.0 xHCI: slot 2 addressed
[  2729 INFO] xHCI: 0000:00:03.0 xHCI: slot 2 — USB 2.00 VID=0x0627 PID=0x0001 class=0x00/0x00/0x00 bcdDevice=0x0000

The delayed completion for 0x57a8020 is rejected by ev.param == cmd_phys, and port 6 receives its own completions for ENABLE_SLOT (0x57a8030 → slot 2) and ADDRESS_DEVICE (0x57a8040) before freeing Input Context.

③ After fix (Fix A + Fix B: Command Abort + `DISABLE_SLOT` timeout recovery)

[  3117 INFO] xHCI: 0000:00:03.0 xHCI: device detected on port 5
[  3128 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 assigned for port 5
[  3130 WARN] xHCI: FAULT INJECTION: simulating timeout on Address Device TRB 0x57a8020
[  3133 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 disabled after error
[  3134 ERROR] xHCI: 0000:00:03.0 xHCI: port 5 enumeration failed: Timeout
[  3135 INFO] xHCI: 0000:00:03.0 xHCI: device detected on port 6
[  3146 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 assigned for port 6
[  3147 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 addressed
[  3148 INFO] xHCI: 0000:00:03.0 xHCI: slot 1 — USB 2.00 VID=0x0627 PID=0x0001 class=0x00/0x00/0x00 bcdDevice=0x0000

Immediately after port 5 times out, abort_command_ring() stops and resets the Command Ring, cleanup_failed_slot() disables slot 1 via DISABLE_SLOT, and port 6 enumerates cleanly.

…ier4#739 The merge of main into this branch resolved the xHCI conflicts with the stage-3 (tier4#739) side and dropped parts of the PL2303 console mirror: - restore HcCore::reset_ep0_after_stall() / control_or_stall(); - call setup_serial() from boot_enumerate() again and keep the adapter; - initialize UsbDevice::serial and restore XhciDevice::hc (accessed by xhci_usb_serial_puts()); - fix ProducerRing::reset(), which ended with a stray constructor; - put "usb_serial" back into the kernel's x86 features (without it the console mirror in console.rs was compiled out). Adapt to the tier4#739 API: bulk OUT completions are now matched by slot, endpoint and TRB address like control transfers, and a device whose serial setup fails has its slot disabled via cleanup_failed_slot(). Co-Authored-By: Claude Opus 5.5 <[email protected]>

nokosaaan and others added 2 commits October 20, 2026 17:50

add: Makefile argument

6264200

Signed-off-by: nokosaaan <[email protected]>

nokosaaan assigned nokosaaan and ytakano and unassigned ytakano and nokosaaan Oct 20, 2026

nokosaaan added 2 commits October 23, 2026 14:59

Merge branch 'main' of https://github.com/tier4/awkernel into usb-xhc…

96e516e

…i/3-enumerate

fix(usb): match xHCI completion events by TRB address and recover on …

bb0357f

…timeout

ytakano approved these changes Oct 26, 2026

View reviewed changes

ytakano merged commit 3d788a2 into tier4:main Oct 26, 2026
1 check passed

Navigation Menu

Repository navigation

feat(usb): enumerate devices on xHCI root hub ports - #739

feat(usb): enumerate devices on xHCI root hub ports#739
ytakano merged 4 commits into
tier4:mainfrom
nokosaaan:usb-xhci/3-enumerate

nokosaaan commented Oct 20, 2026

Uh oh!

ytakano commented Oct 21, 2026

Uh oh!

nokosaaan commented Oct 23, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nokosaaan commented Oct 20, 2026

Description

Related links

How was this PR tested?

Notes for reviewers

Uh oh!

ytakano commented Oct 21, 2026

Uh oh!

nokosaaan commented Oct 23, 2026

1. Summary of Changes (bb0357fa)

2. Verification in QEMU (Delayed Completion Scenario)

① Before fix (62642004): Off-by-one completion mismatch and premature Input Context deallocation

② After fix (Fix A only: TRB physical address matching discards stale completions)

③ After fix (Fix A + Fix B: Command Abort + DISABLE_SLOT timeout recovery)

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

1. Summary of Changes (`bb0357fa`)

① Before fix (`62642004`): Off-by-one completion mismatch and premature `Input Context` deallocation

③ After fix (Fix A + Fix B: Command Abort + `DISABLE_SLOT` timeout recovery)