diff --git a/.claude/ui-overhaul.md b/.claude/ui-overhaul.md new file mode 100644 index 0000000..c5caf17 --- /dev/null +++ b/.claude/ui-overhaul.md @@ -0,0 +1,300 @@ +# Hub Manager UI Overhaul: Follow-up Work + +The web client was redesigned (black + Cornell red, top tab bar, side sheets, three-step flash +plan) in web-client `feat/ui-overhaul`. That change is **frontend only**: no cloud-services, +rpi-hub-server or API contract changes. This file lists what the backend and contract need for +the new UI to be fully supported, with STM32 flashing first, and the frontend shortcuts to +delete once each backend change lands. + +Written for humans and agents. Each item says why it matters, what to change in which repo, +and when it is done. Do them in the order of the table below unless there is a reason not to. + +## Rules (read first) + +- **The contract comes first.** Wire types live in cloud-services (`src/models.py`, + `src/protocol/bench_v1.py`). After changing them, run `python -m src.protocol.export` in + cloud-services to rewrite `contracts/openapi.json` (CI fails if it is stale), then + `npm run gen:types` in web-client. Never hand-edit `web-client/src/types/api.gen.ts`. +- **Keep the mock in step.** `web-client/src/mock/` is typed from the contract. Update it in the + same PR so `npm run dev:mock` shows the new behavior and `npm run type-check` passes. +- **Old hubs keep working.** Hubs are updated one at a time, so every new field is optional and + the UI must behave as it does today when a field is missing. +- Branches are `feat/...` or `fix/...` with a PR into `main`. Merge submodule PRs first and the + hyperloop-gui PR (which bumps the submodule pointers) last, because merging it deploys. + +## What shipped (frontend only) + +| Area | Where in web-client | +|---|---| +| Design tokens (colors, fonts) | `src/index.css`, the `@theme` block | +| Header, socket handlers, sheet host, toasts | `src/components/layout/AppShell.tsx` | +| Pages: Hubs, Devices, Telemetry, Flash | `src/pages/*.tsx` | +| Side sheets: hub, device, terminal, add streams, schema | `src/components/sheets/` | +| SVG charts, merge, export | `src/components/telemetry/ChartPanel.tsx` | +| Ports and connections per hub (shared) | `src/stores/deviceStore.ts` | +| Open sheet, toasts | `src/stores/uiStore.ts` | + +### Frontend shortcuts to remove later + +The UI works around missing backend support in these places. Each points to the item that +replaces it. + +| Shortcut | File | Replaced by | +|---|---|---| +| A pending flash is shown as in progress (animated bar, elapsed time) because hubs never report `running` | `src/pages/Flash.tsx` (`FlashProgress`) | 1 | +| One "Compile and write" timeline row, since there are no stages | `src/pages/Flash.tsx` (`FlashProgress`) | 2 | +| Which tool flashes a board (`arduino-cli` or `openocd`) is a hard-coded map | `src/config/boards.ts` (`FLASHERS`, `flasherFor`) | 3 | +| The board list duplicates `rpi-hub-server/config/boards.yaml` | `src/config/boards.ts` (`BOARDS`) | 3 | +| Idle connections are closed by the browser, only while someone has the Devices page open | `src/stores/deviceStore.ts` (`enforceInactivity`), "Quiet · closes in" hint in `src/pages/Devices.tsx` | 8 | +| Device counts on Hubs cost two requests per hub every 30 s | `src/stores/deviceStore.ts` (`refresh`) | 7 | +| CPU, memory and disk show "—" for up to 30 s after a page load | `src/pages/Hubs.tsx`, `src/components/sheets/HubSheet.tsx` | 6 | + +## Work list + +| # | Change | Repos | Contract change | Size | +|---|---|---|---|---| +| 1 | Hub reports `running` when a task starts | rpi-hub-server | no | S | +| 2 | Flash stages and progress (`stage`, `progress`) | all three | yes | M | +| 3 | Hub reports each board's flasher; cloud serves the board list | all three | yes | M | +| 4 | Typed flash result and a "Show log" view | cloud-services, web-client | yes | S | +| 5 | Cancel a running flash | all three | yes | M | +| 6 | Latest health in REST (no empty meters on load) | cloud-services, web-client | yes | S | +| 7 | Port and connection counts in the hub list | cloud-services, web-client | yes | S | +| 8 | Close idle serial connections on the hub, not in the browser | rpi-hub-server, web-client | no | S | +| 9 | Terminal and chart history after a reload | cloud-services, web-client | yes | M | +| 10 | Custom chart schemas actually used for parsing | web-client (then cloud-services) | later | S | +| 11 | Self-host Uber Move | web-client | no | S | +| 12 | Frontend polish: code splitting, phone layout, tests | web-client | no | M | + +Items 1 to 5 complete STM32 flashing support. Items 6 to 12 are independent. + +## STM32 flashing: full support + +What already works end to end: the STM32F407G-DISC1 is detected by USB id (`boards.yaml`, +`disco_f407vg`), `.ino` is compiled with arduino-cli and programmed with OpenOCD over the +on-board ST-LINK (`write, verify, reset`), `.bin/.elf/.hex` images skip the compile, several +ST-LINKs on one hub are told apart by serial number (`OpenOcdFlasher.adapter_serial`), and the +Flash page accepts those formats and says "OpenOCD over ST-LINK". What is missing is +visibility while it runs, control over a running flash, and one source of truth for boards. + +### 1. Report `running` when a task starts + +**Why:** `rpi-hub-server/src/bench/command_handler.py` reports a task when it is queued +(`handle_command`) and when it finishes (`_task_worker`), never in between. `BaseTask.run()` +sets `RUNNING` internally (`src/bench/tasks/base_task.py`) but nobody sends it. A 5 to 15 minute +STM32 compile therefore looks queued until it is done. The mock backend sends `running`, which +is why `npm run dev:mock` does not show the problem. + +**Change (rpi-hub-server):** in `_task_worker`, emit a status right after the task starts: +create the run task, wait until `task.status == RUNNING` (or emit from inside `BaseTask.run()` +through a callback), then call `self._report_task_status(task)`. No contract change: `running` +is already a valid status. + +**Then (web-client):** nothing is required; `hubStore.updateTaskStatus` records `started_at` and +the timeline marks the step active. Optionally restore distinct pending copy in +`FlashProgress` ("Queued on ") now that pending really means queued. + +**Done when:** flashing an `.ino` to the DISC1 shows "Compiling and flashing…" with the +"Compile and write" row active and its start time filled in while the compile runs. + +### 2. Flash stages and progress + +**Why:** one bar and one row cannot tell "compiling for 6 minutes" from "stuck". The contract +already has `TaskStatusBroadcast.progress` (0 to 100) and the UI already draws it (a determinate +bar replaces the animated one as soon as `progress` is set), but no hub sends it. + +**Contract (cloud-services):** add an optional `stage` to the hub's task status message in +`src/protocol/bench_v1.py` and to `TaskStatusBroadcast`: +`"queued" | "compiling" | "writing" | "verifying" | "resetting"`. Pass it through unchanged like +`progress`. + +**Hub (rpi-hub-server):** +- `src/uplink/hub_agent.py` `send_task_status_update`: forward `stage` (it already forwards + `progress`). +- Flash tasks report `stage` at each step and `progress` where the tool exposes it. Tool runs + need streaming output instead of waiting for exit (`run_tool` in + `src/bench/flashing/tools.py`): add a per-line callback. +- OpenOCD's `program` command prints fixed markers on stderr: `** Programming Started **`, + `** Programming Finished **`, `** Verify Started **`, `** Verified OK **`, + `** Resetting Target **`. Map them to `writing`, `verifying`, `resetting`. +- arduino-cli: `compiling` while `compile` runs (no percentage; keep `progress` null), then + `writing`. avrdude's upload prints `Writing | ####` progress bars, which can be turned into a + percentage. +- Rate-limit progress messages (say one per second): they go over the cellular uplink too. + +**Web-client:** add `stage` to `Task` (`src/types/index.ts`) and store it in +`hubStore.updateTaskStatus`. In `FlashProgress` (`src/pages/Flash.tsx`), replace the single +"Compile and write" row with Compile (`.ino` only), Write, Verify (OpenOCD only) and Reset, each +active or done from `stage`. Update `src/mock/server.ts` to step through the stages. + +**Done when:** an STM32 `.ino` flash walks Compile, Write, Verify, Reset in the timeline, and +the bar fills during the write. + +### 3. One source of truth for boards and flashers + +**Why:** the web client hard-codes the board list (`BOARDS`) and which boards use OpenOCD +(`FLASHERS`) in `src/config/boards.ts`, copied from `rpi-hub-server/config/boards.yaml`. Adding +a board (for example another STM32 Discovery or Nucleo) means editing both, and they drift. + +**Hub (rpi-hub-server):** `Board.summary()` in `src/bench/boards.py` also returns `flasher` and +`reset`, so every `PortInfo.board_profile` carries them. Send the full registry summary (id, +name, fqbn, artifacts, flasher) in the handshake. + +**Contract (cloud-services):** add optional `flasher` (`"arduino-cli" | "openocd" | "none"`) and +`reset` to `BoardProfileInfo`. Add `GET /api/boards`, the union of the registries the connected +hubs reported, so the Flash page's "Board" dropdown lists what the hubs can actually flash. + +**Web-client:** `flasherFor()` reads `board.flasher` and falls back to the current map only for +old hubs; the Board dropdown loads from `/api/boards` with `BOARDS` as the offline fallback. +Delete both once every hub is updated. + +**Done when:** adding a board to `boards.yaml` on a hub makes it appear in the Flash page's +Board dropdown, with the right "Flashed with …" line, without a web-client change. + +### 4. Typed flash result and "Show log" + +**Why:** flash results already carry the tool output: OpenOCD returns +`{board_fqbn, flash_duration_ms, output, flasher}` with the last 2000 characters of its log. +`TaskStatusBroadcast.result` is an untyped dictionary, so the UI does not show it. When a flash +fails, the log is the first thing anyone needs. + +**Contract (cloud-services):** define `FlashResult` (`output`, `flash_duration_ms`, `flasher`, +`board_fqbn`) and document it on `TaskStatusBroadcast.result` for `commandType == "flash"`. + +**Web-client:** under the timeline in `FlashProgress`, add a "Show log" disclosure with the +output in the terminal style (mono, black background) and the duration. + +**Done when:** a failed STM32 flash shows the OpenOCD error text on the Flash page. + +### 5. Cancel a running flash + +**Why:** `BaseTask.cancel()` exists on the hub, but there is no cloud endpoint or command for +it. A wrong board choice or a hung ST-LINK can only be waited out (the GUI gives up after +17 minutes, `DEFAULT_TIMEOUT_MS.flash` in `src/services/commandService.ts`). + +**Contract and cloud (cloud-services):** `POST /api/hubs/{hub_id}/tasks/{task_id}/cancel` +(operators only), sending a `cancel` command to the hub. + +**Hub (rpi-hub-server):** handle `cancel` in `command_handler.py`: call `task.cancel()` on a +running task (kill the arduino-cli or OpenOCD process), drop a queued one, and report +`cancelled`. After a cancelled OpenOCD run, reset the target so the board is not left halted. + +**Web-client:** add a "Cancel" button in `FlashProgress` while the task is pending or running. +The `cancelled` state is already handled ("Flash cancelled", "Try again"). + +**Done when:** cancelling a flash mid-compile stops it on the hub within a few seconds and the +board keeps running its old firmware. + +### STM32 notes for whoever does items 1 to 5 + +- The first STM32 compile on a Pi takes several minutes (the core is large). The DISC1 entry + allows `compile_timeout: 900`; keep `DEFAULT_TIMEOUT_MS.flash` above the largest hub timeout. +- Images go to the cloud as base64 JSON. nginx allows `client_max_body_size 25m` + (`deploy/nginx/hyperloop-gui.conf`), well above an STM32F4's 1 MB of flash even with debug + `.elf` files. +- `.bin` files are written at `STM32_FLASH_BASE` (`src/bench/flashing/openocd.py`). A board with a + different flash base needs that value per board in `boards.yaml`. +- Boards without an ST-LINK (bare chips through the UART or USB DFU bootloader) need a new + flasher (`stm32flash` or `dfu-util`): a `flasher` value, a class next to `openocd.py`, and + registry entries. Item 3 makes that a hub-only change. + +## Other backend changes the new UI is ready for + +### 6. Latest health in REST + +**Why:** health arrives only as `health` WebSocket broadcasts, every 30 s. After a page load the +Hubs table and hub sheet show "—" until the next one. + +**Change:** the cloud keeps the latest health per hub and includes it as an optional `health` +on `HubInfo` (or sends the latest health for every hub to a browser when it connects). In +web-client, seed `hubStore.health` from `fetchHubs`. + +### 7. Port and connection counts in the hub list + +**Why:** the Hubs page shows device counts, which today requires `GET /ports` and +`GET /connections` for every online hub every 30 s (`deviceStore.refresh`). + +**Change:** add optional `portCount` and `connectionCount` to `HubInfo`, or +`GET /api/hubs?include=ports,connections`. web-client then fetches full port lists only on the +Devices and Flash pages. + +### 8. Close idle connections on the hub + +**Why:** "close a connection whose byte counters have not moved for 60 s" runs in the browser +(`deviceStore.refresh({ enforceInactivity: true })`), only while someone has the Devices page +open, and a view-only session gets 403 when it tries. The behavior was kept as it was before +the redesign, not endorsed. + +**Change:** if the behavior is wanted, the hub's serial manager closes idle connections itself +(configurable per profile, off for `pod` mode) and reports a `device_event`. Then delete +`enforceInactivity`, `idleRemaining` and the "Quiet · closes in" hint in web-client. + +### 9. Terminal and chart history after a reload + +**Why:** the serial terminal keeps 500 lines per device and charts keep one hour, all in browser +memory. A reload or a second laptop starts empty, and the Custom time range can only cover what +this browser saw. + +**Change:** `GET /api/hubs/{hub_id}/telemetry` already returns recent telemetry the cloud +keeps. Add `portId` and `since` filters. In web-client, when a device is subscribed, fetch its +recent telemetry and feed it through `telemetryStore.processTelemetry` before live data. + +### 10. Custom chart schemas used for parsing + +**Why:** schemas created under Telemetry → Schemas are saved to `localStorage` +(`customChartSchemas`), but `detectSensorType` in `src/services/sensorParser.ts` only reads +`src/config/sensor-mappings.json`, so a saved schema never produces a chart. This predates the +redesign. + +**Change (web-client):** include `getCustomSchemas()` in detection, custom schemas first +(`getAllSchemas` in `src/lib/customSchemas.ts`). Later, store schemas in the cloud so the team +shares them instead of each browser having its own. + +### 11. Self-host Uber Move + +The font stack is `'Uber Move', 'Uber Move Text', 'UberMove', Archivo, system-ui, …` +(`--font-sans` in `src/index.css`). Uber Move is proprietary and on no public CDN, so today it is +used only on machines that have it installed; everyone else sees Archivo (loaded from Google +Fonts in `index.html`), which the design was drawn with. + +To serve it to everyone, with a license that allows web embedding: + +1. Put the `.woff2` files in `web-client/public/fonts/`. Do not commit them unless the license + allows it in this repository; otherwise have CI copy them in before `npm run build`. +2. Add to `src/index.css`, above `@theme`, one block per weight used (400, 500, 600, 700, 800): + + ```css + @font-face { + font-family: 'Uber Move'; + src: url('/fonts/UberMove-Bold.woff2') format('woff2'); + font-weight: 700; + font-display: swap; + } + ``` + +3. Remove the Archivo request from `index.html` once nothing falls back to it. + +### 12. Frontend polish (no API change) + +- **Code splitting.** The main chunk is about 1.6 MB (gzip 510 kB), mostly CodeMirror, jsPDF and + html2canvas. Load the Flash editor and the PNG/PDF export with dynamic `import()`. +- **Phones.** The Hubs and Devices tables scroll sideways, as in the mockup. A stacked layout + below 640 px would read better on a phone at the track. +- **Tests.** There are none beyond type-check and lint. Add Vitest for `src/lib/format.ts`, the + chart geometry in `ChartPanel.tsx`, and the stores. +- **Mock.** Send a `health` message when a mock socket connects so `dev:mock` shows meters at + once, and add `stage`/`progress` once item 2 lands. + +## Checking a change + +```bash +# web-client +npm run type-check && npm run lint && npm run build +npm run dev:mock # in-browser fake hubs; log in with any NetID and "hyperloop-dev" + +# end to end, against real code paths: cloud-services + rpi-hub-server with HUB_PROFILE=dev-sim +``` + +For STM32 changes, finish on real hardware: a DISC1 on a lab hub, flashed once with `.ino` +(compile path) and once with `.bin` (OpenOCD only), then "Open terminal" to see the new firmware +print. See `.claude/rpi-hub-setup.md` for the hub side (OpenOCD install, `lsusb` checks). diff --git a/cloud-services b/cloud-services index 46e47f2..a10436a 160000 --- a/cloud-services +++ b/cloud-services @@ -1 +1 @@ -Subproject commit 46e47f2b9fc7f7179bef359bf4480d8914a84823 +Subproject commit a10436aff2516c0e1497b21743fabe0d2387d93e diff --git a/web-client b/web-client index 1cd2f53..4da6c21 160000 --- a/web-client +++ b/web-client @@ -1 +1 @@ -Subproject commit 1cd2f53483365ac3755ffa75d419c27fa89d7cd6 +Subproject commit 4da6c21c0f934093b06b0bdfbb1748304f27eb98