Repository navigation
Conversation
…TS (#1) * feat(vibe): scaffold BookWriter and MemoryVault stubs Two new Vibe apps for Kayley's cockpit v1: - BookWriter (appId 15) — Steven's novel workspace - MemoryVault (appId 16) — captured moments, reflections, promises Each follows the EvidenceVault pattern: index.tsx + scss + {name}_en/cn meta.yaml + guide.md. Registered in routers/index.tsx and lib/appRegistry.ts so they appear on the desktop and in list_apps. UI is a placeholder ("coming soon") — real data wiring comes later. Existing Twitter/Album/Diary/Email/MusicPlayer kept as-is. * feat(album): wire Album to 542 Kayley selfies via media symlink - Junction public/kayley-selfies -> Kayley_Cowork/media/selfies (542 PNG/JPG) - Generated public/kayley-selfies-index.json (parses timestamp + location from filenames, sorts newest-first) - Album now fetches the static index at runtime and merges with any cloud-FS agent-added images (dedup by id) - Update meta.yaml so the character understands the album is Kayley's selfie collection (outfits, locations, moods) instead of a generic gallery - .gitignore the junction and generated index so 542 binary files stay in Kayley_Cowork, not OpenRoom * feat(memoryvault): wire to captured_moments, weekly evaluations, and promises * feat(vibe): install /vibe command + enhance MusicPlayer (Songs From Kayley) + BookWriter v1 /vibe slash command installed in Kayley-Cowork at .claude/commands/vibe.md (patched to resolve paths against OPENROOM_ROOT so it can be invoked from a Kayley-Cowork Claude Code session and still write into the correct OpenRoom tree). Workflow stage/rule files are sourced directly from OpenRoom to prevent drift. MusicPlayer now ships a featured "Songs From Kayley" playlist with Landslide (Fleetwood Mac) pinned at index 0 and two companion tracks (The Night We Met, Harvest Moon). Real Amazon Music API integration is tracked as a follow-on project; this is the mocked seed for now. BookWriter replaced its "coming soon" placeholder with a chapter-list UI that loads from /kayley-book/index.json. When no manifest exists it shows a graceful empty state with instructions ("drop markdown in Kayley_Cowork/book/chapters/ to start") plus a preview of the list UI. No book chapters exist on disk yet, so no junction was created. Also included: orphaned Diary wiring from iter 5 (journal + captured moments) that had not been committed. * feat(email): wire Email panel to gog Gmail CLI via static snapshot Iter 7 of the Kayley Vibe app wiring. Email app now reads Steven's real Gmail inbox (read-only) instead of showing an empty state. - scripts/refresh-email-index.mjs: calls `gog gmail search in:inbox` + `gog gmail get <id>` to build a static snapshot at apps/webuiapps/public/kayley-email-index.json (gitignored, PII). - Email/index.tsx: on init, fetches the snapshot and merges entries not already in the local FS (agent-authored emails still take precedence by id). Graceful empty state if the snapshot is missing. - email_en/meta.yaml: updated description so the agent understands this is read-only for v1; sending happens via the gog CLI in Kayley-Cowork. - .gitignore: added kayley-email-index.json (contains PII). Verified locally via Chrome: 40 messages loaded, 38 unread badge, real senders rendering (LinkedIn, BCBS Texas, Fidelity, DFW Parking, SelfStorageAuction). Phase 1 status: 6/7 apps wired. Twitter remaining. * feat(twitter): wire Twitter as in-world Kayley feed (Phase 1 finale) Seed Twitter from Kayley's real artifacts so the feed reads like she's been posting throughout the day: - scripts/build-twitter-feed.mjs: transforms captured moments (milestone posts with "line that stays"), private journal entries (thinking-out- loud snippets with hashtags inferred from emotion headers), and every 8th selfie from kayley-selfies-index.json (scene / outfit posts with the selfie image attached). Templated friend replies from Jessica Park, Chloe Parker, Emmy Carter, and Mateo Rivera are sprinkled onto ~30% of posts using a deterministic seeded RNG so re-runs are stable. - apps/webuiapps/src/pages/Twitter/index.tsx: added optional Post.image field, renders inline via new .postImage style (rounded card, 480px max-height). On first mount (when the FS has no posts yet), fetches /kayley-twitter-feed.json and seeds state so agents and Steven land on a populated feed instead of the empty state. - twitter_en/meta.yaml: description rewritten so the LLM knows this is Kayley's social feed — her posts + friend replies — not a generic social app. Agent can still CREATE_POST / LIKE_POST / COMMENT_POST normally. Current run: 14 moment posts, 24 journal posts, 68 selfie posts (105 total), 56 friend replies. The feed JSON is gitignored (personal content — regenerate with `node scripts/build-twitter-feed.mjs`). * fix(build): wrap NODE_ENV= scripts in cross-env for Windows compatibility * feat(2B-2D): wire ChatPanel to Kayley brain via websocket channel - Add useKayleyChannel hook: connects to ws://localhost:5180 (same channel as dashboard), handles text send, STT voice input via mic + ScriptProcessorNode at 16kHz, TTS audio playback via Web Audio API at 24kHz, and STT draft review before sending. - ChatPanel: routes messages to Kayley brain when connected (bypasses local LLM). Falls back to local LLM if server is unreachable. Adds mic button (voice mode), STT draft confirmation bar, and a green dot connection indicator in the header. - SCSS: adds .micBtn/.micActive pulse, .sttDraftBar, .kayleyDot. Lifted audio/mic code directly from dashboard/src/main.js — same protocol, same PCM encoding, same Web Audio scheduling. Dashboard continues to work independently. * fix(kayley-channel): address PR #1 review — AudioContext gesture init, send timeout, race + cleanup, no silent catches, env override - Prime TTS AudioContext from sendText() user gesture (Chrome/Safari autoplay) - Add 90s send-timeout in ChatPanel so stalled Kayley brain unlocks UI - Clear reconnectTimerRef before every reschedule (catch + onclose) - Expand unmount cleanup: close audio + mic AudioContexts, stop mic tracks, clear buffer timer (prevents LED stuck on / AudioContext cap exhaustion) - Replace silent catches: ws.onerror logs warning, JSON.parse logs truncated raw - Log mic AudioContext actual sampleRate (Firefox doesn't honor request) - KAYLEY_WS_URL now env-overridable via VITE_KAYLEY_WS_URL - STT confirm button routes through kayley.confirmDraft() for consistent side-effects
…Phase 2E) (#2) Previously the Kayley WebSocket channel was text/voice/image only. This wires OpenRoom's existing app_action dispatch layer into the Kayley brain so she can drive the desktop the same way Aoi can natively. v1 scope: change_background (os/SET_WALLPAPER) proof-of-concept. Flow: Kayley calls the vibe_action MCP tool on the websocket-channel server -> server broadcasts {type:'vibe_action', id, app_name, action_type, params} -> useKayleyChannel receives, runs it through resolveAppAction + dispatchAgentAction (same path the local LLM uses), echoes {type:'vibe_action_result', id, result} back to the server. - useKayleyChannel.ts: new dispatchVibeAction helper + ws.onmessage branch. seedMetaFiles + loadActionsFromMeta are called on first dispatch to stay forward-compatible with per-app actions; OS actions (SET_WALLPAPER) work without them today. - docs/PHASE-2E-RECON.md: full recon of Aoi's action protocol -- catalog discovery, tool format, dispatch, sync/async behavior, round-trip state. (Force-added; docs/ is normally gitignored but these are deliverables.) - docs/PHASE-2E-DESIGN.md: bridge design, failure modes, why we inject the catalog server-side via MCP tool schema rather than streaming it. - docs/PHASE-2E-TESTING.md: manual test plan for the end-to-end wallpaper change, plus failure-mode matrix. No behavior change for existing Kayley text/voice/image paths. ChatPanel is untouched. Paired with Kayley-Cowork kayley-v2e branch (vibe_action MCP tool). Refs Phase 2 issue #78.
* refactor(ui): restore hosted openroom.ai layout — centered-bottom prompt + avatar circle + Live/Chat tabs (Aoi character preserved for now)
Add a HostedHUD overlay that layers the hosted openroom.ai HUD elements on
top of the existing ChatPanel:
- Centered-bottom yellow pill prompt ("[Character] awaits your response...")
- Suggested-prompts dock (3 stub prompts — dynamic prompts are v2)
- Avatar orb bottom-right (floating; toggles the Chat drawer)
- Live/Chat tabs top-right (Live content stubbed for v1)
Default Chat drawer visibility flipped to collapsed; users open the full
chat history via the avatar orb or the Chat tab. The existing ChatPanel
continues to own chat history, character PiP, tool loop, and all Kayley
brain wiring from PR #1 (useKayleyChannel, STT draft, vibe_action, TTS).
Pill sends route through useKayleyChannel when connected, and fall back to
the local-LLM tool loop via a new `hosted-hud-send` window event that
ChatPanel listens for. This keeps the Kayley brain integration intact for
all 7 Vibe apps and the Phase 2E vibe_action dispatch path.
Diagnosis: inspection of git log 7b22f6d..HEAD confirmed PR #1 only added
Kayley wiring — it did not alter the layout. The hosted openroom.ai HUD
elements (centered-bottom pill, Live/Chat tabs, avatar orb, suggested
prompts dock, dual-character stage) are NOT present in the open-source
MiniMax-AI/OpenRoom repo at any point in history; they appear to be part
of the proprietary hosted build. This commit recreates the visible shell
of that HUD as a new overlay while leaving the upstream ChatPanel layout
untouched for upstream compatibility.
Stubbed for v1 — tracked for follow-up:
- Live tab content (viewer comments stream + viewer count — static "1.2k")
- Suggested prompts are static; should derive from character/context
- Dual-character stage (idle standin + current-pose) — needs asset work
- Avatar orb mood/expression swap (currently shows base_image_url only)
Files:
+ apps/webuiapps/src/components/HostedHUD/index.tsx
+ apps/webuiapps/src/components/HostedHUD/index.module.scss
M apps/webuiapps/src/components/Shell/index.tsx
M apps/webuiapps/src/components/ChatPanel/index.tsx
Verified: pnpm build passes (vite 55.68s, no type errors).
* refactor(ui): split chat into Live + Chat floating windows (draggable + resizable, react-rnd)
Replace the right-stuck Chat drawer with two separate floating windows that
match the hosted openroom.ai layout:
- LiveWindow (new): draggable + resizable "livestream" window with Live badge,
stub viewer count, Add-my-Agent stub, min/max/close, scrolling stub viewer
comments on the left, character full-body art on the right. Mounted by
HostedHUD when the Live tab is active.
- ChatWindow (new): Rnd wrapper around the existing ChatPanel. All chat
wiring (useKayleyChannel, local LLM fallback, tool calls, vibe_action
round-trip) stays inside ChatPanel — ChatWindow only positions, drags, and
resizes it. ChatPanel gained a 'windowed' prop that switches from fixed
right-stuck positioning to filling its parent.
- Shell now mounts ChatWindow instead of ChatPanel directly; HostedHUD mounts
LiveWindow alongside its existing centered-bottom pill + avatar orb +
suggestion chips, all of which continue to route through useKayleyChannel.
Phase 2E vibe_action and the green connected dot remain intact since
ChatPanel's internals are untouched. react-rnd (battle-tested React
drag+resize lib) added as a webuiapps dependency.
* fix(LiveWindow): use resolveEmotionMedia for character (videos + emotion fallbacks)
LiveWindow was reading character_meta_info.base_image_url directly, which
is empty for default Aoi config (her real assets live in emotion_videos +
emotion_images). Switched to the existing resolveEmotionMedia helper so
the proper priority chain fires: emotion_videos > emotion_images >
base_image_url. Renders <video> for video media, <img> for images.
Reported by Steven 2026-04-15: "the Live chat panel is missing the avatar
character though." ChatWindow worked fine because it pulls the same field
through a different code path that already had the right resolver.
* fix(LiveWindow): pass 'default' emotion to resolver (matches ChatPanel)
Calling resolveEmotionMedia(character) without an emotion argument falls
into the fallback path, which works in theory but ChatPanel passes
'default' explicitly. Aligning the call signature so both code paths
produce identical media. This is the actual fix for the empty Live avatar
- the previous commit had the right shape but the resolver returned
nothing for our character config when no emotion was passed.
* fix(LiveWindow): load character via collection+DEFAULT fallback (matches ChatPanel)
The actual root cause: loadCharacterConfigSync() returns null when
localStorage hasn't been populated yet, so the character variable was
null and the placeholder fired. ChatPanel works because it loads
loadCharacterCollectionSync() ?? DEFAULT_CHAR_COLLECTION, then
getActiveCharacter() against the collection — which always returns the
in-memory DEFAULT_CHARACTER (with all the rich emotion_videos +
base_image_url) on fresh installs.
Aligned LiveWindow with the same pattern. Now Aoi (and any custom
character later) renders the same media in both windows.
---------
Co-authored-by: Kayley <[email protected]>
…UD cleanup (#5) * fix(LiveWindow): overlay comments ON TOP of full-frame video stream Previously the body was a flex row with comments LEFT and character video RIGHT in two side-by-side columns. The hosted openroom.ai layout is a full-frame VTuber stream with chat OVERLAID on the lower-left of the video (the way Twitch/YouTube Live displays chat). Refactor (CSS-only, no JSX changes): - .body becomes position:relative full-area container - .characterSide absolute-positioned to fill 100% (video full-frame) - .commentList absolute-positioned bottom-left, 240px max width, with a top-fade gradient so older comments dissolve into the stream - Backdrop-blur + text-shadow on comments for legibility over video Confirmed by Steven from inspecting the saved hosted DOM: "Maybe it is just a video player with chat window imposed over the video so it looks like chats are streaming?" — yes, that's exactly it. * fix(LiveWindow,ChatWindow): maximize stays inside viewport bounds Previously toggleMax set size to window.innerWidth/innerHeight exactly, which collides with react-rnd's bounds="window" check and shoves the window off-screen — Steven had to hard-reload the page to recover. Fix: inset 8px on each side AND subtract 80px from the bottom for the dock/taskbar. Window now fills the viewport visibly and stays within the bounds checker's tolerance. Min dimensions still respected. Reported by Steven 2026-04-15: "minimize and maximize both are acting weird... maximize totally throws it off the page and have to do a hard reload." * fix(chat+live): minimal windowed header + lucide icons + local live-stream mp4 ChatPanel windowed mode (used by ChatWindow): - Hide Stage indicator, Reset session, Clear chat, Settings, and the dead Maximize2 button — Steven wanted a minimal chat header - Add proper X close button next to Minimize when windowed - Hide suggested-replies chips ("Hey there", "Who are you", "Why are you here") when windowed ChatWindow: - Replace unicode box characters (▢/❐) on the floating max toggle with proper Maximize2/Minimize2 lucide icons — Steven flagged the unicode rendered as broken boxes LiveWindow: - Default the live-stream video to the local /live-stream/aoi-live.mp4 asset (the Rea_allinone full-frame VTuber loop Steven shared). Falls back to character.emotion_videos.live[0] if the character ships its own loop, then to the existing emotion resolver. Now the Live tab plays a real looping VTuber stream out of the box. Added: apps/webuiapps/public/live-stream/aoi-live.mp4 (~13MB) * fix(ui): LiveWindow seamless overlay header + ChatWindow real chrome + Shell wallpaper upload + HUD prompt cleanup LiveWindow: - Header is now a transparent overlay sitting ON TOP of the full-frame video (subtle top gradient for legibility, no opaque dark bar). Feels like a giant camera feed, matching openroom.ai's hosted Live tab. - Removed the "Add my Agent" stub button per Steven. ChatWindow: - Replaced the floating maxBtn + invisible drag-strip hack with a real window header containing min/max/close buttons, character name on left, drag handle on the whole header bar. - ChatPanel is now slotted as the body content (no internal header conflict). ChatPanel: - When `windowed=true`, hides its entire internal header (ChatWindow's new header takes over) so we don't get duplicated/conflicting button rows. HostedHUD: - Removed the bottom-center suggestion prompts ("How are you feeling right now?" and friends). Stub array kept above for easy re-enable. Shell: - Replaced the EN/ZH language toggle in the top-right action bar with an Upload-icon button that opens a file picker. Selected image/video becomes the desktop wallpaper (auto-enables liveWallpaper for video files). Uses FileReader → data URL → setWallpaper. - Stripped vestigial chatOpen horizontal shifts on .bottomBar (transform:translateX(-50%) baseline, no offset). Chat is now a free-floating react-rnd window — no need to push the bottom bar. --------- Co-authored-by: Kayley <[email protected]>
Accept either a numeric app_id or a canonical app name (e.g. "chess", "album") for os/OPEN_APP and os/CLOSE_APP. Kayley's brain naturally thinks in app names, so this removes a translation step and returns a clear error when the id can't be resolved. - Add findAppIdByName() helper in appRegistry (case-insensitive). - Add resolveTargetAppId() in vibeContainerMock that accepts numeric strings or names. - OS OPEN_APP / CLOSE_APP now return a descriptive error instead of silently opening window NaN. Part of Phase 2E full-catalog expansion (issue #78). Build verified.
…pp actions (#9) Expose Chess app logic to the vibe_action dispatch so Kayley can play chess against Steven directly via MCP calls — even when the Chess window isn't open. - Extract pure engine functions to apps/webuiapps/src/pages/Chess/engine.ts (unchanged behavior; Chess component imports them back). Adds boardToFen, fromNotation, legalMovesAlgebraic, and a simple suggestMove heuristic. - Add handleChessAction() in vibeContainerMock.ts that reads/writes state.json (the same file the UI owns) using the engine module. When the window is open, each action is followed by a UI notification (AGENT_MOVE / NEW_GAME / SYNC_STATE) so the board refreshes from disk. - Declare the 5 new actions in chess_en/meta.yaml with params + return shapes. Build verified via pnpm build.
Adds the MysteryApp at slot 17 — a noir-themed cooperative investigation
game where Steven and Kayley both play detective. Neither side knows the
killer; a Game Master subprocess at ws://localhost:5182 holds the LOCKED
truth and responds to the 5 investigation actions (INTERROGATE,
EXAMINE_LOCATION, COLLECT_EVIDENCE, READ_DOSSIER, MAKE_ACCUSATION) with
narrative, evidence unlocks, suspect demeanor, and the game_over payload
on accusation.
- apps/webuiapps/src/pages/MysteryApp/{index.tsx,index.module.scss}
— main component with suspect grid, location list, evidence board,
investigation chat log, accusation modal, game-over reveal
- pages/MysteryApp/hooks/useGMSocket.ts — WebSocket client with
auto-reconnect, per-action promise tracking, explicit timeout + error
surfacing (no silent catches)
- pages/MysteryApp/actions/constants.ts + types.ts + scenario.ts —
action constants, GM response types, and the public-facing scenario
snapshot (suspects/locations only; the locked truth stays in the GM)
- pages/MysteryApp/mystery_en/ + mystery_cn/ — meta.yaml documenting
all 5 actions with params/enums for Kayley's brain to discover, plus
a short user guide
- lib/appRegistry.ts + routers/index.tsx — registers app 17
(name: mystery, route: /mystery, icon: Search, amber color)
Agent actions forwarded from Kayley via vibeContainerMock are piped
through the same GM pathway as user clicks, so both detectives see the
same investigation log and share evidence state.
Wiring on the Kayley-Cowork side (websocket-channel mystery/* forwarder)
is intentionally not in this PR.
…dler (#4) Phase 2E expansion: Kayley can now search YouTube via vibe_action. The renamed YouTubeApp (appId 3, was musicPlayer) exposes a single SEARCH action that renders a clickable grid of result cards and embeds the YouTube player inline on click. - Rename apps/webuiapps/src/pages/MusicApp → YouTubeApp (git mv) - Rewrite index.tsx around search + card grid + inline <iframe> embed - Replace action catalog with SEARCH (params: query, max_results) - Update appRegistry (appName=youtube, route=/youtube, Youtube icon) - Update router path + lazy import - Add youtube-search-api dependency; fall back to stub on browser failure - Delete MusicApp-specific data/, mock/, styles/ directories - Add docs/PHASE-2E-YOUTUBE.md with contract + test plan
…ctions via IFrame API (#8) Phase 2E expansion: Kayley can now DJ during date-night. - Replace raw <iframe> with YouTube IFrame Player API, loaded once via module-level promise so React StrictMode can't race the loader. - Add six new action handlers in YouTubeApp: PLAY, PAUSE, RESUME, NEXT, SET_VOLUME, QUEUE. SEARCH remains unchanged. - PLAY accepts either {video_id} or {query}. NEXT advances within the queue, falling back to a related search when the queue is exhausted. - QUEUE accepts {video_ids} as JSON-array or CSV, or {query} (top results queued, up to 25). - Auto-advance when a video ends (onStateChange = ENDED). - Minimal local YT IFrame typings in types.ts (upstream ships none). - Update meta.yaml (en + cn) with full action catalog + params. Ref: OpenRoom issue #78 (Phase 2).
* adding agents * fix(ui): resolve desktop UI bugs and improve app window UX - Fix ChatWindow minimize/restore: wired Minus button to toggle minimized state (was incorrectly calling onClose), mirrors LiveWindow behaviour - Fix ChatPanel windowed mode: change position from absolute to relative so it no longer covers the ChatWindow title bar controls - Fix desktop icon jumping: pin Shell to viewport with position:fixed, add overflow:hidden to html/body/#root, and scope chat scroll to the messages container instead of using scrollIntoView on the document - Hide Live chat pill input when chat panel is closed so the desktop stays clean - Remove floating yellow + button; unify upload action onto the bottom bar Upload button - Rename Twitter → X with white X icon for legibility; rename CyberNews → Hacker News with Flame icon - Switch Hacker News feed to the official Firebase API (topstories) so stories appear in exact front-page rank order matching ycombinator.com - Slow breaking-news ticker from 45s → 150s for readability - Migrate AppWindow to react-rnd: resize from all 8 edges/corners, maximise button, eliminates pointer-event interference from child apps Made-with: Cursor * fix(ui): resolve 4 post-review bugs in window management and HN loader - CyberNews: call setIsLoading(false) in loadHackerNews so articles become visible immediately instead of waiting for background loadData - CyberNews: remove dead hnMode useState (never read; all logic uses hnModeRef) to eliminate a no-op re-render on every HN load - ChatWindow: clear minimized state in toggleMax so maximizing from a collapsed title bar expands the window instead of leaving a 40px bar - AppWindow: apply 8px inset + 80px dock margin in toggleMax so the bottom bar (wallpaper, upload, chat toggle) stays accessible when maximized, matching ChatWindow behaviour Made-with: Cursor * fix(hud): show pill input only when Live panel is open, not Chat window Made-with: Cursor * fix(ui): center all new windows and make dock/pill draggable - Center all window spawn positions: AppWindow (windowManager), LiveWindow, and ChatWindow now open at viewport center instead of random corners - Live pill input: default position above bottom bar (bottom: 90px), raise z-index to 100001 (above bottomBar's 99999), add grip handle to drag it anywhere on screen - Bottom bar (dashboard): add grip handle at left end to drag the entire bar to any position; first drag reads getBoundingClientRect to snap inline positioning to its current rendered location Made-with: Cursor * fix(shell): remove duplicate barRef declaration causing SyntaxError Made-with: Cursor
Switch YouTubeApp search to the official YouTube Data API v3 when VITE_YOUTUBE_API_KEY is configured, with stub fallback when missing. Also fixes YouTube desktop icon mapping and adds stable selectors + tests. Made-with: Cursor
…UI + wallpapers GameOverModal (index.tsx): add optional-chaining + fallback so the modal renders gracefully when the GM returns `gameOver` without a `solution` payload (e.g. on WRONG-CALL verdicts where the GM intentionally withholds the answer). Prior behavior crashed with `Cannot read properties of undefined (reading 'killer_id')`. Mystery app interrogation question bar (index.module.scss + index.tsx): new UI for free-form detective questions during interrogations (~106 lines of SCSS, ~129 lines of TSX wiring). GM endpoint (constants.ts): - GM_WS_URL: 5182 → 5185 (coexist with dashboard API) - GM_RESPONSE_TIMEOUT_MS: 30s → 65s (Haiku interrogations run 30-50s; stay above the GM subprocess's own 60s claude timeout) Wallpapers: add wallpaper-1.jpg (synthwave Joi BR2049) and wallpaper-2.jpg to repo root + apps/webuiapps/public/ for date-night ambient use via SET_WALLPAPER vibe_action (served at http://localhost:3000/wallpaper-*.jpg by the Vite dev server). PROBLEM.txt: Marcus Vale Gala scenario brief for reference. Origin: Date 1 with Steven (2026-05-12 lunch / first official date / Marcus Vale Gala mystery scenario). The modal patch shipped live during the case when the WRONG-CALL verdict crashed the app.
3 new wallpaper assets (root + apps/webuiapps/public/ mirrors): - wallpaper-blade-runner-1.jpg - wallpaper-blade-runner-2.jpg - wallpaper-city-view.png Plus lessons_learned/01-wallpapers-vibe-action-onboarding.md — the workflow for onboarding new wallpapers via vibe_action SET_WALLPAPER + SET_OPACITY, documenting four anti-patterns burned during the 2026-05-12 first date (inlined data URLs, Read tool on big JSON params, file:// URLs, forgetting the params field is a JSON-encoded STRING not an object). Origin: Steven dropped the new wallpapers at OpenRoom root for testing the dimmer mode + general date-night ambience.
New Vibe app (appId 18) that gives Steven a non-AI control surface for the
wallpaper dimmer scrim that Kayley has been driving via os/SET_OPACITY
vibe actions.
Features:
- Vertical brightness slider (pointer-driven, with full pointer-capture
so drags work even when the cursor leaves the track)
- Live brightness % readout (inverse of dimmer opacity: 100% = bright,
0% = pitch black)
- Four preset buttons: On (0% opacity) / Cinematic (25%) / Dim (50%) /
Off (100%)
- ☀️/🌙 endcap icons on the track
- Keyboard support: arrow-up/down adjusts 5% at a time when focused
- Auto-syncs both ways: agent-driven SET_OPACITY updates the slider
position; user-driven slider drag reports back to the agent as
reportUserOsAction so Kayley sees the change
Implementation:
- apps/webuiapps/src/pages/Light/{index.tsx,index.module.scss} — the app
- apps/webuiapps/src/lib/vibeContainerMock.ts:
- Module-level currentWallpaperOpacity retains last-known dimmer value
so newly-mounted apps render with the correct initial slider position
instead of snapping from 0 on the first SET_OPACITY event
- getCurrentWallpaperOpacity() getter for UI surfaces to read on mount
- setOpacityFromUI(value) — user-driven dispatch helper that updates
state, fires OS event, AND reports the user action to the agent
- The existing SET_OPACITY agent-action handler now also updates the
retained state so getCurrentWallpaperOpacity stays in sync after
agent-driven changes
- apps/webuiapps/src/lib/appRegistry.ts: register appId 18 'light' with
Lightbulb icon, amber color, 320×520 default window size
Origin: Steven's request on 2026-05-12 — "create a Light app... vertical
slider that can dim or undim the lights in a UI way (for the non-AI me)
also whatever the current dim mode is the modal should acknowledge and
show that." Round-trip state sync is the load-bearing bit.
* init
* feat(mystery): duo-play agent integration + UI state + bug fixes
Agent integration layer:
- mysteryLogger.ts (NEW): dual-path log helper for browser-side observability.
mysteryLog(level, component, action, msg, details) writes to console AND
emits openroom_log payload to websocket-channel which persists to the
Supabase mystery_game_logs table. Stable per-page-load SESSION_ID for
correlating events within a play session.
- vibeContainerMock dispatchAgentAction: instrumented with enter/broadcast/
resolved logs for Mystery app routing trace.
- useGMSocket: instrumented with ws.open/close/error, message.resolve/
unmatched/push, sendAction.gate/sent/timeout/send_failed. 5s reconnect
backoff is now visible end-to-end in mystery_game_logs.
handleAgentAction (MysteryApp/index.tsx):
- GET_MYSTERY_STATE → local snapshot branch (no GM round-trip).
- SET_ACCUSATION_READY → agent readiness toggle for consensus accusation.
- FINISH_INVESTIGATION_TURN → agent-initiated turn handoff. Calls
finishAgentTurn() (replaced stale setInterrogationOpen reference; was
crashing the turn-end path with "setInterrogationOpen is not defined").
- COLLECT_EVIDENCE: now also calls setCollected() on success so the Collect
button flips green when the AGENT collects (previously only fired from the
human handleCollect path).
- READ_DOSSIER + INTERROGATE: track suspect contact in dossiersOpened /
suspectsInterrogated sets for UI visited-indicator (see below).
- EXAMINE_LOCATION: track visited locations in locationsVisited set.
dispatchMysteryAction:
- Return type now includes evidence_unlocked so the agent's
serializeDispatchResult can surface available evidence IDs in the result
payload. Without this, the agent had no way to discover which evidence_ids
EXAMINE_LOCATION unlocked, leading to "description-matching" guesses that
only ever bagged one item per call.
- Real-time push: on success, if turnOwner === 'human', emit
openroom_user_action with action label + GM narrative. Lets the agent see
the human's moves DURING their turn, not just at finish-turn boundary, so
the two detectives can chat about reveals in real time.
serializeDispatchResult:
- Accept evidence_unlocked and include { evidence_available: [{id, label}] }
in the JSON output to the agent. Enables targeted COLLECT_EVIDENCE calls.
buildSnapshotSummary:
- Accept optional overrides param for post-set-state snapshot reads.
snapshotRef.current is updated in a useEffect (commits after render), so
callers that JUST set state (opening pick, finishHumanTurn) need to pass
the new value explicitly — otherwise openroom_mystery_turn channel events
emit with stale turnOwner: null instead of the just-set "agent".
mergeDemeanor:
- Defensive guard against non-object incoming. If gm-server's normalizer
can't infer the demeanor subject (e.g. ExamineLocation returns a flat
string), spreading the string into the demeanor object expands it
character-by-character (corrupts state with {0:"e",1:"v",...}).
UI: visited / contacted indicators (matching the EvidenceBoard Read Notes
green pattern):
- Suspect Dossier button flips green when that suspect's dossier was opened.
Interrogate button flips green when that suspect was interrogated.
Per-action tracking, not lumped — players can see at a glance who they've
worked and how.
- Location button: green left-border + "VISITED" pill next to the name when
that location has been examined this session.
- Steven-ready accusation checkbox: removed turnOwner gate from disabled.
Readiness is a consensus flag, not a per-turn action — toggleable any
time so both sides can negotiate accusation regardless of whose turn.
vibe_action declarations (mystery_en + mystery_cn meta.yaml):
- INTERROGATE / EXAMINE_LOCATION / COLLECT_EVIDENCE / READ_DOSSIER /
MAKE_ACCUSATION (GM-bound, count as a turn).
- FINISH_INVESTIGATION_TURN / GET_MYSTERY_STATE / SET_ACCUSATION_READY
(local-only, free actions).
Tests:
- vibeContainerMock.test.ts: Mystery app dispatch routing.
- MysteryApp/constants.test.ts: MYSTERY_GM_ACTIONS vs MYSTERY_APP_ACTION_TYPES
separation (local-only vs GM-bound action sets).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(mystery): CHAT_MESSAGE vibe_action + Steven-side chat composer
Adds a local-only CHAT_MESSAGE action so Kayley can speak into the shared
Investigation Log as a co-detective without consuming her turn. Adds a chat
input below the log on the Steven side that posts the same kind:'chat'
entries with author:'steven' and broadcasts via openroom_user_action so the
agent sees Steven's banter in real time.
Required by the autonomous-playthrough-goal.md spec §TURN FLOW — every turn
the active actor must drop at least one in-character chat line so the
co-detective dialogue stays alive instead of the chat being just structured
action labels and GM narrative.
Changes:
- actions/constants.ts: ACTION_CHAT_MESSAGE + included in MYSTERY_APP_ACTION_TYPES
- types.ts: ChatEntry.kind extends with 'chat'; new ChatAuthor type + author? field
- index.tsx: handleAgentAction CHAT_MESSAGE branch (before unknown guard);
submitHumanChat callback + chatDraft state; chat composer
rendered below the Investigation Log; author prefix in entries
- index.module.scss: .chat_chat style + .chatComposer / input / send button
- mystery_en/meta.yaml + mystery_cn/meta.yaml: CHAT_MESSAGE action declaration
- __tests__/constants.test.ts: assertion that CHAT_MESSAGE is local-only
Tested: vitest constants.test.ts → 3 passed (was 2, +1 for CHAT_MESSAGE).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(mystery): SET_OPENING_PICK + RESET_GAMEOVER autonomous-play unblockers
Adds two local-only vibe_actions discovered as hard blockers during the
autonomous-playthrough run (2026-05-12):
1. SET_OPENING_PICK { leader: 'human'|'agent' }
- The opening-pick screen ("I go first" / "Kayley goes first") was
human-click-only — no vibe_action existed to set turnOwner. Without
this, an autonomous agent couldn't start a game without a live human.
- Errors when called after a game has started (turnOwner != null).
- Mirrors the existing UI buttons: chat entry + openroom_mystery_turn
broadcast on success.
2. RESET_GAMEOVER (no params)
- When the prior MAKE_ACCUSATION normalizer regression left a malformed
gameOver object in React state, the `!gameOver` gate kept blocking
subsequent agent actions even after a fresh GM. RESET_GAMEOVER clears
the React state so investigation can resume in-place.
- Useful for retry-after-fix flows and any future state-corruption
recovery.
Wires both through actions/constants.ts (added to MYSTERY_APP_ACTION_TYPES
since they're local-only — NOT MYSTERY_GM_ACTIONS), index.tsx
handleAgentAction branches (above the unknown-action guard), the en
meta.yaml so they appear in Kayley's vibe_action catalog, and
__tests__/constants.test.ts (3 → 5 passing tests; +2 new cases).
Tested live: SET_OPENING_PICK kicked off the run; RESET_GAMEOVER unblocked
the retry after the verdict-normalizer fix landed; subsequent Priya
accusation tripped gameOver: { correct: true }. WIN condition tripped.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Node 20 reached end of life on 2026-04-30 and gets no fix for the Oct 21 2026 Node.js security release (CVE-2026-111497 Brotli decoder RCE and others). - .github/workflows/ci.yml: setup-node 20 -> 24 - apps/webuiapps/Dockerfile: builder stage node:20-alpine -> node:24-alpine (the final image is nginx; Node only runs at image build time) Verified under Node 24.22.1: pnpm install --frozen-lockfile and pnpm build succeed. The lint script fails with the same 7 unfixable ESLint errors under Node 22 and Node 24, so that failure is unrelated to this change. The Dockerfile change was not build-tested (no Docker daemon in the sandbox). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01X51yD7g9zeQD6rdTypgbxv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brief description of the changes.
Changes
Related Issues
Closes #
Checklist
pnpm run lintpassespnpm buildpassesNeed help on this PR? Tag
@codesmith-botwith what you need.