Skip to content

Optional browser extension: richer recording from everyday Chrome (network, iframes, shadow DOM) #1872

Open

Description

@abrichr

Founder goal: recording is fully automatic, with no flags. The multi-surface work records the user's everyday Chrome through the OS accessibility tree, with no install. Some things accessibility can't see: network requests (needed for API steps, about 3x faster replay), cross-origin iframes, and shadow DOM detail.

Build an optional, install-once Chrome extension (Manifest V3) that sends these to the recorder:

  • a content script for DOM selectors in iframes and shadow roots;
  • chrome.debugger or webRequest for request and response capture of saves;
  • native messaging to the local recorder.
    It must:
  • capture only while a recording is active, with a visible indicator;
  • redact secrets and form fields marked sensitive, as the recorder does;
  • keep data local (no remote endpoints);
  • degrade cleanly: without the extension, recording still works through accessibility.

Keep it backend-agnostic in the engine: the extension is one more evidence source for a web surface, never required. Add real-browser headless tests. Ship it unpacked first; store listing later. Coordinate with the multi-surface PRs (MULTI_SURFACE.md) so its events merge into the same recording format.

Activity

  1. added
    backend-parityA capability that works on one app surface but not another (packages/flow/docs/BACKEND_PARITY.md)
    agent-readyReady for an automated agent to pick up
    claimedAn agent is working on this
    on Nov 4, 2026
  2. abrichr commented on Nov 5, 2026

    @abrichr
    MemberAuthor

    Claimed by Grok

  3. abrichr commented on Nov 5, 2026

    @abrichr
    MemberAuthor

    Draft PR for the first slice: #1977 (Part of #1872).

    Covers the optional MV3 extension scaffold + open shadow-DOM target paths + secret redaction + recording-only gate + extension_evidence.jsonl ingest. Network capture and cross-origin iframes are next. 19 local unit tests passed; real Chrome / native-messaging e2e not run (lab unreachable).

  4. abrichr commented on Nov 6, 2026

    @abrichr
    MemberAuthor

    Network-save slice: #2014 (Part of #1872). It adds webRequest capture of POST/PUT/PATCH/DELETE from tabs while recording, as redacted network_save lines in extension_evidence.jsonl. Redaction is fail-closed and runs in the extension and again on ingest: header values come only from an allowlist, Authorization/Cookie/Set-Cookie/token headers are always withheld, secrets are redacted in the URL and in JSON/form bodies, other bodies are dropped, and a record that still contains a withheld value is dropped. Real headless Chromium with the unpacked extension is covered. Deferred: response bodies, compiling saves into API steps, an installed native host e2e, and the store listing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyReady for an automated agent to pick upbackend-parityA capability that works on one app surface but not another (packages/flow/docs/BACKEND_PARITY.md)claimedAn agent is working on this

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions