Skip to content

ci(previews): deploy a live Studio preview for each PR - #2054

Draft
dawsontoth wants to merge 6 commits into
stagefrom
claude/studio-pr-previews
Draft

dawsontoth wants to merge 6 commits into
stagefrom
claude/studio-pr-previews

Conversation

@dawsontoth

@dawsontoth dawsontoth commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor
A fox painter hoists a fresh canvas into its own treehouse for each acorn letter a friend posts, while an owl lowers an empty frame from a treehouse whose letter was torn up

⊙ Problem

A Studio PR has no running build to try: reviewers check out the branch, or wait for dev. Harper 5.4 hosts a preview per PR (HarperFast/harper#3042), and Harper's own repos should dogfood it with HarperFast/deploy-action, as HarperFast/documentation#797 does for the docs site.

🤔 Previews live on the central manager's own cluster

That is Dawson's plan: only the hdbms front end gets previews, at https://<branch>.app.dev.studio.harperfabric.com/, and they call the one dev CM. Each preview is a worker thread in CM's Harper process, not a sandbox, and the CM answers CORS only for origins it lists (http://localhost:5173 today).

💡 Solution

🤔 Static files only, not the deploy template

The template is a fastify route plus @fastify/static: server code the PR controls. On CM's cluster that code could read CM's deployed .env (its AWS keys) and reach the cluster's CA through server.operation, so the preview runs none; preview.yaml writes the config. The cost: template changes aren't previewed, and previews lack the template's frame guard and cache headers.

🤔 Trusted users, shared data

A preview's JavaScript runs with its visitor's dev CM session, so the HARPER_PREVIEW_AUTHORS variable lists who gets one; unset, nobody does. Previews share data because branching forks every database but system, CM's included. OAuth sign-in lands on dev Studio (AGENTS.md); use email and password on a preview.

🤔 Things to look at

The gate, polling, conflicts, permissions, and what isn't here
  • The gate reads the PR's author and the event's actor, so neither an unlisted user's push to a listed user's PR nor a listed user's push to an unlisted user's PR deploys anything. Cleanup checks the author only, on its step rather than its job, because any other term in the job's if: makes harper preview setup unable to verify that cleanup runs on close. An author dropped from the list after deploying leaves a preview that expires on its own.
  • Polling. GitHub starts both workflows at once, and the trust policy matches only pull_request_target, so workflow_run can't deploy. The deploy job polls for the build of its exact commit, by a run title naming the PR. Unlike documentation#797, another PR's build of the same commit can't be picked, and a failed API call isn't read as "no build started". It deploys nothing if the PR moves on or closes, and fails if the build fails, takes over 30 minutes, or never starts.
  • A PR with merge conflicts gets no preview: GitHub starts no pull_request run for it, so the deploy job fails after 10 minutes and says why.
  • Only PRs into stage get one. Both workflows filter on it, the default branch the trust policy pins; PRs into dev or prod run neither.
  • actions: read on the deploy job, beyond the example, reads the build's runs and downloads its artifact.
  • No sweep workflow. The cluster expires a preview 7 days after its last deploy, which covers a missed close. A sweep needs its own trust policy; add one if previews pile up.
  • Direct connections to clusters (Fabric Connect direct mode) probably fail from a preview, since those instances' CORS lists hold Fabric's own Studio origins. Not verified; proxied connections go through the CM.

⚖️ Alternatives

Four weighed
  • One job that builds and deploys: it would hold the preview identity while running the PR's install and build, and the trust policy accepts only pull_request_target.
  • A workflow_run deploy instead of polling: the trust policy pins event_name: pull_request_target, so that run could not exchange a token.
  • Required reviewers on preview instead of a list: every push would wait for an approval.
  • A deploy by reference: it would run the PR's own config.yaml, so a PR could load server code into CM's process.

📋 Before this merges (Dawson)

None of this has been done. This PR changes no cluster or repository settings.
  1. Harper 5.4 on the CM cluster. Previews need 5.4.0-alpha.2 or later (alpha.1 predates enable_previews and harper preview setup). The CM-previews session's notes say dev and stage run harper-pro 5.3.2; I couldn't check (no credentials, and nothing public reports it).

  2. Make previews reachable. A preview's worker binds only a UDS mirror of the secure port, so the cluster needs http.securePort and tls.unixDomainSockets, or deploys are refused. *.app.dev.studio.harperfabric.com needs DNS (none today), a certificate, and routing to the preview's socket: the shared ports answer 503 for a preview host. threads.maxIsolated (default 8) caps live previews.

  3. Let previews call the CM. Add https://*.app.dev.studio.harperfabric.com to the cluster's http.corsAccessList, keeping http://localhost:5173. Wildcard entries need Harper 5.4.0-alpha.1 or later; 5.3 matches exact origins only. The entry trusts every app previewed on that cluster, so keep its previews to Studio.

  4. harper preview setup app=hdbms dataMode=shared, signed in as a super_user, from a checkout of this branch (setup reads .github/workflows/preview.yaml and checks it). It creates the hdbms-ci-preview role and user and both trust policies, enables previews for hdbms, and sets the HARPER_PREVIEW_TARGET repository variable. Without dataMode=shared, the cluster refuses the workflow's branched_databases: false.

  5. The HARPER_PREVIEW_AUTHORS repository variable: a JSON array of the GitHub logins that get previews, such as ["dawsontoth"]. Until it's set, nobody gets one.

  6. Environments (Settings > Environments): create preview and preview-cleanup (only e2e-dev exists) and allow stage under Deployment branches and tags on both. No required reviewers: the list is the gate, and reviewers would hold every deploy.

  7. An Actions policy allowing pull_request_target for .github/workflows/preview.yaml. Studio is public, and GitHub blocks the event in public repositories from December 2. The repository has no policies yet. From the action's README:

    curl --fail --silent --show-error --location https://raw.githubusercontent.com/HarperFast/deploy-action/main/examples/preview-actions-policy.json | jq '{name, enforcement, conditions, rules}' | gh api --method POST repos/HarperFast/studio/actions/policies --header 'X-GitHub-Api-Version: 2026-03-10' --input -

🔧 Changes

✅ Verification

  • Harper's own check: checkPreviewWorkflowFile from harper origin/main (907ff6bb1), for branch stage: no problems, nothing unverified, nothing unsafe. A PR-head checkout, a missing fork guard on either job, and branch main are each refused. A list term on the cleanup job's if: is reported unverified, hence the step gate.
  • The bundle (--mode dev, no VITE_TELEMETRY_ENABLED) calls dev.studio.harperfabric.com and holds no Datadog token, GTM loader or Reo script: 30 MB without sourcemaps.
  • The dev CM's CORS: its preflight allows http://localhost:5173 with credentials and answers nothing for https://x.app.dev.studio.harperfabric.com.
  • Gate: vitest (618 files), tsc -b, oxlint, dprint, commitlint. Removing any of the new test's checks, or the cleanup job's fork guard, turns it red; so does adding a checkout.
  • The wait step, against verify-pr.yaml runs: fix(auth): verify email with the sign-up browser proof or the password #2050's head found its run and wrote run-id; a commit the PR had left, and merged feat(previews): turn on and manage PR previews from Studio #2045, exited 0 with a notice; another title matched nothing; a failing listing ran to the time limit.
  • On GitHub: Verify PR's actionlint passed, and before the list gate this PR's own Build PR Preview #2054 run uploaded pr-2054-build (12 MB). It now skips, with the list unset.
  • Not verified: a real deploy or cleanup, which needs steps 1 to 7.
Review findings left open
  • cleanup shares neither the deploy's concurrency group nor its by_ref. deploy-action re-reads the PR after deploying and drops its own preview if the PR closed (preview.ts), drops by app and PR number only (dropClosedPreview), and its README says to keep cleanup out of the deploy's group.
  • A forged run title. A writer could build a base branch whose build claims another PR's title at that PR's commit, and have it deployed as that preview. They can already push to that PR's branch, and the identity deploys only hdbms-pr-* previews.
  • The guard test doesn't catch a run: step that executes the artifact.

🤖 Generated with Claude Code

Related PRs: 1 independent
Complexity: medium

Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=5; full=2 @ 5813c0c

Review-Attention: deep ~40m (sensitive: preview-build.yaml, preview.yaml; decisions: do-less-alternative, preview-in-cm-process, cleanup-gated-by-author-only, allowlist-repo-variable; raised: degraded review) @ 5813c0c

dawsontoth and others added 3 commits November 4, 2026 11:10
Each PR into stage from a branch of this repository gets a live Studio at
https://<branch>.app.<cluster domain>/, dropped when the PR closes.

preview-build.yaml builds the PR's head commit in dev mode, without
telemetry, in an unprivileged pull_request run and uploads web/.
preview.yaml (pull_request_target, from stage) waits for that build,
writes its own static config.yaml and deploys the payload as
hdbms-pr-<N> with HarperFast/deploy-action v1.0.0 over OIDC. Fork PRs
get no preview, and nothing in it checks out PR code.

workflowPrivilege.test.ts now checks that every pull_request_target job
has the fork guard and checks out nothing, since the cluster's trust
policy pins the file's path, not its content.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
- The wait step also matches the build run's head branch, so another
  PR's run at the same commit is never picked.
- The guard test requires the fork guard as a top-level && term, with
  string literals stripped and no ||, so a negated, quoted or bypassed
  guard fails it.
- deploying.md says previews need the dev CM's CORS entry, sign in with
  email and password, and serve Harper's plain 404.
- Workflow comments keep only the constraints.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
- preview-build.yaml names each run "Build PR Preview #<N>", and the
  wait step matches that title, so another PR's run of the same commit,
  even from the same branch, is never picked.
- A failed runs listing no longer counts as "no build started" after
  ten minutes; the 30-minute limit still applies.
- The fork-guard test refuses any condition with parentheses, braces or
  ||, so a guard inside a negated group or between two interpolations
  fails it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds documentation for PR previews in .github/deploying.md and introduces a test suite in src/lib/workflowPrivilege.test.ts to ensure that pull_request_target workflows are properly guarded and do not execute untrusted code. The feedback recommends enhancing the robustness of the tests by normalizing whitespace when verifying the fork guard condition and adding defensive checks to handle malformed workflow files without crashing.

Comment thread src/lib/workflowPrivilege.test.ts Resolved
Comment thread src/lib/workflowPrivilege.test.ts Resolved
Previews will live on the central manager's own cluster, where a
preview is a worker thread in CM's Harper process.

- Both workflows run only when a login in the HARPER_PREVIEW_AUTHORS
  repository variable (a JSON array) opened the PR and caused the
  event. Unset, nobody gets a preview: its JavaScript runs with the
  visitor's dev CM session.
- The deploy passes branched_databases: false. Branching would fork
  every database on the node but system, CM's included; the payload
  runs no server code, so sharing touches nothing.
- The fork-guard test splits the condition at parenthesis depth 0, so
  a function call such as contains(fromJSON(...)) passes while a
  guard inside a call, a negated group or a top-level || still fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 82.62% 17291 / 20928
🔵 Statements 82.51% 18353 / 22242
🔵 Functions 78.11% 4732 / 6058
🔵 Branches 77.74% 13957 / 17952
File CoverageNo changed files found.
Generated in workflow #2509 for commit 5813c0c by the Vitest Coverage Report Action

dawsontoth and others added 2 commits November 4, 2026 12:00
…shared

- The cleanup step runs only when HARPER_PREVIEW_TARGET is set and the
  PR's author is listed, so an unset variable doesn't fail every merge.
  The gate sits on the step: any other term in the job's if: makes
  harper preview setup unable to verify that cleanup runs on close.
- The setup command in the header and deploying.md now says
  dataMode=shared, which branched_databases: false needs.
- The fork-guard test also refuses index expressions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…est can't read

The fork-guard check now compares with whitespace collapsed, so a guard
split across lines still counts. A jobs, job, steps or step value of the
wrong type is reported as a problem rather than crashing the test or
being skipped, so it fails instead of passing unchecked.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant