Skip to content

ci: build E2E once on Depot and test while shards set up - #988

Merged
snario merged 1 commit into
mainfrom
claude/e2e-ci-under-5-min
Oct 14, 2026
Merged

snario merged 1 commit into
mainfrom
claude/e2e-ci-under-5-min

Conversation

@snario

@snario snario commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #986 to get the E2E check under 5 minutes. After #986 each E2E shard still rebuilt the site on a GitHub ARM runner (~5 min) before ~85s of Playwright tests.

  • E2E Build (new, Depot 8-core) compiles the site once. The build is mostly single-threaded plugin work; Depot's cores ran it in 111–172s vs 294–316s on GitHub ARM in a probe run. It runs inside the shards' Playwright container so absolute paths baked into the server output (rootDir) match where shards serve it.
  • E2E Tests (1/3)–(3/3) keep their required names and stay on GitHub-hosted runners. They start immediately, install dependencies while the build runs, then wait for and download the e2e-dist artifact (scripts/wait-for-e2e-build.mjs).
    • Why not run them on Depot: Moderato returned 429 too many connections from this IP to the browser from the first request on Depot, whose egress IPs are shared (Tempo's own CI runs there too). The two passkey + faucet live tests failed on every Depot attempt and pass on GitHub runners. fix(ci): authenticate live E2E RPC requests #984's authenticated RPC would fix this once TEMPO_RPC_API_KEY is configured — it's currently empty in CI.
  • Sitemap builds on Depot (was ~7.7 min on ubuntu-latest).
  • Set the DEPOT_DISABLED repository variable to true to fall back to GitHub runners.
  • pnpm store cache gets a restore-keys fallback so a lockfile change reuses unchanged packages instead of a cold install.

Fewer browser page loads, same coverage

document-semantics made 280 full browser loads (42% of all E2E test time) to check one title/H1/description/og:title/canonical per route. It now:

  • checks every route's prerendered HTML in one concurrent request sweep (all 278 routes, a few seconds), and
  • hydrates each docs section's landing and deepest page in the browser (38 routes) to catch tags added on the client.

Mutation-checked: injecting a second <h1> into /docs/accounts fails the sweep with /docs/accounts: 2 h1.

Expected timeline

E2E Build ≈ setup 25s + build ~2 min + upload ~10s; shards overlap their ~50s setup with it, then download (~15s) and test (~50s). Target: E2E checks complete in ~4 min. This PR's own CI run will confirm.

Co-Authored-By: Liam Horne [email protected]

🤖 Generated with Claude Code

Prompted by: Liam

E2E shards each rebuilt the site on GitHub ARM runners (~5 minutes)
before running ~85s of Playwright tests.

- E2E Build compiles the site once on a Depot runner, about twice as
  fast for this single-threaded build, inside the shards' Playwright
  container so absolute paths in the server output still match.
- The three required E2E shards stay on GitHub-hosted runners, install
  dependencies in parallel with the build, then wait for and download
  its artifact. Moderato limits RPC connections per IP, and Depot's
  shared egress IPs exhaust it for the live-transaction tests.
- Sitemap builds on Depot too. Set DEPOT_DISABLED to fall back.
- pnpm store cache falls back to the previous store on lockfile changes.
- document-semantics checks every route's prerendered head and H1 in
  one request sweep and hydrates each section's landing and deepest
  page in the browser: 282 page loads become 40 tests (~42% of E2E
  test time).

Co-Authored-By: Liam Horne <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@snario snario self-assigned this Oct 14, 2026
@vercel

vercel Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
tempo-docs Ready Ready Preview Oct 14, 2026 6:02am UTC

Request Review

@snario
snario merged commit 505d465 into main Oct 14, 2026
15 checks passed
@snario
snario deleted the claude/e2e-ci-under-5-min branch October 14, 2026 06:04

This branch was successfully deployed

1 active deployment
Preview — 6c4b5688 Deployed Oct 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant