Skip to content

Add the Metrics API guide - #819

Draft
vorporeal wants to merge 3 commits into
mainfrom
david/metrics-api-docs
Draft

vorporeal wants to merge 3 commits into
mainfrom
david/metrics-api-docs

Conversation

@vorporeal

@vorporeal vorporeal commented Oct 2, 2026 •

Copy link
Copy Markdown

What this feature does

The Metrics API gives customers the numbers the factory dashboard shows, over any date range and bucketed by day, week, or month, plus one record per run and one per pull request, so they can build their own reports and keep their own copy. It ships in the Tuesday 2026-10-06 release. This PR stays in draft until then.

Summary

Adds a Metrics API guide at src/content/docs/factories/metrics-api.mdx (/factories/metrics-api/), written from the signed-off product contract. It covers what the API lets a customer do and what is not included, the behavior every endpoint shares (authentication, UTC dates and their defaults, group_by_period and the 365-bucket cap, as_of, team_uid and factory_uid, dollars), the eight endpoints in the contract's order with their filters, response fields, and one curl example each, the Definitions section nearly verbatim, access and limits, and a short section on syncing a local copy.

Hold in draft. The endpoints are not yet in the generated OpenAPI reference in this repo (developers/agent-api-openapi.yaml), which syncs from warp-server releases. Keep this PR in draft until that file shows them, then re-check every name in the guide (paths, filters, fields, defaults, limits) against it before marking the PR ready.

Placement: the page sits in the Factories tab under "Management & observability", next to "Factory dashboard", because six of the eight endpoints are /factory/{uid}/... routes and the repo already documents factory endpoints there (factories/factory-api.mdx) rather than under reference/. The two record endpoints are documented on the same page since they share the dashboard's definitions.

No JSON response bodies are shown. The contract names every field but does not fix the JSON nesting (for example where series and period_list sit relative to each metric), and the closest sibling pages (factories/factory-api.mdx, the Agent API quickstart) show requests only. Response examples can be added from the OpenAPI schemas once they land.

Changes

src/content/docs/factories/metrics-api.mdx

  • New reference page: capabilities and exclusions, shared behavior, six aggregate endpoints, two record endpoints, definitions, access and limits, syncing, related pages.

src/sidebar.ts

  • Added factories/metrics-api ("Metrics API") to Factories > Management & observability, after Factory dashboard.

src/content/docs/factories/factory-api.mdx, factory-dashboard.mdx, measure-and-improve.mdx

  • One cross-link each to the new page in their closing link lists.

Related issues

None linked. Source is the product contract for the 2026-10-06 metrics API release in warp-server (branch david/spec-out-ga-launch-metrics-apis, 9eb3edc4f395).

Validation

  • npm run build (Node 24 via fnm): passed; /factories/metrics-api/ rendered with all sections and the sidebar entry.
  • npm run typecheck: passed, 0 errors (7 pre-existing hints in unrelated components).
  • python3 .agents/skills/check_for_broken_links/check_links.py --internal-only: 0 broken links across 4,323 internal links.
  • python3 .agents/skills/style_lint/style_lint.py --changed: 0 errors; 18 unrecognized-term warnings on the new page, all on bold lead terms in * **Term** - ... list items (the same class fires on the untouched dashboard page).
  • npm run lint (Trunk): not run, Trunk is not installed locally. [email protected] with default rules on the new page reports only MD013/line-length, which the existing factory pages also trip.
  • Identifier cross-check: every backticked name in the contract appears on the page; the page adds only WARP_API_KEY, YOUR_FACTORY_UID, the base URL, and page_info.next_cursor.

Content design plan

Audience and JTBD: A platform or analytics engineer on a team running a Warp factory who has been asked to put factory productivity and spend into the team's own dashboard or warehouse, and who already has a Warp API key from using the Agent API.

Problem: The factory dashboard shows the numbers but offers no way to export them, compare factories, or break them down by user or agent. Without this page the reader has to guess at endpoint paths, filter names, and what each number counts, and cannot reproduce the dashboard's figures from the records.

Goals:

  • The reader can call each endpoint with the right filters and know what every response field means.
  • The reader can reproduce a dashboard number (for example average human interactions per PR, or autonomy) exactly from the records, using the definitions.
  • The reader can keep an incremental local copy of runs and pull requests and join the two.

Purpose and value: No existing page documents these endpoints; the generated OpenAPI reference will carry schemas but not the definitions (what counts as a human interaction, which period a merged PR lands in, how cost is attributed across a run tree) that make the numbers interpretable. Without this page, customers would read the dashboard and the API differently.

Content type: Reference — the reader is looking up endpoints, filters, and field meanings mid-task, not learning a concept or following a procedure.

Skill and template: draft_reference / .agents/templates/reference.md

High-impact scenarios:

  • Covers: rebuilding a dashboard chart over a custom range; spend by user, agent, source, or model; listing a factory's PRs with cycle time and human-interaction counts; incremental sync of runs and PRs.
  • Excludes: JSON response bodies — the contract does not fix the nesting, and the OpenAPI schemas will. SDK examples — the SDK methods for these endpoints do not exist yet. Cross-factory rollups — not provided by the API; the page says to query each factory and combine.

Unverified claims

  • Date and time format for start_date, end_date, and the *_after / *_before filters — metrics-api.mdx, every example. The contract says only that dates are UTC; the examples use RFC 3339 timestamps (2026-09-01T00:00:00Z), matching the existing created_after parameter on GET /agent/runs. Confirm against the OpenAPI parameter schemas.
  • page_info.next_cursor on the run list — "Syncing your own copy". Taken from the current ListRunsResponse in developers/agent-api-openapi.yaml. The pull request list's cursor field is not named in the contract and is not named on the page; confirm and add it.
  • Response nesting — the page lists fields but shows no JSON. Confirm as_of, team_uid, factory_uid, period_list, and per-metric series placement against the schemas before adding examples.
  • "the web app" in the Human interaction definition — Definitions. Kept as the contract's wording; confirm which web app is meant before replacing with a product name.
  • "Every plan that includes Warp Factories" — Access and limits. Paraphrases the contract's "every plan that has Factory access".
  • Pull request updated_at does not move on follow-up messages alone — "Syncing your own copy", step 2. Added at the owner's request after the contract was signed off; this sentence is not in the product contract text. Confirm against the implementation.
  • X-Warp-Team-Uid header on the run list example — Records > Runs. Per the contract and the existing header parameter in the spec; confirm it is required for the metrics use case or only for keys that span teams.
  • All endpoint paths, filter names, field names, defaults, and limits — from the product contract. None are in developers/agent-api-openapi.yaml yet; re-check them when the release syncs the spec.

Documentation risk

Risk: engineering-review-required
Rationale: New page documenting eight API endpoints that are not yet in the generated OpenAPI reference; every path, filter, field, default, and limit must be re-checked against the spec once the 2026-10-06 release syncs it.
Source files consulted: warp-server:david/spec-out-ga-launch-metrics-apis (product contract)@9eb3edc4f395, developers/agent-api-openapi.yaml@dbaf7ae2d45f
Engineering review status: pending
Docs override: none

Follow-ups

  • After the spec syncs (contract now at 9eb3edc4f395): re-check every name in the guide against developers/agent-api-openapi.yaml, add trimmed JSON response examples from the schemas, name the pull request list's cursor field, and confirm the date format in the examples.
  • Add SDK examples once the Python and TypeScript SDKs expose these methods.

Co-Authored-By: Warp [email protected]

Documents the eight public metrics endpoints: six factory-scoped aggregates
plus the run list and the pull request list. Written from the signed-off
product contract; endpoint paths, filter and field names, defaults, limits,
and definitions follow it.

Adds the page to the Factories sidebar next to the factory dashboard and
links to it from the factory API, factory dashboard, and measure-and-improve
pages.

Co-Authored-By: Warp <[email protected]>
@vercel

vercel Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 2, 2026 10:27pm UTC

Request Review

@cla-bot cla-bot Bot added the cla-signed label Oct 2, 2026
@vorporeal vorporeal added the warpy-factory Opened by the Warp factory agents label Oct 2, 2026
vorporeal and others added 2 commits October 2, 2026 18:20
Cost definition: for now dollars use the current per-credit price, moving to
the price at time of use once per-request price history exists. Access and
limits: a request that runs too long is cut off with an error. Syncing: a
PR's updated_at does not move on follow-up messages alone, so re-pull recent
PRs when follow_up_message counts matter.

Co-Authored-By: Warp <[email protected]>
Dropped from the contract in review; status_message.error_code is the
stable error code.

Co-Authored-By: Warp <[email protected]>

This branch was successfully deployed

1 active deployment
Preview — 7e59eccc Deployed Oct 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed warpy-factory Opened by the Warp factory agents

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant