[lewisjared]

Adds a performance suite for the API in backend/tests/perf. It requests 20 common endpoints against the production-shaped synthetic database from climate-ref (Climate-REF/climate-ref#1009). Each endpoint is an Endpoint(id, path, params) entry in ENDPOINTS, so adding one is a single line.

• Each endpoint is timed with pytest-benchmark. It fails when one of its queries reads a whole large table that isn't in its allowed_scans.
• make perf-compare checks out the base ref in a temporary worktree and runs this branch's tests against both. It fails when an endpoint's median is over 30% slower. Running the same code twice drifts up to about 22%. A 10 ms sleep added to datasets.get failed it at +679%.
• A new CI job backend-perf runs the compare on every PR.
• Fixes a 500 from GET /diagnostics/{provider}/{slug} when the provider registry doesn't know the diagnostic. The list endpoints already fell back to an empty description, and all three lookups now share one helper.

7 endpoints already read a whole large table, so their allowed_scans record today's behaviour rather than what they should read. Worth a look: diagnostics/?mip_era=CMIP6 scans execution_group while the unfiltered list doesn't, and executions/statistics is the slowest at about 240 ms at a tenth of production size.

Blocked on a climate-ref release with #1009. Until then the perf tests skip, so the CI job passes without checking anything. Locally against that branch, all 20 endpoints pass and make test-perf takes about 15 s including the 9 s database build.

Summary by CodeRabbit

• Bug Fixes
  • Fixed an error that could occur when requesting a diagnostic whose provider is unavailable. These requests now return successfully with the available diagnostic information.
• Tests
  • Added API performance checks using a production-shaped synthetic database. Pull requests now compare performance results with the base branch and flag significant slowdowns or unexpected full-table scans.

[coderabbitai]

📝 Walkthrough

The pull request adds synthetic-database API performance tests and pull-request comparisons against a base revision. It also changes diagnostic summary construction to handle diagnostics whose providers are not registered.

Changes: API performance checks

Layer / File(s)	Summary
Synthetic database endpoint benchmarks backend/tests/perf/conftest.py, backend/tests/perf/test_endpoints.py, backend/tests/perf/README.md, backend/pyproject.toml	Session fixtures configure a cached synthetic database and test client. Parameterised tests benchmark API endpoints, check responses, and detect unapproved full-table scans. The README documents the tests and the development dependencies include pytest-benchmark.
Base-branch comparison and CI wiring backend/scripts/perf-compare.sh, backend/Makefile, .github/workflows/ci.yml, backend/README.md, changelog/145.trivial.md	The comparison script runs the tests against the base and current revisions, then compares benchmark results. Make targets and a pull-request CI job invoke it. Documentation and the changelog describe the performance checks.

Unregistered diagnostic providers

Layer / File(s)	Summary
Diagnostic lookup and fallback response backend/src/ref_backend/models/diagnostics.py, backend/tests/test_api/test_routes/test_diagnostics.py, changelog/145.fix.md	Diagnostic summary builders use a shared registry lookup and fallback values when a diagnostic is not registered. A route test checks that requesting an existing diagnostic with an empty provider registry returns HTTP 200.

Sequence Diagram(s)

sequenceDiagram
  participant CI
  participant PerfCompareScript
  participant DetachedBaseWorktree
  participant PerformanceTests
  participant BenchmarkStorage
  CI->>PerfCompareScript: Runs comparison for the pull request
  PerfCompareScript->>DetachedBaseWorktree: Creates worktree at the selected base revision
  PerfCompareScript->>PerformanceTests: Runs tests against base and current revisions
  PerformanceTests->>BenchmarkStorage: Stores benchmark results
  PerfCompareScript->>BenchmarkStorage: Reads results for comparison

Loading

Possibly related PRs

• Climate-REF/ref-app#146: Updates the locked climate-ref version and adds a migration, affecting the dependency and database setup used by the performance tests.

Priority: ➖ Normal

Change: Other · Severity of issue fixed: Low

Merge Risk: 🟡 Moderate · up to c17f7

The new performance regression check can pass without comparing some endpoints when their baseline run fails, which weakens its protection. It can also report a false change if the base branch moves during the job. Neither affects production behaviour, but both should be tightened before relying on the CI gate.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 52.94% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 5 files. (7 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: adding API performance tests against a synthetic database.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 52.94% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 5 files. (7 skipped: 7 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

---

ℹ️ Review info

⚙️ Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 6a4a2500-8e49-41c8-8b6c-532dfcd16093

📥 Commits

Reviewing files that changed from the base of the PR and between 877a9cc and c17f77c.

⛔ Files ignored due to path filters: 1

• backend/uv.lock is excluded by !**/*.lock

📒 Files selected for processing: 12

• .github/workflows/ci.yml
• backend/Makefile
• backend/README.md
• backend/pyproject.toml
• backend/scripts/perf-compare.sh
• backend/src/ref_backend/models/diagnostics.py
• backend/tests/perf/README.md
• backend/tests/perf/conftest.py
• backend/tests/perf/test_endpoints.py
• backend/tests/test_api/test_routes/test_diagnostics.py
• changelog/145.fix.md
• changelog/145.trivial.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.