Skip to content

ci: run the full CI matrix on merge queue groups - #858

Merged
krisztian-gajdar merged 1 commit into
mainfrom
ci/merge-queue-trigger
Oct 26, 2026
Merged

krisztian-gajdar merged 1 commit into
mainfrom
ci/merge-queue-trigger

Conversation

@krisztian-gajdar

@krisztian-gajdar krisztian-gajdar commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Problem

CI / Required is computed when a pull request's CI runs, against the main of that moment. Branches are not required to be up to date, so a pull request can merge days later on that old result. Three of the last 20 CI push runs on main failed this way:

Main commit Pull request CI / Required passed on the PR head Merged Failing lanes on main
51af6a4 #824 2026-10-19 21:22 UTC 2026-10-26 10:26 UTC Bundle resolution, Python, CPU containers
89f5cc5 #819 2026-10-19 13:29 UTC 2026-10-21 09:21 UTC Rust
b94c193 #809 2026-10-19 11:41 UTC 2026-10-21 11:45 UTC Rust

#824's lock change predates the bundle-lock binding check from #802; #809 and #819 predate the RUSTSEC-2026-0382/0383 advisories.

Requiring branches to be up to date would close the gap, but dismiss_stale_reviews_on_push is on, so every forced update would dismiss the approval and need a new review. A merge queue tests the pull request merged onto the current main (and onto the entries ahead of it) without pushing to the pull request branch.

Change

  • ci.yml gains a merge_group: trigger. Nothing else in the workflow needs to change for it:
    • no job behind CI / Required has an if:, and CI / Required keeps if: always();
    • no required job, and neither reusable workflow they call (release-python.yml, release-npm.yml), reads github.event.pull_request, github.head_ref or github.base_ref;
    • the distribution builds pass and check out github.sha, which on merge_group is the merge group commit, so release_guard.py build compares the checkout with the commit under test;
    • the concurrency group falls back to github.ref, which is unique per queue entry (gh-readonly-queue/main/pr-<n>-<sha>), and only pull_request runs cancel in progress, so a queue run is never cancelled by another run.
  • test_required_check_reports_on_merge_queue_groups (run by CI / Policy) pins the trigger set, the concurrency settings, and the absence of pull-request-only context in the required jobs and the reusable workflows they call. Each of these mutations fails it: dropping the trigger, cancel-in-progress: true, checking out github.event.pull_request.head.sha in release-python.yml, and passing it as source_ref.

No other workflow has an unfiltered push trigger, so queue branches start no extra push runs.

Full job set on merge_group

I measured whether a lighter job set would be safe on merge_group. Over the last 300 CI push runs on main (2026-10-02 to 2026-10-26), 20 failed. The failing lanes were Python (7), Rust (4), Fresh bootstrap (3), TypeScript (3), Helm (2), Policy (2), and one each for Bundle resolution, Contracts, CPU containers, Live SDK CPU and Python distributions. Every lane except npm distributions has broken main at least once.

A lighter set would also not be faster. CI / Python is the critical path on its own (median 19 min on push and 37 min on pull requests, p90 35 and 46 min), and it is also the lane that failed most often. The next longest, CPU containers, has a median of 14 to 15 minutes. So the queue runs the full set.

Enabling the queue

This PR only adds the trigger and does not change behaviour by itself. A maintainer adds the merge queue rule to the main ruleset separately. After that, gh pr merge adds a pull request to the queue instead of merging it immediately.

Validation

  • pytest -q tools/ci/tests/test_required_ci.py tools/ci/tests/test_public_tree.py tools/ci/tests/test_release_contract.py: 228 passed
  • python tools/ci/check_public_tree.py: passed
  • actionlint (1.7.7, with the policy job's ignore): clean
  • ruff format --check tools/ci and ruff check --select E,F,I,UP,B tools/ci: clean

Summary by CodeRabbit

  • New Features
    • CI checks now run for merge queue events, in addition to pull requests and pushes to the main branch.
  • Tests
    • Added checks to verify required CI behavior across pull requests, merge queues, and pushes.

Add a merge_group trigger to ci.yml so a merge queue can test each pull
request against the current main before it merges. Every job behind
CI / Required runs unchanged on merge_group: no job is conditional on the
event, none reads pull-request-only context, the distribution builds check
out github.sha (the merge group commit), and only pull_request runs cancel
in progress, so queue runs are never cancelled by a newer run.

A policy test pins the trigger, the concurrency settings, and the absence
of pull-request-only context in the required jobs and the reusable
workflows they call.
@krisztian-gajdar
krisztian-gajdar requested a review from a team as a code owner October 26, 2026 20:04
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Team
  • Run ID: ad72e9de-1b0a-493d-b508-0b835867d0e6
Commits

Reviewing files that changed from the base of the PR and between d278019 and a1cefcc.

Files selected for processing: 2
  • .github/workflows/ci.yml
  • tools/ci/tests/test_required_ci.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

Walkthrough

The CI workflow now runs for GitHub merge_group events, in addition to pull requests and pushes to main. A test checks the workflow triggers, concurrency settings, required job name, and mandatory-job requirements.

Changes: Merge queue CI

Layer / File(s) Summary
Merge queue trigger and validation
.github/workflows/ci.yml, tools/ci/tests/test_required_ci.py
The workflow adds the merge_group trigger. The test checks workflow triggers, concurrency settings, the required job name, and restrictions for mandatory and reusable jobs.

Suggested reviewers: svonava

Priority: Normal

Merge Risk: Minimal · up to a1cef

The merge queue trigger is consistent with the existing required CI workflow and presents no actionable merge-blocking risk.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (1 skipped: 1 … Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: running the full CI matrix for GitHub merge queue groups.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@krisztian-gajdar
krisztian-gajdar added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 7a86c3a Oct 26, 2026
56 of 58 checks passed
@krisztian-gajdar
krisztian-gajdar deleted the ci/merge-queue-trigger branch October 26, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant