Skip to content

chore: Add allowlisted /override-check command - #5331

Draft
vvaliyev wants to merge 1 commit into
mainfrom
ci/override-required-check
Draft

vvaliyev wants to merge 1 commit into
mainfrom
ci/override-required-check

Conversation

@vvaliyev

Copy link
Copy Markdown
Member

Description

Adds the "Override required check" workflow: a maintainer comments /override-check <check> <reason> on a PR to mark one allowlisted required check as passed for the PR head commit (also runnable from the Actions tab).

Allowlist, fixed in the workflow:

  • visual-regression: posts the existing visual-regression-override status and re-runs the failed deploy jobs, same as today.
  • codecov: posts success to codecov/patch and codecov/project directly. Known gap: a later Codecov upload for the same commit overwrites the override, so the command has to be posted again.

Any other check name is refused and nothing is posted. The override applies to one commit only; a new push needs a new override.

/override-visual-regression <reason> stays as a transition alias for /override-check visual-regression <reason>, to be removed later.

Replaces .github/workflows/visual-regression-override.yml. Adds issues: write so the workflow can reply when a command is refused.

Related links, issue #, if available: n/a

How has this been tested?

YAML parsed; the comment parsing and 140-character description cap tested locally in bash. Not run on GitHub yet: needs a real PR comment after merge (issue_comment workflows run from the default branch).

Review checklist

The following items are to be evaluated by the author(s) and the reviewer(s).

Correctness

  • Changes include appropriate documentation updates.
  • Changes are backward-compatible if not indicated, see CONTRIBUTING.md.
  • Changes do not include unsupported browser features, see CONTRIBUTING.md.
  • Changes were manually tested for accessibility, see accessibility guidelines.

Security

Testing

  • Changes are covered with new/existing unit tests?
  • Changes are covered with new/existing integration tests?

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@vvaliyev

Copy link
Copy Markdown
Member Author

Replaces #5330. Includes the Copilot fix from that PR: cancel-in-progress is false, so overrides for different checks on the same PR queue instead of canceling each other.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow can silently replace queued override requests and misleadingly report that no changes were applied after partial success.

2 open issues
What changed in this PR

Generalizes the visual-regression override workflow into a maintainer-only command for overriding allowlisted checks on a single commit.

Changes:

  • Adds /override-check and manual dispatch for visual-regression and Codecov overrides.
  • Preserves /override-visual-regression as a transition alias and adds failure replies.
  • Updates documentation and deploy error messages to use the new command.
File Description
docs/​RUNNING_TESTS.md Documents the new command, allowlist, and Codecov overwrite limitation.
.github/​workflows/​visual-regression.yml Updates the workflow path in an explanatory comment.
.github/​workflows/​visual-regression-override.yml Removes the replaced visual-only override workflow.
.github/​workflows/​override-check.yml Adds authorization, command parsing, allowlisted status overrides, and visual-check reruns.
.github/​workflows/​deploy.yml Updates failure messages to recommend the new command.

Review effort: Balanced

Comment thread .github/workflows/override-check.yml Resolved
Comment thread .github/workflows/override-check.yml Resolved
@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.86%. Comparing base (3651909) to head (b909b9c).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5331   +/-   ##
=======================================
  Coverage   97.86%   97.86%           
=======================================
  Files        1016     1016           
  Lines       32890    32890           
  Branches    12200    12200           
=======================================
  Hits        32189    32189           
  Misses        661      661           
  Partials       40       40           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Add the "Override required check" workflow
(.github/workflows/override-check.yml). A maintainer comments
`/override-check <check> <reason>` on a PR, or runs the workflow
manually with a check, commit SHA and reason, to mark one allowlisted
required check as passed for the PR head commit.

The allowlist is fixed in the workflow:
- visual-regression: posts a success status to
  visual-regression-override and re-runs the failed jobs of deploy.yml
  so the "Visual regression result" gate re-evaluates.
- codecov: posts success statuses to codecov/patch and codecov/project.
  A later Codecov upload for the same commit overwrites them.

Any other check name is refused and nothing is posted. The comment is
read only through an environment variable, the reason is reduced to one
line and the status description is capped at 140 characters. The
commenter must have write, maintain or admin access. When the command
does not complete, the workflow replies on the PR with a link to the run.

`/override-visual-regression <reason>` is kept as a transition alias
for `/override-check visual-regression <reason>`.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Validation on a real PR is needed to confirm that Actions-posted Codecov statuses satisfy required checks and any expected-app restrictions in the repository’s unavailable branch-protection settings.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

This branch was successfully deployed

3 active deployments
dev-pages-react18 — b909b9c9 Deployed Nov 19, 2026 by vvaliyev via deploy (React 18) / deploy #3414
dev-pages-react16 — b909b9c9 Deployed Nov 19, 2026 by vvaliyev via deploy (React 16) / deploy #3414
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants