Skip to content

ci: lint workflows and test the PR check's key-sharing decision - #75

Open
scott-lowe-vapi wants to merge 1 commit into
test/docs-integrityfrom
ci/workflow-hardening
Open

scott-lowe-vapi wants to merge 1 commit into
test/docs-integrityfrom
ci/workflow-hardening

Conversation

@scott-lowe-vapi

@scott-lowe-vapi scott-lowe-vapi commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Value

V.A.L.U.E. tier: small — touches .github/workflows/ (a blast-radius path) and the logic deciding which PRs receive the repository's Vapi keys.

  • Problem: the PR check workflow decides, in a few lines of bash, whether a pull request runs live (with the repository's Vapi keys) or as a keyless dry run. That decision had no test, and no workflow in this repo had ever been linted. A mistake there either leaks keys to fork or Dependabot PRs, or silently turns every check into a dry run.
  • Who it affects: every repo that enables PR checks, and the orgs whose keys are stored in it.
  • What changes:
    • New tests/vapi-checks-workflow.test.ts runs the workflow's real steps, read from vapi-checks.yml, with bash:
      • the live-or-dry decision: own branch and manual dispatch run live; a fork, a fork with a look-alike repo name, Dependabot as actor or as PR author, a missing head repo, and a pull_request_target event all run dry;
      • the run step's arguments, with a stub node: dry vs live, a named check on dispatch, --changed-since only when the base ref exists, and empty key variables unset;
      • statically: secrets are only passed when the decision says live, the triggers are exactly pull_request and workflow_dispatch, permissions are contents: read and statuses: write, and checkout doesn't persist credentials.
    • The decision now also requires a pull_request event before going live, so adding a trigger later can't widen who gets the keys.
    • ci.yml gains an actionlint job: a pinned release whose sha256 is verified before it runs. It also runs shellcheck over every run: block.

Evidence of value

Check Result
Remove the Dependabot-author guard from the decision step decision test fails on "maintainer re-runs Dependabot's PR"
Current workflow 4 tests pass
actionlint first run is this PR's CI; findings, if any, are fixed in this PR

Testing plan

  • npm test (509 tests) and npx tsc --noEmit pass.
  • Not tested: the workflow on GitHub's runners end to end; the PR check itself only runs once a repo sets VAPI_CHECKS_ENABLED.

Refs TEST-141

🤖 Generated with Claude Code

The PR check workflow decides whether a pull request gets this
repository's Vapi keys. That decision was untested bash, and no workflow
had ever been through actionlint.

- tests/vapi-checks-workflow.test.ts runs the workflow's real steps from
  vapi-checks.yml with bash: the live-or-dry-run decision across own
  branches, manual runs, forks (including a look-alike repo name),
  Dependabot as actor or author, a missing head repo and other events;
  the run step's arguments (dry vs live, named check, merge-base
  selection, with and without a base ref) using a stub node; empty key
  variables being unset; and statically, that secrets are gated on a live
  run, the triggers are pull_request and workflow_dispatch only, the
  permissions are contents: read and statuses: write, and checkout
  doesn't persist credentials. Dropping the Dependabot-author guard fails
  the decision test.
- The decision step now also requires a pull_request event before going
  live, so adding a trigger later can't widen who gets the keys.
- ci.yml gets an actionlint job: a pinned release whose sha256 is checked
  before it runs, with shellcheck linting every run: block.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant