feat(deps): agent-driven Dependabot PR verifier (phase 1)

.agents/skills/verify-dependency-pr/SKILL.md

@@ -0,0 +1,150 @@
+---
+name: verify-dependency-pr
+description: Verify a Dependabot (or other dependency-update) PR on ldcli to the standard of a diligent reviewer, and produce a verdict comment (safe to merge, needs a human decision, block, or incomplete). Use to verify, review, triage, or check a dependency update PR, given its number or branch.
+---
+
+# Verify a dependency PR
+
+You verify the PR to the standard of a diligent reviewer. After your run, a person must not have to verify anything again. A person only answers the decisions that your comment asks for.
+
+You do not approve, request changes, merge, or push to the PR branch. Verification writes files only. The caller runs `post-comment.sh` to post the comment.
+
+All paths are relative to the repository root. Output goes to `.verify-out/pr-<N>/`, which git ignores.
+
+## Verdicts
+
+| Verdict | Meaning | Exit code |
+|---|---|---|
+| `block` | The PR must not merge as it is. The comment gives the fix. | 1 |
+| `needs-human` | Verification is complete. A person must answer one or more specific questions. | 1 |
+| `incomplete` | A required check, gate, or review did not run, or proved nothing. A rerun or more agent work closes it, not a person. | 2 |
+| `safe-to-merge` | Every required check and review ran and passed. | 0 |
+
+A gate is a check that must pass, for example the build or the tests. A failure that also happens on base is "pre-existing" and does not count against the PR. Some checks list separate findings: actionlint, golangci-lint, prettier, and `npm ls`. If each finding of the PR also occurs on base, the failure is pre-existing. A pre-existing failure never satisfies a gate.
+
+## The diligent reviewer standard
+
+This table shows what must be true before the PR can be safe to merge. The tier comes from `result.json` (`.tier`). You can raise the tier, but you cannot lower it.
+
+| Tier | Typical updates | Required |
+|---|---|---|
+| low | A patch of a direct dependency, a dev dependency, or a transitive-only change | The baseline checks pass. |
+| medium | A minor update of a runtime dependency, or any update in a risk area (CGO, LaunchDarkly SDKs, CLI libraries, UI framework, Docker base image) | The low items, plus an impact review of each direct update. If upstream behavior changes reach ldcli, at least one generated check must prove the change. |
+| high | A major update (including a 0.x minor), a code generator, release tooling, a go directive change, or more than three direct updates | The medium items, plus each breaking change mapped to the ldcli code that it touches. |
+
+The baseline checks cover each ecosystem: build, tests, generated code, UI dist, smoke tests, transitive changes, licenses, upstream notes, and Actions interfaces. The playbooks list them.
+
+## Procedure
+
+### 1. Run the baseline
+
+```bash
+scripts/dependency-pr/verify.sh --pr <N>          # or --branch <name>
+```
+
+The script makes two worktrees. `work/base` is the tip of the base branch. `work/pr` is the PR merged into that tip. The script classifies the updates, runs the baseline checks, and writes `result.json`, `comment.md`, and `logs/`. If Docker is available and the update touches CGO, the go directive, or release tooling, add `--profile full`.
+
+Read these fields in `result.json`:
+
+- `.classification`: the updates, the tier, the risk tags, and the reasons for the tier.
+- `.blocks`, `.decisions`, `.incomplete`: the items that set the verdict.
+- `.checks[]`: the outcome of each check. The log of a check is in `logs/<side>/<id>.log`. Its evidence is in `state/checks/<id>/<side>/`.
+- `.fixes`: machine-applicable fix recipes. Do not apply them. A later step will.
+
+If the exit code is 2 because a tool is missing, install the tool and run the script again.
+
+### 2. Do the impact review (medium and high tier)
+
+Open the playbook for each ecosystem in `.classification.ecosystems`:
+
+| Ecosystem | Playbook |
+|---|---|
+| gomod | [playbooks/gomod.md](playbooks/gomod.md) |
+| npm-ui (`internal/dev_server/ui`) | [playbooks/npm-ui.md](playbooks/npm-ui.md) |
+| npm-wrapper (root `package.json`) | [playbooks/npm-wrapper.md](playbooks/npm-wrapper.md) |
+| github-actions | [playbooks/github-actions.md](playbooks/github-actions.md) |
+| docker | [playbooks/docker.md](playbooks/docker.md) |
+
+For each direct update, do these steps:
+
+1. Read the upstream notes. The `upstream-changes` check saves them in `state/checks/upstream-changes/pr/notes/`, and its details give the compare link. For an npm package in a monorepo, the notes come from the package `CHANGELOG.md`. For `golang.org/x/*` modules, the notes are the commit messages. The summary gives the coverage of each update. If the notes are partial or missing, read the upstream diff from the compare link for the rest of the range. Also read the PR body. If the `pr-disclosure` check names direct updates that the body does not name, review those updates the same as the others. Only Dependabot's own text counts as disclosure. The check ignores bot summaries and quoted release notes. An update that a named update requires is "forced", and it needs no decision.
+2. Find breaking changes, security fixes, deprecations, new minimum versions (Go, Node, runner), new peer requirements, and license changes.
+3. Find where ldcli uses the dependency. For Go, use `rg -l '"<module>' --glob '*.go'` and `go list -deps ./... | rg <module>`. For the UI, use `rg "from '<pkg>" internal/dev_server/ui/src`.
+4. Map each upstream change to the ldcli code that it reaches. A change that no ldcli code reaches is "not reachable".
+5. Read the `transitive-changes` and `license-changes` details. Explain each new module, major transitive update, and license change.
+
+Write `agent/impact.json` in the output directory:
+
+```json
+{
+  "schema": 1,
+  "summary": "What changed upstream, and what in ldcli it reaches.",
+  "tier": "medium",
+  "tier_reasons": ["Only if you raise the tier: the reason"],
+  "changelog": [{"package": "github.com/mattn/go-sqlite3", "range": "1.14.28..1.14.52", "notes": "…", "breaking": false, "url": "https://…"}],
+  "usage": ["internal/dev_server/db/sqlite.go"],
+  "behavior_changes_reachable": true,
+  "no_local_proof": "Only if no local check can run the changed code: the reason",
+  "findings": [
+    {"severity": "info", "text": "A fact for the reader.", "evidence": ["url or path"]},
+    {"severity": "decide", "question": "Accept X for Y?", "text": "Why only a person can decide this.", "evidence": ["…"]},
+    {"severity": "block", "text": "Concrete breakage, for example a removed API that ldcli calls.", "evidence": ["…"]}
+  ]
+}
+```
+
+The verdict is `incomplete` if `changelog` does not name every direct update, or if `behavior_changes_reachable` is missing.
+
+Sometimes no local check can run the changed code. An example is an action that only release workflows use. In that case, set `no_local_proof` to the reason. The verdict then asks a person to accept the change without a local proof. Do not use `no_local_proof` to avoid a check that you can write.
+
+### 3. Write generated checks
+
+If `behavior_changes_reachable` is true, write at least one discriminating check. Follow [reference/generated-checks.md](reference/generated-checks.md). Put the checks in `generated/` in the output directory. The important rules:
+
+- A discriminating check must fail on the old version and pass on the new version. Only such a check counts as proof.
+- A guard asserts behavior that must not change. It passes on both versions. A guard never counts as proof, but a guard that fails on the PR blocks the PR.
+- Assert behavior, not version strings. Test code that the update reaches.
+
+### 4. Run the generated checks
+
+```bash
+scripts/dependency-pr/verify.sh --pr <N> --phase generated
+```
+
+This command runs each generated check on base and on the PR. It reads `agent/impact.json`, calculates the verdict again, and writes `comment.md` again. If you change only `impact.json`, use `--phase render`. If a generated check is `error`, `invalid`, or `fails-both`, fix it and run the phase again.
+
+### 5. Report
+
+Report the verdict, the blocks, the decisions, the incomplete items, and the path to `comment.md`. Give pre-existing problems on main separately. A generated check can find the same type of problem in future updates. In that case, recommend it for the baseline (see "Promotion" in the reference).
+
+To post, the caller runs:
+
+```bash
+scripts/dependency-pr/post-comment.sh --out-dir .verify-out/pr-<N> --dry-run   # then without --dry-run
+```
+
+The script edits one comment in place. It does not post if the PR head moved after verification.
+
+## When to ask a person
+
+Ask a person only when the evidence is complete and the next step is a choice. Each decision is one question that a person can answer with yes or no. Put the evidence next to it.
+
+These items are decisions:
+
+- Accept a change that no automated check can run, for example "Accept actions/checkout v6 in release workflows that PR CI does not run?"
+- Accept a new license, a license change, or a new npm install script.
+- Accept a visible change to the CLI, for example different help text or flags.
+- Choose between options, for example "Remove the unused react-window instead of updating it?"
+
+These items are not decisions:
+
+- A stale dist, untidy `go.mod`, or generated code that is out of date. These are blocks with a fix.
+- A check that did not run because a tool is missing. This item is incomplete.
+- A breaking change that you have not mapped yet. Do the mapping.
+
+## Rules
+
+- Do not fix the PR yourself. Each failing check records a fix recipe in `result.json` for a later step.
+- Do not run ad hoc commands and report them as baseline results. Put them in generated checks, so that they run on both versions.
+- Do not trust the local environment. `verify.sh` removes `LD_*` variables and uses private XDG directories, because local credentials make `go test` fail. Generated checks get the same environment.
+- A check that fails because of a tool or environment problem must report `incomplete`, not `fail`.

.agents/skills/verify-dependency-pr/playbooks/docker.md

@@ -0,0 +1,24 @@
+# Playbook: Docker base image (`Dockerfile.goreleaser`)
+
+The published image is `FROM alpine:<tag>` plus the static `ldcli` binary from goreleaser. The release builds the images. PR CI does not build them.
+
+## Diligent reviewer standard
+
+1. The new tag exists.
+2. The release image builds from the PR with a binary made as the release makes it: CGO for SQLite, musl, static link.
+3. In the image, `ldcli --version` runs, HTTPS to LaunchDarkly works with the CA bundle of the image, and the dev server starts and serves its API. The dev server uses SQLite, so this step tests CGO on musl.
+4. The changes between the two base image versions are known (musl, CA bundle, busybox, end of support).
+
+## What the baseline checks
+
+`docker-image` (gate) covers items 1 to 3. It needs Docker and `musl-gcc` (package `musl-tools`). If one of them is missing, or if a registry lookup or image pull fails, the check reports `incomplete`, not `fail`. `binary-smoke` also runs.
+
+## What the agent must do
+
+- Read the Alpine release notes for each minor version in the range. ldcli links statically, so it needs only the kernel interface and the certificates in `/etc/ssl`.
+- Write down the support status. Alpine branches get security fixes for about two years. A move away from a branch that has no more support is a security improvement.
+- If a scanner is available (`docker scout cves`, `trivy image`), compare the CVE counts of the old and new image.
+
+## When to ask a person
+
+If the image builds and runs and the impact review is complete, the PR can be safe to merge. Ask a person only if the new base image changes what users see, for example a removed shell or a different user.

.agents/skills/verify-dependency-pr/playbooks/github-actions.md

@@ -0,0 +1,38 @@
+# Playbook: GitHub Actions (`.github/workflows`, `.github/actions`)
+
+## Diligent reviewer standard
+
+1. Each third-party action uses a full commit SHA with a version comment (SEC-7924, #668).
+2. The workflows pass `actionlint`.
+3. For each updated action, the inputs that the repository passes still exist, and no new required input is missing. The outputs that later steps read still exist.
+4. Changed input defaults, the runtime (for example `node20` to `node24`), and the runner requirements are known.
+5. The `permissions` blocks of the workflows did not change, or the change is approved.
+6. PR CI runs each workflow that uses the action. For a workflow that runs only at release, a person accepts the risk.
+7. The upstream release notes between the two versions are known.
+
+## What the baseline checks
+
+| Statement | Check |
+|---|---|
+| 1 | `actions-pinning`. Actions from `actions/`, `github/`, and `launchdarkly/` are exempt, as in the current repository. |
+| 2 | `actionlint` (v1.7.7, installed through `go install` when it is missing) |
+| 3, 4, 5 | `actions-coverage`. It reads the upstream `action.yml` at both refs and traces composite actions back to the workflows that call them. It also compares the `permissions` blocks on base and PR. If the interface does not match, the PR is blocked. If workflows pin the action at different old refs (v4 and v5), the check compares each old ref with the new ref. An output that no version declares is set at run time, so the check cannot compare it. The details name it. |
+| 6 | `actions-coverage`. A breaking update that a workflow without a `pull_request` trigger uses becomes a decision. |
+| 7 | `upstream-changes` |
+
+PR CI runs only `go.yml`, `dev-server-ui.yml`, `dependency-scan.yml`, and `lint-pr-title.yml`. `release-please.yml`, `manual-publish.yml`, `check-openapi-updates.yml`, and the `publish` and `publish-npm` composite actions do not run on a PR.
+
+## What the agent must do
+
+- Read the release notes of each major version in the range. Note new runner minimum versions, changed defaults, and changed credential or token behavior.
+- Look at the steps that use the action in workflows that PR CI does not run. Make sure that their inputs, outputs, and side effects (for example `git push` after `actions/checkout`) still work. Put each check that you can write as a guard in `generated/`.
+- Dependabot does not scan the composite actions in `.github/actions/`. Note any version difference that the update creates between a workflow and a composite action.
+- For release-please majors, compare `release-please-config.json` and `.release-please-manifest.json` with the configuration schema of the new version, and the outputs that `release-please.yml` reads.
+
+## When to ask a person
+
+- A breaking update is used in workflows that PR CI does not run. Ask: "Accept <action> <version> in <workflows>, which PR CI does not run?" Give the interface comparison, the runtime change, and the guard results as evidence.
+- A workflow `permissions` block changed.
+- A non-breaking update changes the default of an input that the repository does not set.
+
+No local check can run a release workflow. The comment can recommend a `manual-publish` dry run (`dry-run: true`) as follow-up.

.agents/skills/verify-dependency-pr/playbooks/gomod.md

@@ -0,0 +1,54 @@
+# Playbook: Go modules (`go.mod` and `go.sum`)
+
+## Diligent reviewer standard
+
+A diligent reviewer makes sure that these statements are true for a Go update:
+
+1. ldcli builds, passes `go vet`, and passes its tests with the new version.
+2. `go mod tidy` and `go generate ./...` make no change. If a generator changed, the regenerated code builds.
+3. The binary runs. All commands print help, and the dev server serves its UI and API.
+4. Every release target compiles. The release uses CGO for SQLite on linux (musl, static), windows (mingw), and macOS (osxcross).
+5. The upstream changes between the two versions are known. Each breaking change, security fix, and behavior change is mapped to ldcli code, or is shown to be not reachable.
+6. Each new or changed module that goes into the binary is known, and its license is in the policy.
+7. A go or toolchain directive change works with every tool that CI and the release use.
+8. No new reachable vulnerability appears.
+
+## What the baseline checks
+
+| Statement | Check |
+|---|---|
+| 1 | `go-build-vet`, `go-test` (both are gates) |
+| 2 | `go-mod-tidy`, `go-generate-drift` |
+| 3 | `binary-smoke` (gate), `cli-help-diff` |
+| 4 | `release-snapshot` (full profile, needs Docker). It is a gate for updates with the `cgo` or `go-directive` tag. If the private release image cannot be pulled, the check uses the public `goreleaser/goreleaser-cross:v1.24.2` image and musl.cc toolchains, both pinned. The summary names the image, and the details list the fidelity gaps. |
+| 5 | `upstream-changes` collects the notes and the compare link. For `golang.org/x/*` modules, it uses the `github.com/golang` mirror, and the notes are the commit messages. The agent does the mapping. |
+| 6 | `transitive-changes`, `license-changes` |
+| 7 | `go-directive`, plus `golangci-lint` and `release-snapshot` as gates when the tag `go-directive` is set |
+| 8 | `govulncheck`. Only a new reachable advisory fails. The details list the advisories that the PR fixes (reachable, in an imported package, or in a required module) and the reachable advisories that stay. |
+
+## What the agent must do
+
+- Read the notes in `state/checks/upstream-changes/pr/notes/`. If a module has no notes, read the compare diff. Look at the API changes in the packages that ldcli imports.
+- Find the ldcli code that uses the module: `rg -l '"<module>' --glob '*.go'`. For an indirect update, find the reason with `go mod why -m <module>` in `work/pr`.
+- For each changed transitive module in the `transitive-changes` details, find what pulls it in, and say if it changes behavior that ldcli uses.
+- If a directive change appears, read the release notes of the new Go version for changes that affect ldcli.
+
+## Risk areas
+
+| Tag or module | What to look at | Generated-check ideas |
+|---|---|---|
+| `codegen`: oapi-codegen, kin-openapi, strcase | The generator and its runtime library must change together (for example `oapi-codegen/runtime`). `go-generate-drift` must not become worse than on base. | Regenerate in `work/pr`, build, and run `go test ./internal/dev_server/api/...`. |
+| `mocks`: go.uber.org/mock | The repository commits the mocks. A diff after regeneration means that the mock format changed. | Regenerate the mocks and run `go test ./internal/dev_server/...`. |
+| `cgo`, `dev-server`: mattn/go-sqlite3 | The dev server stores its state in SQLite: `internal/dev_server/db`, `events_db`, `db/backup`. | A temporary test in the affected package. For example, the #829 check measures allocations per row in `events_db.QueryEvents`. |
+| `ld-sdk`: go-server-sdk, go-sdk-common | The dev server forwards SDK streams and evaluations (`internal/dev_server/sdk`, `adapters`). | Start the dev server, import a project from a fixture, and evaluate a flag through the `/sdk` endpoints. |
+| `setup`: sdk-meta | It supplies the SDK lists for `setup` and quickstart. | Compare the SDK list output on base and PR. Make sure that each SDK ID that ldcli uses still exists. |
+| `cli-surface`: cobra, pflag, viper, mapstructure | Flag parsing, the order of `LD_*` variables and the configuration file, and the usage templates. `cli-help-diff` shows the help changes. | A test of the configuration order for `--base-uri`: the flag first, then `LD_*`, then the file. |
+| `tui`: charmbracelet | The interactive flows have few tests. | Help output and output without a terminal for `setup` and quickstart. |
+
+## When to ask a person
+
+- `cli-help-diff` shows a change in help text or flags. Ask if the change is acceptable.
+- A new module or a changed module has a license outside the policy, or its license changed.
+- The impact review finds a behavior change that ldcli users will see. Ask if the change is acceptable, and state the change.
+
+A pre-existing `go-generate-drift` failure (from #720) is not caused by the PR. The comment lists it once. It does not block the PR.