Repository navigation
Release v1.19.8 - #864
Merged
Merged
Release v1.19.8#864
Conversation
Stacked on #6913 (TASK-27.5, the other five roles). Review that first; this PR's own changes are the 7 commits above it. This promotes the last two change-analysis roles to GPT-6 for every account, as Dylan decided in TASK-36. After it, no role has an `Internal*` (FlagInternal) override. | Role | Before (all accounts) | After (all accounts) | |---|---|---| | Primary (V6 investigator) | `gpt-5-4` (internal: `gpt-6-1-sol`) | `gpt-6-1-sol` | | BlastRadius | `gpt-5-mini` (internal: `gpt-5-4-mini-low`) | `gpt-6-luna-medium` | ## Primary: Sol with the not-disproven guidance and the low cap - **Sol:** the default `PrimaryProviderName` is now `gpt-6-1-sol`. `InternalPrimaryProviderName` is removed. - **Not-disproven guidance, on for everyone.** The investigator reports a concrete mechanism that it tried to disprove and could not as a risk. It also says what it checked. Its answer gains `risk_basis` (`confirmed`, `not_disproven` or `none`). - Until now this was gated on FlagInternal or the `not-disproven-risks` flag. That gate is removed, so the guidance is always appended to the system prompt and the investigator always uses the `risk_basis` response schema. - The `FlagNotDisprovenRisks` constant and the benchmark-only `V6SpecificArgs.NotDisprovenRisks` switch are removed with the gate. The PostHog flag can be deleted. - Without this change, other accounts would get Sol with the type list but no guidance. TASK-27.18 measured that combination: about 4 risks per run, and no credential or namespace risks found. - **Low cap, in code.** Only a risk with `risk_basis: confirmed` keeps a level above low. A not-disproven, missing, `none` or unknown basis is lowered to low (Dylan, TASK-27.21). The span records `ovm.analysis.oneshot.riskBasis` and `notDisprovenLevelCapped`. - **Citation stripping:** web-search citation markers are removed from risk titles, descriptions and reasoning. - **Type list:** the investigator's prompt lists the types it can query, for every account. The stale "check the available types list" paragraph is gone from both V6 prompts. The NetworkPolicy adapter now declares its query methods, so it is included in the list. TASK-27.21's `TestV6Benchmark` (mean of 3 runs, judge gpt-5.4; bastion-sg-open-egress aborted in every run, so 9 scenarios per run): | | gpt-5.5 (old internal) | Sol, this PR | |---|---|---| | Model cost per run | $18.89 | $4.96 (−74%) | | Wall-clock per run | 1,502s | 996s (−34%) | | Risks found | 16.7 | 14.0 | | Golden accuracy | 0.83 | 0.59 | | Not-disproven risks above low | | 0 of 29 | | Risks with citation markers | | 0 | | Query calls rejected as unknown type (3 runs) | | 2 (157 without the list) | - Credential and namespace hypotheses are raised as often as in TASK-27.19. - Dylan accepted the lower golden accuracy (TASK-36). The score is mostly risk count against each scenario's expected range. The gap comes from the bastion abort and from extra low "could not verify" risks on update-gitops and verify-email, not from worse findings. - Also accepted: confirmed single-replica risks are mostly rated high (8 of 13). ## BlastRadius: luna at medium effort, the v1 line and the stall guard - **luna-medium:** a new `gpt-6-luna-medium` provider (`gpt-6-luna`, reasoning effort medium) is the default `BlastRadiusProviderName`. `InternalBlastRadiusProviderName` is removed. - **v1 line:** `blast_radius/system.tmpl` gains one line. It tells the model to call `record-risk` only for a concrete risk, never to record "no impact". This fixes luna's "no impact" record-risk calls (TASK-27.22). - **Stall guard:** `WithStreamIdleTimeout(20s)` and `WithStreamIdleMaxRetries(2)`. A stream that sends nothing for 20s is retried, up to twice. This guards against the 42s to 62s stalls seen at low effort in TASK-27.20 and TASK-27.22. 20s is above luna's p99 call latency and well under the 60s hard timeout. No retry fired in TASK-27.24's 1,071 calls, so the retry path has not been tested against a live stall. It also caused no false retries. - **Deadline:** `gpt-5-mini-2025-08-07` retires on **2026-12-11**, so this has to ship before then. After this PR no default pin uses a `gpt-5-mini*` provider. The providers stay registered. TASK-27.24, a replay of 119 TASK-24 chunks with 3 repeats: | | gpt-5-mini | luna-medium + v1 | |---|---|---| | Recall (95% CI) | 0.687 (0.61–0.74) | 0.798 (0.70–0.87) | | Precision (95% CI) | 0.852 (0.74–0.92) | 0.995 (0.99–1.00) | | "No impact" record-risk calls per repeat | 0 | 6.7 (25.7 for luna-low + v1) | | Cost per 1,000 chunks | $1.75 | $0.54 | | Latency p50 / p99 | 6.6 / 23.5s | 6.4 / 25.7s | The recall and precision gains over gpt-5-mini both have CIs that exclude 0. TASK-27.25, `TestV6Benchmark` on the 4 scenarios that depend on observations, 3 runs: | | gpt-5-mini | luna-medium + v1 | |---|---|---| | Rollout/singleton risk reported (of 12 scenario-runs, hand-checked) | 11 | 10 | | "No impact" hypotheses | 0 | 0 | | Golden accuracy | 0.90 | 0.95 | | Blast-radius model cost per run | $0.141 | $0.043 | | Blast-radius time per scenario | 42.5s | 54.3s | | Blast-radius call p50 / p95 | 2.7 / 8.2s | 3.6 / 11.9s | Accepted cost (TASK-36): luna-medium adds about 12s of blast-radius time per change analysis. ## Tests - `TestLoadLLMConfigPins` now asserts the new defaults and that every `Internal*` pin is empty. - `TestAppendPromptGuidance` asserts that the not-disproven guidance is always appended, last. - The tests from the source commits cover the type list, the cap, `risk_basis` and citation stripping. - Local runs: - `go test -race` passes for `service/config` and `service/changeanalysis/...`, including the Postgres-backed tests against the local apiserverdb. - `go test -race` passes for `k8s-source/adapters`. - golangci-lint v2.13.2 reports 0 issues on the touched packages. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **High Risk** > Rolls out new default LLM models and investigator risk semantics (not-disproven + severity capping) to all customers, directly affecting change-analysis cost, latency, and reported risk levels. > > **Overview** > Ships **GPT-6 for the last two change-analysis roles** for every account: V6 investigator **Primary** becomes `gpt-6-1-sol` and blast radius becomes **`gpt-6-luna-medium`** (new provider with medium reasoning, 20s stream idle timeout/retries). All `Internal*` model overrides are cleared; config tests pin the new defaults. > > **Investigator behavior (always on, no feature flag):** not-disproven risk guidance is appended to every system prompt; responses use `risk_basis` (`confirmed` / `not_disproven` / `none`) and **code caps** any non-confirmed real risk to **low**. Web-search citation junk is stripped from risk text. The prompt gets a **sorted, order-stable “available types”** section; `Query` tool validation errors reference that list only when it’s present. **NetworkPolicy** k8s metadata now declares query methods so it can appear in that list; a test enforces all adapters declare methods. > > **Blast radius:** one prompt line—use `record-risk` only for concrete risks, not “no impact.” **`FlagNotDisprovenRisks`** is removed from feature flags. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 6c2cbb63311046756c7fcf04e074f381aa08f8fc. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 5.5 <[email protected]> GitOrigin-RevId: 55a3189fac5b4ca84f6a78ae991f0dd4af6093f3
Mechanical dependency bumps only. These do not change functionality. ## Ready now (scan) - cloud.google.com/go/dataplex v1.38.1 -> v1.39.0 - cloud.google.com/go/kms v1.36.1 -> v1.37.0 - cloud.google.com/go/run v1.24.1 -> v1.25.0 - github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cosmos/armcosmos/v4 v4.0.0 -> v4.1.0 - github.com/auth0/go-auth0/v3 v3.11.0 -> v3.12.0 - github.com/aws/aws-sdk-go-v2 v1.47.3 -> v1.48.0 - github.com/aws/aws-sdk-go-v2/config v1.33.9 -> v1.33.18 - github.com/aws/aws-sdk-go-v2/credentials v1.20.9 -> v1.21.0 - github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.20.3 -> v1.20.9 - github.com/aws/aws-sdk-go-v2/service/apigateway v1.52.1 -> v1.52.9 - github.com/aws/aws-sdk-go-v2/service/autoscaling v1.78.4 -> v1.82.0 - github.com/aws/aws-sdk-go-v2/service/cloudfront v1.75.0 -> v1.75.6 - github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.73.3 -> v1.76.0 - github.com/aws/aws-sdk-go-v2/service/directconnect v1.55.1 -> v1.56.0 - github.com/aws/aws-sdk-go-v2/service/dynamodb v1.70.3 -> v1.74.0 - github.com/aws/aws-sdk-go-v2/service/ecs v1.100.3 -> v1.105.1 - github.com/aws/aws-sdk-go-v2/service/efs v1.49.4 -> v1.51.6 - github.com/aws/aws-sdk-go-v2/service/eks v1.104.1 -> v1.105.6 - github.com/aws/aws-sdk-go-v2/service/elasticloadbalancing v1.41.4 -> v1.42.0 - github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.63.4 -> v1.64.0 - github.com/aws/aws-sdk-go-v2/service/kms v1.61.4 -> v1.62.6 - github.com/aws/aws-sdk-go-v2/service/networkmanager v1.50.4 -> v1.51.6 - github.com/aws/aws-sdk-go-v2/service/rds v1.130.3 -> v1.133.0 - github.com/aws/aws-sdk-go-v2/service/route53 v1.70.4 -> v1.71.0 - github.com/aws/aws-sdk-go-v2/service/s3 v1.114.3 -> v1.114.10 - github.com/aws/aws-sdk-go-v2/service/sesv2 v1.79.1 -> v1.79.9 - github.com/aws/aws-sdk-go-v2/service/sns v1.47.5 -> v1.48.0 - github.com/aws/aws-sdk-go-v2/service/sqs v1.52.4 -> v1.53.0 - github.com/aws/aws-sdk-go-v2/service/sts v1.51.4 -> v1.52.0 - github.com/aws/smithy-go v1.28.8 -> v1.28.9 - github.com/jackc/pgx/v5 v5.12.0 -> v5.13.0 - github.com/nats-io/jwt/v2 v2.8.2 -> v2.8.3 - github.com/nats-io/nats-server/v2 v2.15.2 -> v2.15.3 - github.com/resend/resend-go/v4 v4.12.0 -> v4.12.1 - github.com/zclconf/go-cty v1.19.0 -> v1.20.0 - gopkg.in/ini.v1 v1.67.3 -> v1.68.0 Modules raised by minimum version selection: - github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op -> v0.9.0-default-no-op - github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.22 -> v1.7.27 - github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.6 -> v1.5.12 - github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.6 -> v2.8.12 - github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.6 -> v1.5.12 - github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.21 -> v1.13.25 - github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.11.7 -> v1.11.13 - github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.13.6 -> v1.13.12 - github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.6 -> v1.14.12 - github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.20.6 -> v1.20.12 - github.com/aws/aws-sdk-go-v2/service/signin v1.10.4 -> v1.13.1 - github.com/aws/aws-sdk-go-v2/service/sso v1.38.4 -> v1.39.5 - github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.4 -> v1.44.5 - github.com/klauspost/compress v1.20.3 -> v1.20.4 - github.com/nats-io/nkeys v0.4.16 -> v0.4.17 - golang.org/x/oauth2 v0.37.0 -> v0.38.0 - golang.org/x/sys v0.49.0 -> v0.50.0 Left out: `cloud.google.com/go/compute` stays at v1.70.0. v1.75.0 removes `ReturnPartialSuccess` from the `AggregatedList*Request` types used in `sources/gcp/manual`, so the bump needs a code change. Left out: `github.com/openai/openai-go/v3` stays at v3.70.0. With v3.99.0, `TestCalculateBlastRadius` failed 2 of 3 runs (the model returned no affected resources), so the client bump will be evaluated separately from this mechanical batch. ## Devcontainer build fix `devcontainers/features#1744` made the go, terraform, github-cli and node feature `install.sh` scripts source helpers from a `scripts/` directory next to them, so piping them into bash has failed on `main` since 2026-11-13. `.cursor/devcontainer-build/Dockerfile` now extracts those four feature directories from the upstream tarball and runs each `install.sh` in place. ## Vulnerability fixes (High/Critical) - node 24.22.1 -> 24.22.3 (Critical CVE-2026-123864; High CVE-2026-123863, CVE-2026-123865, CVE-2026-123866, CVE-2026-123867, CVE-2026-123868, CVE-2026-123872, CVE-2026-123874, CVE-2026-123875, CVE-2026-123877, CVE-2026-126051) - `services/frontend/Dockerfile` (`node:24.22.3-alpine3.23`, both stages) - `services/frontend/Dockerfile.playwright` (`node:24.22.3-trixie-slim`) - Copied pins: `NODE_VERSION` in `plugins/private-evals/container/Dockerfile` and `.cursor/devcontainer-build/Dockerfile`, and `node-version` in the CI workflows under `.github/workflows/` and `plugins/.github/workflows/` ## Test plan - [x] `go build ./...` - [x] `go vet` on `sources/gcp`, `services/until-backend`, `go/` - [ ] CI green on this draft <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Wide Go client bumps (especially AWS and Auth0) and uniform Node runtime changes affect build, CI, and containers; risk is limited by mechanical version-only updates but warrants a green CI pass. > > **Overview** > This PR is primarily a **mechanical dependency refresh**: `go.mod` / `go.sum` pick up patch and minor releases across cloud clients (GCP, Azure Cosmos, AWS SDK v2), Auth0, `pgx`, NATS, Terraform’s `go-cty`, and related transitive modules, with no application code changes in the diff. > > **Node.js is pinned to 24.22.3** everywhere it was 24.22.1—CI `setup-node` jobs, frontend and Playwright Docker bases, private-evals and devcontainer `NODE_VERSION`—to address reported High/Critical Node CVEs. > > The **devcontainer image build** no longer pipes `install.sh` from GitHub; it downloads the devcontainers features tarball, extracts the go/terraform/github-cli/node feature trees, and runs each script in place so sibling `scripts/` helpers resolve correctly, then removes the temp tree after the node install. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 75e8c8ba3a691dbc26a58c69b19a27cb03240121. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Cursor <[email protected]> GitOrigin-RevId: 128c73cec5fbe38f5c75049f491d1a898f7f648f
<!-- CURSOR_AGENT_PR_BODY_BEGIN --> Fixes [ENG-6865](https://linear.app/overmind/issue/ENG-6865/exclude-user-data-from-azure-vm-and-scale-set-items). Azure VM and scale set converters copied the full compute model into SDP attributes, including `properties.userData` / `virtualMachineProfile.userData` and OS-profile `customData`. Those bootstrap payloads can contain tokens, keys, or passwords, and were then stored in the account graph and snapshots. ## Changes - Add `shared.ToAttributesWithPresence` to replace named JSON paths with `true` when the field exists. Add/remove of bootstrap data remains visible; the payload itself is not stored. - A digest is intentionally not stored. Unkeyed SHA-256 and HMAC keyed by the public item scope still let someone confirm a guessed low-entropy payload from graph data, and Azure converters have no tenant HMAC secret from key management. - Redact VM `properties.userData` and `properties.osProfile.customData` before building attributes. - Redact scale set `properties.virtualMachineProfile.userData` and `properties.virtualMachineProfile.osProfile.customData` the same way. Fixing the VM converter alone does not cover scale sets. - Unit tests assert the sentinel payloads never appear in attributes and that the fields become `true`. - Adapter-author guidance and a BUGBOT rule so new converters do not reintroduce the leak. Run-command scripts and extension protected settings are out of scope for this ticket. ## Testing - `go test -race ./sources/shared/ -run TestToAttributes` - `go test -race ./sources/azure/manual/ -run 'TestComputeVirtualMachine|TestComputeVirtualMachineScaleSet'` - `golangci-lint run ./sources/shared/ ./sources/azure/manual/` <!-- CURSOR_AGENT_PR_BODY_END --> <div><a href="https://cursor.com/agents/bc-05689b44-472b-44a5-9b87-58d40f7eb19c?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-05689b44-472b-44a5-9b87-58d40f7eb19c&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div> Fixes ENG-6865 <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches secret handling in graph snapshots for core compute adapters; behavior change is intentional but could affect change detection expectations for bootstrap fields (presence only, not content). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 68d949e2e9d344bbac6960492891d22acf654f10. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Cursor Agent <[email protected]> GitOrigin-RevId: 269a0b4a1f4395d91a7253c26ab3240e8f5fb9a2
tphoney
approved these changes
Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copybara Sync - Release v1.19.8
This PR was automatically created by Copybara, syncing changes from the overmindtech/workspace monorepo.
Original author: TP Honey ([email protected])
What happens when this PR is merged?
tag-on-mergeworkflow will automatically create thev1.19.8tag on mainReview Checklist