Skip to content

Release v1.19.8 - #864

Merged
tphoney merged 4 commits into
mainfrom
copybara/v1.19.8
Nov 17, 2026
Merged

tphoney merged 4 commits into
mainfrom
copybara/v1.19.8

Conversation

@github-actions

Copy link
Copy Markdown

Copybara Sync - Release v1.19.8

This PR was automatically created by Copybara, syncing changes from the overmindtech/workspace monorepo.

Original author: TP Honey ([email protected])

What happens when this PR is merged?

  1. The tag-on-merge workflow will automatically create the v1.19.8 tag on main
  2. This tag will trigger the release workflow, which will:
    • Run tests
    • Build and publish release binaries via GoReleaser
    • Upload packages to Cloudsmith

Review Checklist

  • Changes look correct and match the expected monorepo sync
  • Tests pass (see CI checks below)

dylanratcliffe and others added 4 commits November 17, 2026 09:39
Stacked on #6913 (TASK-27.5, the other five roles). Review that first;
this PR's own changes are the 7 commits above it.

This promotes the last two change-analysis roles to GPT-6 for every
account, as Dylan decided in TASK-36. After it, no role has an
`Internal*` (FlagInternal) override.

| Role | Before (all accounts) | After (all accounts) |
|---|---|---|
| Primary (V6 investigator) | `gpt-5-4` (internal: `gpt-6-1-sol`) |
`gpt-6-1-sol` |
| BlastRadius | `gpt-5-mini` (internal: `gpt-5-4-mini-low`) |
`gpt-6-luna-medium` |

## Primary: Sol with the not-disproven guidance and the low cap

- **Sol:** the default `PrimaryProviderName` is now `gpt-6-1-sol`.
`InternalPrimaryProviderName` is removed.
- **Not-disproven guidance, on for everyone.** The investigator reports
a concrete mechanism that it tried to disprove and could not as a risk.
It also says what it checked. Its answer gains `risk_basis`
(`confirmed`, `not_disproven` or `none`).
- Until now this was gated on FlagInternal or the `not-disproven-risks`
flag. That gate is removed, so the guidance is always appended to the
system prompt and the investigator always uses the `risk_basis` response
schema.
- The `FlagNotDisprovenRisks` constant and the benchmark-only
`V6SpecificArgs.NotDisprovenRisks` switch are removed with the gate. The
PostHog flag can be deleted.
- Without this change, other accounts would get Sol with the type list
but no guidance. TASK-27.18 measured that combination: about 4 risks per
run, and no credential or namespace risks found.
- **Low cap, in code.** Only a risk with `risk_basis: confirmed` keeps a
level above low. A not-disproven, missing, `none` or unknown basis is
lowered to low (Dylan, TASK-27.21). The span records
`ovm.analysis.oneshot.riskBasis` and `notDisprovenLevelCapped`.
- **Citation stripping:** web-search citation markers are removed from
risk titles, descriptions and reasoning.
- **Type list:** the investigator's prompt lists the types it can query,
for every account. The stale "check the available types list" paragraph
is gone from both V6 prompts. The NetworkPolicy adapter now declares its
query methods, so it is included in the list.

TASK-27.21's `TestV6Benchmark` (mean of 3 runs, judge gpt-5.4;
bastion-sg-open-egress aborted in every run, so 9 scenarios per run):

| | gpt-5.5 (old internal) | Sol, this PR |
|---|---|---|
| Model cost per run | $18.89 | $4.96 (−74%) |
| Wall-clock per run | 1,502s | 996s (−34%) |
| Risks found | 16.7 | 14.0 |
| Golden accuracy | 0.83 | 0.59 |
| Not-disproven risks above low | | 0 of 29 |
| Risks with citation markers | | 0 |
| Query calls rejected as unknown type (3 runs) | | 2 (157 without the
list) |

- Credential and namespace hypotheses are raised as often as in
TASK-27.19.
- Dylan accepted the lower golden accuracy (TASK-36). The score is
mostly risk count against each scenario's expected range. The gap comes
from the bastion abort and from extra low "could not verify" risks on
update-gitops and verify-email, not from worse findings.
- Also accepted: confirmed single-replica risks are mostly rated high (8
of 13).

## BlastRadius: luna at medium effort, the v1 line and the stall guard

- **luna-medium:** a new `gpt-6-luna-medium` provider (`gpt-6-luna`,
reasoning effort medium) is the default `BlastRadiusProviderName`.
`InternalBlastRadiusProviderName` is removed.
- **v1 line:** `blast_radius/system.tmpl` gains one line. It tells the
model to call `record-risk` only for a concrete risk, never to record
"no impact". This fixes luna's "no impact" record-risk calls
(TASK-27.22).
- **Stall guard:** `WithStreamIdleTimeout(20s)` and
`WithStreamIdleMaxRetries(2)`. A stream that sends nothing for 20s is
retried, up to twice. This guards against the 42s to 62s stalls seen at
low effort in TASK-27.20 and TASK-27.22. 20s is above luna's p99 call
latency and well under the 60s hard timeout. No retry fired in
TASK-27.24's 1,071 calls, so the retry path has not been tested against
a live stall. It also caused no false retries.
- **Deadline:** `gpt-5-mini-2025-08-07` retires on **2026-12-11**, so
this has to ship before then. After this PR no default pin uses a
`gpt-5-mini*` provider. The providers stay registered.

TASK-27.24, a replay of 119 TASK-24 chunks with 3 repeats:

| | gpt-5-mini | luna-medium + v1 |
|---|---|---|
| Recall (95% CI) | 0.687 (0.61–0.74) | 0.798 (0.70–0.87) |
| Precision (95% CI) | 0.852 (0.74–0.92) | 0.995 (0.99–1.00) |
| "No impact" record-risk calls per repeat | 0 | 6.7 (25.7 for luna-low
+ v1) |
| Cost per 1,000 chunks | $1.75 | $0.54 |
| Latency p50 / p99 | 6.6 / 23.5s | 6.4 / 25.7s |

The recall and precision gains over gpt-5-mini both have CIs that
exclude 0.

TASK-27.25, `TestV6Benchmark` on the 4 scenarios that depend on
observations, 3 runs:

| | gpt-5-mini | luna-medium + v1 |
|---|---|---|
| Rollout/singleton risk reported (of 12 scenario-runs, hand-checked) |
11 | 10 |
| "No impact" hypotheses | 0 | 0 |
| Golden accuracy | 0.90 | 0.95 |
| Blast-radius model cost per run | $0.141 | $0.043 |
| Blast-radius time per scenario | 42.5s | 54.3s |
| Blast-radius call p50 / p95 | 2.7 / 8.2s | 3.6 / 11.9s |

Accepted cost (TASK-36): luna-medium adds about 12s of blast-radius time
per change analysis.

## Tests

- `TestLoadLLMConfigPins` now asserts the new defaults and that every
`Internal*` pin is empty.
- `TestAppendPromptGuidance` asserts that the not-disproven guidance is
always appended, last.
- The tests from the source commits cover the type list, the cap,
`risk_basis` and citation stripping.
- Local runs:
- `go test -race` passes for `service/config` and
`service/changeanalysis/...`, including the Postgres-backed tests
against the local apiserverdb.
  - `go test -race` passes for `k8s-source/adapters`.
  - golangci-lint v2.13.2 reports 0 issues on the touched packages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **High Risk**
> Rolls out new default LLM models and investigator risk semantics
(not-disproven + severity capping) to all customers, directly affecting
change-analysis cost, latency, and reported risk levels.
>
> **Overview**
> Ships **GPT-6 for the last two change-analysis roles** for every
account: V6 investigator **Primary** becomes `gpt-6-1-sol` and blast
radius becomes **`gpt-6-luna-medium`** (new provider with medium
reasoning, 20s stream idle timeout/retries). All `Internal*` model
overrides are cleared; config tests pin the new defaults.
>
> **Investigator behavior (always on, no feature flag):** not-disproven
risk guidance is appended to every system prompt; responses use
`risk_basis` (`confirmed` / `not_disproven` / `none`) and **code caps**
any non-confirmed real risk to **low**. Web-search citation junk is
stripped from risk text. The prompt gets a **sorted, order-stable
“available types”** section; `Query` tool validation errors reference
that list only when it’s present. **NetworkPolicy** k8s metadata now
declares query methods so it can appear in that list; a test enforces
all adapters declare methods.
>
> **Blast radius:** one prompt line—use `record-risk` only for concrete
risks, not “no impact.” **`FlagNotDisprovenRisks`** is removed from
feature flags.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
6c2cbb63311046756c7fcf04e074f381aa08f8fc. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
GitOrigin-RevId: 55a3189fac5b4ca84f6a78ae991f0dd4af6093f3
Mechanical dependency bumps only. These do not change functionality.

## Ready now (scan)
- cloud.google.com/go/dataplex v1.38.1 -> v1.39.0
- cloud.google.com/go/kms v1.36.1 -> v1.37.0
- cloud.google.com/go/run v1.24.1 -> v1.25.0
-
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cosmos/armcosmos/v4
v4.0.0 -> v4.1.0
- github.com/auth0/go-auth0/v3 v3.11.0 -> v3.12.0
- github.com/aws/aws-sdk-go-v2 v1.47.3 -> v1.48.0
- github.com/aws/aws-sdk-go-v2/config v1.33.9 -> v1.33.18
- github.com/aws/aws-sdk-go-v2/credentials v1.20.9 -> v1.21.0
- github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.20.3 -> v1.20.9
- github.com/aws/aws-sdk-go-v2/service/apigateway v1.52.1 -> v1.52.9
- github.com/aws/aws-sdk-go-v2/service/autoscaling v1.78.4 -> v1.82.0
- github.com/aws/aws-sdk-go-v2/service/cloudfront v1.75.0 -> v1.75.6
- github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.73.3 -> v1.76.0
- github.com/aws/aws-sdk-go-v2/service/directconnect v1.55.1 -> v1.56.0
- github.com/aws/aws-sdk-go-v2/service/dynamodb v1.70.3 -> v1.74.0
- github.com/aws/aws-sdk-go-v2/service/ecs v1.100.3 -> v1.105.1
- github.com/aws/aws-sdk-go-v2/service/efs v1.49.4 -> v1.51.6
- github.com/aws/aws-sdk-go-v2/service/eks v1.104.1 -> v1.105.6
- github.com/aws/aws-sdk-go-v2/service/elasticloadbalancing v1.41.4 ->
v1.42.0
- github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.63.4 ->
v1.64.0
- github.com/aws/aws-sdk-go-v2/service/kms v1.61.4 -> v1.62.6
- github.com/aws/aws-sdk-go-v2/service/networkmanager v1.50.4 -> v1.51.6
- github.com/aws/aws-sdk-go-v2/service/rds v1.130.3 -> v1.133.0
- github.com/aws/aws-sdk-go-v2/service/route53 v1.70.4 -> v1.71.0
- github.com/aws/aws-sdk-go-v2/service/s3 v1.114.3 -> v1.114.10
- github.com/aws/aws-sdk-go-v2/service/sesv2 v1.79.1 -> v1.79.9
- github.com/aws/aws-sdk-go-v2/service/sns v1.47.5 -> v1.48.0
- github.com/aws/aws-sdk-go-v2/service/sqs v1.52.4 -> v1.53.0
- github.com/aws/aws-sdk-go-v2/service/sts v1.51.4 -> v1.52.0
- github.com/aws/smithy-go v1.28.8 -> v1.28.9
- github.com/jackc/pgx/v5 v5.12.0 -> v5.13.0
- github.com/nats-io/jwt/v2 v2.8.2 -> v2.8.3
- github.com/nats-io/nats-server/v2 v2.15.2 -> v2.15.3
- github.com/resend/resend-go/v4 v4.12.0 -> v4.12.1
- github.com/zclconf/go-cty v1.19.0 -> v1.20.0
- gopkg.in/ini.v1 v1.67.3 -> v1.68.0

Modules raised by minimum version selection:
- github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op ->
v0.9.0-default-no-op
- github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.22 ->
v1.7.27
- github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.6 -> v1.5.12
- github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.6 -> v2.8.12
- github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.6 -> v1.5.12
- github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.21
-> v1.13.25
- github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.11.7 ->
v1.11.13
- github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery
v1.13.6 -> v1.13.12
- github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.6 ->
v1.14.12
- github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.20.6 ->
v1.20.12
- github.com/aws/aws-sdk-go-v2/service/signin v1.10.4 -> v1.13.1
- github.com/aws/aws-sdk-go-v2/service/sso v1.38.4 -> v1.39.5
- github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.4 -> v1.44.5
- github.com/klauspost/compress v1.20.3 -> v1.20.4
- github.com/nats-io/nkeys v0.4.16 -> v0.4.17
- golang.org/x/oauth2 v0.37.0 -> v0.38.0
- golang.org/x/sys v0.49.0 -> v0.50.0

Left out: `cloud.google.com/go/compute` stays at v1.70.0. v1.75.0
removes `ReturnPartialSuccess` from the `AggregatedList*Request` types
used in `sources/gcp/manual`, so the bump needs a code change.

Left out: `github.com/openai/openai-go/v3` stays at v3.70.0. With
v3.99.0, `TestCalculateBlastRadius` failed 2 of 3 runs (the model
returned no affected resources), so the client bump will be evaluated
separately from this mechanical batch.

## Devcontainer build fix
`devcontainers/features#1744` made the go, terraform, github-cli and
node feature `install.sh` scripts source helpers from a `scripts/`
directory next to them, so piping them into bash has failed on `main`
since 2026-11-13. `.cursor/devcontainer-build/Dockerfile` now extracts
those four feature directories from the upstream tarball and runs each
`install.sh` in place.

## Vulnerability fixes (High/Critical)
- node 24.22.1 -> 24.22.3 (Critical CVE-2026-123864; High
CVE-2026-123863, CVE-2026-123865, CVE-2026-123866, CVE-2026-123867,
CVE-2026-123868, CVE-2026-123872, CVE-2026-123874, CVE-2026-123875,
CVE-2026-123877, CVE-2026-126051)
- `services/frontend/Dockerfile` (`node:24.22.3-alpine3.23`, both
stages)
- `services/frontend/Dockerfile.playwright` (`node:24.22.3-trixie-slim`)
- Copied pins: `NODE_VERSION` in
`plugins/private-evals/container/Dockerfile` and
`.cursor/devcontainer-build/Dockerfile`, and `node-version` in the CI
workflows under `.github/workflows/` and `plugins/.github/workflows/`

## Test plan
- [x] `go build ./...`
- [x] `go vet` on `sources/gcp`, `services/until-backend`, `go/`
- [ ] CI green on this draft

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Wide Go client bumps (especially AWS and Auth0) and uniform Node
runtime changes affect build, CI, and containers; risk is limited by
mechanical version-only updates but warrants a green CI pass.
>
> **Overview**
> This PR is primarily a **mechanical dependency refresh**: `go.mod` /
`go.sum` pick up patch and minor releases across cloud clients (GCP,
Azure Cosmos, AWS SDK v2), Auth0, `pgx`, NATS, Terraform’s `go-cty`, and
related transitive modules, with no application code changes in the
diff.
>
> **Node.js is pinned to 24.22.3** everywhere it was 24.22.1—CI
`setup-node` jobs, frontend and Playwright Docker bases, private-evals
and devcontainer `NODE_VERSION`—to address reported High/Critical Node
CVEs.
>
> The **devcontainer image build** no longer pipes `install.sh` from
GitHub; it downloads the devcontainers features tarball, extracts the
go/terraform/github-cli/node feature trees, and runs each script in
place so sibling `scripts/` helpers resolve correctly, then removes the
temp tree after the node install.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
75e8c8ba3a691dbc26a58c69b19a27cb03240121. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Cursor <[email protected]>
GitOrigin-RevId: 128c73cec5fbe38f5c75049f491d1a898f7f648f
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
Fixes
[ENG-6865](https://linear.app/overmind/issue/ENG-6865/exclude-user-data-from-azure-vm-and-scale-set-items).

Azure VM and scale set converters copied the full compute model into SDP
attributes, including `properties.userData` /
`virtualMachineProfile.userData` and OS-profile `customData`. Those
bootstrap payloads can contain tokens, keys, or passwords, and were then
stored in the account graph and snapshots.

## Changes

- Add `shared.ToAttributesWithPresence` to replace named JSON paths with
`true` when the field exists. Add/remove of bootstrap data remains
visible; the payload itself is not stored.
- A digest is intentionally not stored. Unkeyed SHA-256 and HMAC keyed
by the public item scope still let someone confirm a guessed low-entropy
payload from graph data, and Azure converters have no tenant HMAC secret
from key management.
- Redact VM `properties.userData` and `properties.osProfile.customData`
before building attributes.
- Redact scale set `properties.virtualMachineProfile.userData` and
`properties.virtualMachineProfile.osProfile.customData` the same way.
Fixing the VM converter alone does not cover scale sets.
- Unit tests assert the sentinel payloads never appear in attributes and
that the fields become `true`.
- Adapter-author guidance and a BUGBOT rule so new converters do not
reintroduce the leak.

Run-command scripts and extension protected settings are out of scope
for this ticket.

## Testing

- `go test -race ./sources/shared/ -run TestToAttributes`
- `go test -race ./sources/azure/manual/ -run
'TestComputeVirtualMachine|TestComputeVirtualMachineScaleSet'`
- `golangci-lint run ./sources/shared/ ./sources/azure/manual/`
<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-05689b44-472b-44a5-9b87-58d40f7eb19c?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-05689b44-472b-44a5-9b87-58d40f7eb19c&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

Fixes ENG-6865

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Touches secret handling in graph snapshots for core compute adapters;
behavior change is intentional but could affect change detection
expectations for bootstrap fields (presence only, not content).
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
68d949e2e9d344bbac6960492891d22acf654f10. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Cursor Agent <[email protected]>
GitOrigin-RevId: 269a0b4a1f4395d91a7253c26ab3240e8f5fb9a2
@tphoney tphoney closed this Nov 17, 2026
@tphoney tphoney reopened this Nov 17, 2026
@tphoney
tphoney merged commit 4f59c04 into main Nov 17, 2026
2 checks passed
@tphoney
tphoney deleted the copybara/v1.19.8 branch November 17, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants