Skip to content

Evaluate generated effects through shared typed predicates - #1996

Draft
abrichr wants to merge 3 commits into
agent/1891from
agent/1986
Draft

abrichr wants to merge 3 commits into
agent/1891from
agent/1986

Conversation

@abrichr

@abrichr abrichr commented Nov 5, 2026

Copy link
Copy Markdown
Member

Generated save contracts can now bind task, before and after observations to typed predicates in the shared expression language. The effect profile adds immutable records, keyed collections and exact allowed-delta checks to the existing parser, type checker and evaluator, with one input/work budget across a model. The published text-transform manifest remains unchanged.

The shadow consumer checks independently supplied reader scope, freshness and source identities, preserves prior refutations, and returns its candidate separately from canonical state. Replay admission requires complete independently labelled history against matching model, language, schema, reader and source identities. Tests cover unintended changes, missing evidence, budget exhaustion and signed-receipt isolation.

Validation on WSL: 209 targeted tests passed; focused Ruff lint/format and safety-style mypy checks passed. Engine generality and release consistency checks also passed.

Depends on #1983.
Closes #1986.

No model-construction or real-app qualification study ran. The general-profile defects reported on #1983 remain its owner’s work; this effect profile excludes regex and fuzzy operators. This PR remains blocked on that dependency.

@abrichr abrichr added the review-needed Needs an independent review from another vendor's agent label Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-needed Needs an independent review from another vendor's agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant