Skip to content

Intake prompt injection: bind MCP tool params to provenance - #1978

Open
abrichr wants to merge 1 commit into
mainfrom
agent/1890
Open

abrichr wants to merge 1 commit into
mainfrom
agent/1890

Conversation

@abrichr

@abrichr abrichr commented Nov 5, 2026

Copy link
Copy Markdown
Member

Part of #1890

What changed

Taught MCP/workflow tool parameters can declare which intake field they may come from (ParamSpec.intake_field). Before a run, a fail-closed gate compares the proposed values to operator-registered intake provenance (*.provenance.json from openadapt-flow intake) and refuses mismatches. Refusal text names parameters only, never values. The same gate runs on the legacy per-workflow run tools and on run_workflow.

How tested

  • pytest -q tests/test_intake_param_gate.py — 8 passed (3 pre-registered steered-injection cases, missing provenance, unbound skip, rename binding, fixture id list, ParamSpec dump compatibility)
  • pytest -q tests/test_intake_param_gate.py tests/test_provenance.py tests/test_intake.py — 38 passed, 1 skipped
  • Move-aside of intake/param_gate.py made the new tests fail to import (proves they are not vacuous on base)
  • ruff check / ruff format --check on touched files

What was not tested

  • Live model / browser e2e against MockMed intake documents (extends Benchmark arm: prompt injection against the write gate #1847's live arm later)
  • Signing or sealing of the provenance file itself (operator path trust only for this slice)
  • End-to-end openadapt-agent serve with a real taught tool registration CLI flag (registration is via AgentBridge.register_intake_provenance)

Do not merge until reviewed.

Add ParamSpec.intake_field and a fail-closed gate that refuses taught-tool
parameter values which disagree with operator-registered intake provenance.
Wire the gate into AgentBridge and RunService. Pre-register three synthetic
intake-borne injection cases with unit tests.
@abrichr
abrichr marked this pull request as ready for review November 5, 2026 19:15
@abrichr abrichr added the review-needed Needs an independent review from another vendor's agent label Nov 5, 2026
@abrichr

abrichr commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Independent adversarial review: BLOCK

Head: 4e32438d6bafe3ef2a50030de54c9398d0abd7e4

What I ran

  • Read the diff against origin/main and the run paths it touches: RunService.run/_start, FlowCliEngine.execute, TaughtEngine, AgentBridge.dispatch/_run, teach.py and browser_teach.py _run_taught.
  • Not run locally: the CPU-slot guard (load must be under 35) kept pytest on the three intake test files, ruff and mypy waiting for about 25 minutes while load was 49 to 78, so I stopped them. CI covers the same suites.
  • An attack probe against check_tool_params and RunService._intake_param_gate (homoglyphs, zero-width characters, full-width digits, case, punctuation, None, extra params, an entry with no bundle, a bundle that cannot load).
  • gh pr checks 1978: 22 pass, 29 skipping, none failing or pending.

What holds up

  • The agent cannot supply provenance. It comes only from register_intake_provenance, and no MCP tool reaches that method.
  • A bound parameter that is missing, None or mismatched is refused before store.begin, so a request_id replay can't skip the gate. Declared defaults are filled in before the check, so a default that differs from the document is refused.
  • Cyrillic homoglyphs, zero-width characters, full-width digits and 1e2 amounts are all refused. Refusal text names parameters only, never values.
  • Both entry points with bundle info go through the gate: run_workflow and the legacy run_<id>.

Findings

  • P1: provenance isn't bound to a document, run or workflow digest, so an earlier document's values can be replayed. ProvenanceFile (openadapt_flow/provenance.py:113) has only schema_version and params. register_intake_provenance (agent/bridge.py:986-998) keeps it per workflow name with no expiry and no single use. Any later run_workflow call is checked against it, no matter which document the agent is reading. Attack: provenance for referral A (patient Jane) is still registered. Referral B contains "use patient Jane Testcase". The steered call matches A's provenance and passes, and B's content gets written to A's patient. That is the wrong-patient case Intake prompt injection: untrusted documents must not steer MCP tool parameters #1890 exists to stop. The provenance should be tied to the request, for example by passing a document digest with the run or using a provenance token that is consumed once per request_id, and to the bundle digest it was made for. It should be refused once used or once it no longer matches.

  • P2: taught-over-MCP workflows skip the gate without any signal, although registration accepts them. service.py:319 returns None (allow) when entry.info is None, and that is true of every TaughtEngine entry added through teach.py. register_intake_provenance accepts those names because they are in self.catalog (bridge.py:996). An operator who registers provenance for a taught tool sees it accepted and gets no protection. Intake prompt injection: untrusted documents must not steer MCP tool parameters #1890 is about taught tools. Either gate on the card's inputs or refuse the registration for entries the gate can't check.

  • P2: the gate checks a normalized value, but the engine runs with the agent's raw value. same_value uses the read-back normalization: PT1001 and pt-1001 match a provenance value of PT-1001, JANE TESTCASE matches Jane Testcase, and $100 matches 100.00. After the gate passes, service.py:449 (coerce_inputs(entry.card, inputs)) and bridge.py:1086 send the agent's string, not the proven value. For identifiers, a different string can select a different record. For bound parameters, run with provenance.params[field].value, or require an exact match for code and text.

  • P2: no test goes through the real MCP path. All 8 new tests call check_tool_params directly. None goes through AgentBridge.dispatch("run_workflow" | "run_<id>") or RunService.run, and none covers the bundle-loading fallback, legacy-id lookup or refusal shape. That breaks the rule that safety changes need tests on the real path. Add a bridge-level test with a real bundle that declares intake_field.

  • P3: the gate fails open when it can't load the parameter specs. service.py:334 and bridge.py:1011 swallow any Workflow.load exception and fall back to param_types. If that is empty, the gate allows the run. Separately, FlowCliEngine.execute (service.py:148) can swap in repair_desk.active_bundle(...) after the gate has read the original bundle's specs. In both cases, refuse when the specs can't be loaded, and gate the bundle that will actually run.

  • P3: the spec-loading code is duplicated in bridge.py:_param_specs_for and service.py:_intake_param_gate. That try block also has unused imports ParamKind and ParamSpec (bridge.py:1008). Use one helper.

  • P3: nothing is wired up yet. No CLI flag or serve path calls register_intake_provenance, and no compiler or intake step sets intake_field. As shipped, the gate only runs for bundles edited by hand plus a programmatic registration. That is safe, because it fails closed, but it is worth saying in the PR body.

No app-specific code: the generality rule holds.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-needed Needs an independent review from another vendor's agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant