Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 66 additions & 38 deletions .agents/skills/verification-handoff-prototype/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: verification-handoff-prototype
description: Carries an instruction change from owner-confirmed goals through review and authorized Behavior Diff verification to a decision-ready recommendation. Use when the owner wants to hand off verification of a change or PR without coordinating the agent's next steps.
description: Carries an instruction change from one owner-approved handoff proposal through review and Behavior Diff verification to a decision-ready recommendation. Use when the owner wants to hand off verification of a change or PR without coordinating the agent's next steps.
---

# Verification handoff prototype
Expand All @@ -13,16 +13,26 @@ Resolve bundled files relative to this loaded skill, never the target repository

## Ownership and authority

- The owner has three touchpoints: the request, one handoff proposal, and the
adoption decision. Skip the proposal when the request already confirms the
question and authorizes every run term. Any other owner input is an exception.
- Own discovery, planning, tool selection, preflight, execution, bounded recovery
and interpretation. Never ask the owner to select evidence IDs, design tests,
debug the harness or remind you to take the next agreed step.
- Ask only for material goal corrections, unavailable domain facts, or missing
authority. Each interruption states the concrete decision, recommendation and
consequence. Continue useful independent work while awaiting that answer.
- Before paid trials, obtain approval covering purpose, trial count, model/provider,
cost estimate or explicit budget, and outgoing data. Reuse approval within that
scope; changes to scope or budget need new approval. Editing this skill is not
run approval. Do not send private evidence to a new provider without consent.
debug the harness, choose a fallback model, ask what a report means or remind
you to take the next agreed step.
- Ask outside the proposal only for material goal corrections, unavailable domain
facts, or authority beyond approved terms. Each interruption states the concrete
decision, recommendation and consequence. Never end a turn by asking what to do
next or by giving the owner a phrase to type.
- Model-backed preparation, trials and assessment need approval of these terms:
purpose, trials per side, retry allowance, preparation/trial/assessment models
with named fallbacks, provider and account, cost estimate or quota boundary, and
outgoing data. An approval given before the terms were stated covers only the
terms the owner named. Put the remaining terms in the proposal and wait for
approval before any model-backed step. Within approved terms, continue without
further prompts. Switch only to a named fallback; changes outside the terms need
new approval. Editing this skill is not run approval. Do not send private
evidence to a new provider without consent.
- No source edits, global configuration writes, publication, adoption or merge are
implied. Accepting findings or answering a study question records a decision
only. A separate explicit operational request must name the action and target;
Expand All @@ -33,45 +43,63 @@ Resolve bundled files relative to this loaded skill, never the target repository

## Workflow

1. **Understand the change.** Inspect the selected diff, relevant unchanged rules
and available discussion. Pin revisions and source paths; prepare historical
inputs through Behavior Diff's supported provenance and isolation procedure.
Do not require the owner to assemble an evidence packet.
2. **Confirm the question once.** Propose concise intended outcomes, preserved
requirements and bounded scope with explicit/inferred sources. Distinguish
added rule value from reliability in the owner's current environment. Reuse
prior confirmation; ask only about material uncertainty. Keep study questions
out of this gate. Goal corrections require reassessing affected evidence.
1. **Understand the change and route it without model calls.** Load the
available `behavior-diff` skill. Inspect the selected diff, relevant unchanged
rules and available discussion. Pin revisions and source paths. Then settle the
route with deterministic checks: fixture fit, file mapping, host
authentication, model availability and isolation, as
[execution handoff](references/execution.md) describes. Do not require the
owner to assemble an evidence packet or confirm a route the repository answers.
2. **Send one handoff proposal.** In one message, propose the intended outcomes,
preserved requirements, tested scope and route, plus the run terms above, with
explicit/inferred sources and a recommendation. Distinguish added rule value
from reliability in the owner's current environment. Keep study questions out.
Reuse confirmation and authorization already given. A goal correction requires
reassessing affected evidence and resending only the changed terms.
3. **Review, then prepare verification.** Inspect instructions and saved evidence.
Review completion is not verification completion. For changed agent behavior,
proceed directly to preparing a Behavior Diff comparison; do not wait for
“now run Behavior Diff.” Existing evidence can satisfy the question only when
revisions, conditions and coverage match; explain that conclusion. Report a
material ambiguity without editing the source or silently testing a repaired
version. Preserve the agreed question unless the owner explicitly changes it.
4. **Preflight before spending.** Load the available `behavior-diff` skill and
follow [execution handoff](references/execution.md). Own fixture preparation,
host compatibility, scenario reach and confound checks. Present any missing
run authorization as one concrete bounded proposal, not an open-ended menu.
5. **Run and assess.** Once preflight and authorization are satisfied, launch and
follow through to raw actions, returns and artifacts. Separate setup failures,
missed cases and unavailable assessment from behavioral failures. Recover
within the approved boundary; never rerun merely to obtain a difference.
6. **Return a decision-ready result.** Lead with retain/revise/hold and why. State
what changed versus Before, what stayed intact, what remains unknown, and
whether additional instructions showed added value. Link inspectable evidence
proceed directly to preparing the Behavior Diff comparison. Existing evidence
can satisfy the question only when revisions, conditions and coverage match;
explain that conclusion. Report a material ambiguity without editing the source
or silently testing a repaired version. Keep the agreed question.
4. **Preflight before spending.** Follow the execution handoff. Own fixture
preparation, host compatibility, isolation, scenario reach and confound checks.
A blocker with an in-scope route is work to do, not a stopping point. Ask again
only when preflight needs terms outside the approval.
5. **Run and assess.** After approval, launch and follow through to raw actions,
returns and artifacts. Monitor status after each launch and handle failed runs
as the execution handoff describes. Separate setup failures, missed cases and
unavailable assessment from behavioral failures. Never rerun merely to obtain
a difference.
6. **Return a decision-ready result.** Lead with retain, revise or hold and why.
State what changed versus Before, what stayed intact, what remains unknown, and
whether additional instructions showed added value. Disclose how files were
mapped and any generated-report claim you corrected. Link inspectable evidence
after the conclusion. Equal outcomes mean no observed added benefit under
these conditions, not proof that a rule is universally unnecessary. Simulated
behavior does not prove real-service reliability. Do not wait for the owner
to ask what the report means. Do not merge.
to ask what the report means.

## Adoption questions

Merge is the owner's decision, not a verification outcome. When asked whether to
merge, restate the retain/revise/hold result and name the merge-readiness checks
that did and did not run, such as code review, CI and mergeability. A summary of
intended changes is not a code review. Do not upgrade the recommendation under a
leading question. Do not merge.

When the owner asks whether the journey needed them, answer from the session:
list each owner input and whether it was necessary, without agreeing to a framing
the record contradicts.

## Completion and blockers

Finish with the bounded question answered, or an evidence-backed blocker naming
what failed, what was tried, what remains unverified and the smallest recommended
resolution. Missing capability is not permission to fake a worker or shrink scope.
An intermediate review, helper result or successful process exit is not completion.
Keep the exact run handle and private continuation context for interrupted work.
resolution. Stop only when no in-scope route exists. Missing capability is not
permission to fake a worker or shrink scope. An intermediate review, helper
result or successful process exit is not completion. Keep the exact run handle
and private continuation context for interrupted work.

After delivery, optional feedback may ask what coordination the owner still had
to do. Do not require an interview to finish. Distinguish necessary owner choices
Expand Down
109 changes: 84 additions & 25 deletions .agents/skills/verification-handoff-prototype/references/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,29 @@ source; elsewhere use the discovered installed skill. Never silently mix version
Missing tooling is a concrete prerequisite to resolve or report, not permission
to invent a substitute runner.

## Handoff proposal

Settle the route before writing the proposal, so it does not stop on a fixture
question the repository already answers. Use one message shaped like this
synthetic example:

```text
Question: does the change make the agent <intended outcomes> while keeping
<preserved requirements>? Route: ordinary-agent Before/After comparison of
<mapped files> against a local simulated <service>. Not tested: <live services>.
Run: 3 Before + 3 After trials, at most one startup retry. Models:
preparation and trials <primary>, or <fallback> if <primary> is rate limited
or unavailable; assessment <assessment model>, no fallback approved.
Provider/account: <host, sign-in type>. Cost/quota: <estimate or boundary,
including one fallback relaunch>.
Outgoing data: pinned instruction files and fixture data to <provider>; no new
provider.
Recommendation: approve as proposed.
```

Name a fallback for each model role, or state that none is approved. A budget
alone never authorizes a different model.

## Prepare a meaningful comparison

- Follow Behavior Diff's historical source isolation, purpose derivation, scenario
Expand All @@ -27,17 +50,25 @@ to invent a substitute runner.
## Route by the behavior that must execute

Distinguish an application's runbook decisions from actual workflow-engine state
or inter-agent handoffs. A mention of a worker or gate is not evidence that a Go
source project is needed. When Behavior Diff directs you to its Spacedock
reference, inspect the rule and fixture prerequisites before invoking its builder.
Do not apply a source-repository fixture to an unrelated application repository.

If the loaded skill requires a fixture that cannot represent the requested
behavior, report that specific contract/capability mismatch with a recommended
resolution. Do not silently ignore the downstream contract or replace a real
worker handoff with a scripted reply. A general decision scenario is valid only
when it answers the agreed question; narrowing away a real handoff requires owner
approval. Do not ask the owner to diagnose the mismatch.
or inter-agent handoffs. Decide from the target repository, not from words in its
instructions: a workflow engine is under test only when the repository contains
its source, configuration or state, or the owner says it runs one. A mention of a
worker, gate or Spacedock is not evidence that a Go project is needed. When
Behavior Diff directs you to its Spacedock reference, inspect the rule and fixture
prerequisites before invoking its builder. Do not apply a source-repository
fixture to an unrelated application repository.

If the repository does not run the engine, the ordinary-agent decision comparison
is the correct route, not a narrowed scope; propose it without first reporting a
blocker. If the repository does run it and the loaded skill's fixture cannot
represent the requested behavior, report that contract/capability mismatch with a
recommended resolution. Do not replace a real worker handoff with a scripted
reply; narrowing away a real handoff requires owner approval. Do not ask the owner
to diagnose the mismatch.

A multi-file change must keep its file mapping. If the runner accepts only one
target, a combined file is a disclosed test construction: state it in the
proposal and result, and do not present it as the repository layout.

## Check compatibility and confounds before launch

Expand All @@ -57,40 +88,68 @@ boundary and verify it with harmless probes. Disabling injection alone is not
filesystem isolation. Any model-backed probe belongs in the approved run budget.
Do not claim complete isolation if a relevant channel cannot be checked.

Run the fresh historical preparer as a host subprocess with its own isolated
context. Use an in-session subagent only when the host verifiably gives it no
injected session memory. If Behavior Diff provides no isolation for the host, use
a disposable launcher in the private run root, for example a wrapper first on
`PATH` that applies process-local flags disabling memory and plugins and denies
reads of memory stores and the original checkout. Prefer the host's existing
sign-in. If a private host home is unavoidable, disclose it in the proposal, keep
any credential copy mode 0600 inside the mode-0700 run root, never print it, and
delete it when the run ends or is interrupted. Verify the launcher with harmless
non-model probes before model use, keep it with the evidence, and record it in
the result. Never change the owner's global host settings instead.

Follow Behavior Diff's preflight and frozen-handoff helpers rather than rebuilding
their validators. Simulation checks prove the fixture's mechanics, not that a
real website or workflow engine works. Run a real-service check only with its
specific authorization; do not replace the agreed question with a simulation.

## Authorization, execution and recovery

This handoff requires the bounded live-run approval in SKILL.md before entering
Behavior Diff's automatic-launch step. Do not treat its normal no-extra-prompt
rule as overriding this boundary. Once approved, carry execution through without
another stage-by-stage permission loop. A study answer or “accept limitations”
is never run or merge authorization.
Owner approval of the handoff proposal, or a request that already authorizes
every term, supplies the bounded run approval required before Behavior Diff's
automatic-launch step. Sending the proposal is not approval. Do not treat Behavior
Diff's normal no-extra-prompt rule as overriding this boundary. Once approved,
carry execution through without another stage-by-stage permission loop. A study
answer or “accept limitations” is never run or merge authorization.

Keep the printed run handle, frozen inputs, exact host/model settings and failed
attempts. Diagnose tooling failures from their outputs. Repair routine local
setup within the authorization, without editing the target change or global
settings. Retry only when the approved budget includes it and the cause is
understood; otherwise propose one bounded recovery. Do not retune after a null
result. A changed scenario or isolation condition is a new comparison, not a
replacement result. Stop repeated unsuccessful recovery with a concrete blocker.
attempts. In the launch message, say that the runner opens its report by itself
when the run ends. Check status right after launch and keep monitoring while the
run or any trial is live or uncertain (`pending`, `preparing`, `running`,
`normalizing`, `uncertain`); zero completed trials in those states is normal, not
a failure. When the run ends or its runner is lost (`owner_lost`) with any trial
not `finished`, tell the owner at once, even if the report is already open, and
name each failed phase or trial state that status reports. Say that no model ran
only when startup errors and empty native traces show it; a failure during
normalization can follow model output. Preserve that run, diagnose its
outputs, and relaunch only within the retry allowance. Repair routine local setup
without editing the target change or global settings. On rate limiting or model
unavailability, retry once, then switch to the named fallback and record the
switch; never idle on repeated failures. A fallback switch relaunches the whole
comparison on one model, does not consume the startup retry, and is a new
comparison rather than a mix of models. Without a named fallback, send one
bounded proposal. Do not retune after a null result. A changed scenario or
isolation condition is a new comparison, not a replacement result.
Stop repeated unsuccessful recovery with a concrete blocker.

## Interpretation and return

Audit raw actions, returns and artifacts for each case, including minority results.
If automated assessment is rejected, retain the report and distinguish a documented
raw-evidence assessment from the unavailable automated result. Never re-label it
as an automatic pass. Distinguish rule effects from pre-existing instructions,
shared memory, task hints and environment failures.
shared memory, task hints and environment failures. Check the generated headline
against raw decisions; a wording difference is not a changed outcome. Correct it
in your assessment without editing the report.

Deliver the recommendation without requiring “what does the report mean?”:
what question was tested; added benefit observed or not demonstrated; regressions
and preserved behavior; decisive evidence; unresolved scope and its decision impact.
A process change without an outcome change is not automatically an improvement.
Recommend retain/revise/hold at the tested scope. Do not infer universal redundancy
from a few equal trials or promise the next real run will succeed. Finish the
Recommend retain/revise/hold at the tested scope; never merge, and do not turn
that result into a merge recommendation. Do not infer universal redundancy from
a few equal trials or promise the next real run will succeed. Finish the
verification handoff before offering optional process feedback; never operate the
PR in response to that feedback.
Loading
Loading