Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
44cfca6
test: require result-aware migration head
Air000000 Sep 20, 2026
d69b904
feat: add result-aware AgentRun provenance migration
Air000000 Sep 20, 2026
06f12f0
feat: persist AgentRun source summary provenance
Air000000 Sep 20, 2026
20ea6e4
feat: expose AgentRun source summary provenance
Air000000 Sep 20, 2026
40856bf
feat: round trip AgentRun source summary provenance
Air000000 Sep 20, 2026
1e514fb
feat: return AgentRun source summary provenance
Air000000 Sep 20, 2026
54baa68
test: expect result-aware migration head
Air000000 Sep 20, 2026
0cfd7bd
test: expect result-aware migration head
Air000000 Sep 20, 2026
e147f7b
test: expect result-aware migration head
Air000000 Sep 20, 2026
d11ac32
test: expect result-aware migration head
Air000000 Sep 20, 2026
b74c02f
test: include AgentRun source summary column
Air000000 Sep 20, 2026
7e0eb5d
test: cover AgentRun provenance migration contract
Air000000 Sep 20, 2026
799c34a
test: expose null provenance for initial AgentRun
Air000000 Sep 20, 2026
4e253f9
fix: shorten AgentRun source summary FK name
Air000000 Sep 20, 2026
e83f615
fix: shorten AgentRun source summary FK name
Air000000 Sep 20, 2026
51eb237
fix: shorten AgentRun source summary FK name
Air000000 Sep 20, 2026
39cb9c3
test: define ResultContext compiler contract
Air000000 Sep 20, 2026
cfbde9a
feat: add Agent-owned ResultContext contract
Air000000 Sep 20, 2026
9ab38b7
feat: compile bounded ResultContext for follow-up runs
Air000000 Sep 20, 2026
fbab443
test: cover ResultContext global message bound
Air000000 Sep 20, 2026
c49dbb6
style: format ResultContext test fixture
Air000000 Sep 20, 2026
8c1834d
fix: narrow compiled system context type
Air000000 Sep 20, 2026
266f67a
fix: serialize frozen Proposal mappings in ResultContext
Air000000 Sep 20, 2026
b17ff79
test: serialize frozen Proposal mapping for validation case
Air000000 Sep 20, 2026
c817ed7
feat: add result-aware follow-up binding
Air000000 Sep 20, 2026
e3116ec
feat: add follow-up AgentRun service API
Air000000 Sep 21, 2026
fdb9738
feat: add same-experiment Agent proposal revision
Air000000 Sep 21, 2026
ae079be
feat: materialize follow-up Agent proposals
Air000000 Sep 21, 2026
4e8bcab
test: prove result-aware two-iteration closure
Air000000 Sep 21, 2026
5e1c735
test: prove raw metric stays outside result-aware context
Air000000 Sep 21, 2026
ab73e2c
test: use unique raw metric sentinel
Air000000 Sep 21, 2026
e8caaf6
test: synchronize result-aware batch lock race
Air000000 Sep 22, 2026
631b761
docs: close P1 result-aware iteration
Air000000 Sep 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 11 additions & 45 deletions DOCUMENT-STATUS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# LabPilot 文档状态

> 当前状态:R0 仍为 V1 rebaseline authority root;Slice 1 已 Accepted;Slice 2 bounded MVP 已验证并集成到 `main`,production hardening deferred;Slice 3 Result Plane implementation 已完成。Competition Core Demo Runtime/E2E 已在产品基线 `c92bf45b63053b1ae00209660ed351cf309f91c8` 上验收并冻结,competition console 已合并回 `main`,并行 Core Demo 工作中已验证的 exact read-only API workspace acceptance contract 也已收敛。比赛后 remaining-V1 已重新进入 planning gate:当前只授权 P1 Result-Aware Experiment Iteration 的设计 focused review,尚未授权实现。
> 当前状态:R0 仍为 V1 rebaseline authority root;Slice 1 已 Accepted;Slice 2 bounded MVP 已验证并集成到 `main`,production hardening deferred;Slice 3 Result Plane implementation 已完成。Competition Core Demo Runtime/E2E 已验收并冻结。P1 Result-Aware Experiment Iteration implementation completed and accepted; deterministic two-iteration acceptance + regression/full-CI evidence closed。

| 文档 | 状态 | 用途 |
|---|---|---|
Expand Down Expand Up @@ -112,7 +112,12 @@ Slice-specific detailed plans must record their exact planning baseline commit a
- P1 Result-Aware Iteration detailed implementation plan: **APPROVED / CURRENT**
- P1 Result-Aware Iteration plan doc: `docs/superpowers/plans/2026-09-20-labpilot-p1-result-aware-iteration.md`
- P1 Result-Aware Iteration focused plan review: **APPROVE**; P1-PLAN-001 CLOSED; direct regressions NONE
- P1 Result-Aware Iteration Product implementation: **AUTHORIZED under approved detailed plan**
- P1 Result-Aware Iteration implementation: **COMPLETED / ACCEPTED**
- P1 last production-code commit: `ae079be252fd62eff415ef040af4228ab45e2d61`
- P1 tranche acceptance/test baseline: `e8caaf668fed2b5e5878393fa0c47c001ba218e9`
- P1 deterministic two-iteration acceptance: **PASS**
- P1 Task 8 focused regression: **PASS**
- P1 Task 8 GitHub CI #312: **PASS**

Under Sequencing Amendment v0.3.1, the bounded Slice 2 MVP closeout is sufficient to unlock Slice 3. It does not retroactively complete the deferred Slice 2 production-hardening campaign or establish Slice 2 production acceptance.

Expand All @@ -122,51 +127,12 @@ The Git commit containing this status transition is the exact Slice3 implementat

The competition workstream is closed. Its Runtime/E2E evidence and UI are frozen unless a concrete regression or demo-critical defect is demonstrated.

The detailed plan is approved. The next authorized remaining-V1 action is **implementation under the approved task sequence**:
P1 Result-Aware Experiment Iteration implementation is accepted. Repository integration is completed through the normal reviewed PR process; any further remaining-V1 tranche requires its own planning/authority gate.

```text
Approved P1 Result-Aware Experiment Iteration Design
→ approved detailed implementation plan
→ P1-A Tasks 1–4
→ focused P1-A implementation review
→ P1-B Tasks 5–7
→ focused P1-B implementation review
→ final regression/full-CI gate
```

Current proposed design:

```text
exact BatchSummaryRevision
→ deterministic ResultContext
→ Follow-up AgentRun
→ Stop | Request Human Input | Follow-up Proposal
→ existing deterministic ProposalVerification
→ Human Approval
→ next immutable ExperimentVersion in the same Experiment
```

Focused design and focused plan review are closed. Product implementation is authorized only within the approved P1-A/B task sequence and review gates.

Explicitly not authorized in this first P1 tranche:

- Slice 4 as a new roadmap label;
- Multi-Agent;
- controlled/autonomous code modification;
- editable Worktree;
- cross-Experiment history search;
- automatic Batch start after Agent proposal;
- LLM semantic verifier;
- new Runtime-hardening campaign;
- competition UI redesign.

The proposed design remains subordinate to:

```text
V1 Scope v0.3
> Architecture 10 v0.3
> Master Roadmap v0.3
> approved P1 Result-Aware authority after review
implementation accepted
→ reviewed PR integration
→ separate planning/authority gate for any further tranche
```

Competition closure still does not complete deferred Slice 2 production hardening or establish Slice 2 production acceptance.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# LabPilot P1 Result-Aware Experiment Iteration — Implementation Closeout

Date: 2026-09-22

Status: Accepted

## 1. Authority

- Design: `docs/superpowers/specs/2026-09-20-labpilot-p1-result-aware-iteration-design.md`
- Implementation plan: `docs/superpowers/plans/2026-09-20-labpilot-p1-result-aware-iteration.md`
- PR: #9
- Last production-code commit: `ae079be252fd62eff415ef040af4228ab45e2d61`
- Final pre-closeout acceptance/test baseline: `e8caaf668fed2b5e5878393fa0c47c001ba218e9`

## 2. Implemented loop

```text
Experiment E
→ ProposalRevision R1
→ deterministic verification
→ Human Approval
→ ExperimentVersion V1
→ Batch B1
→ deterministic Result processing
→ exact BatchSummaryRevision S1
→ Follow-up AgentRun A2
→ Stop
OR
→ Follow-up ProposalRevision R2 in the same Experiment
→ deterministic verification
→ Human Approval
→ ExperimentVersion V2
```

No automatic Batch is created for V2.

## 3. Accepted invariants

- Follow-up AgentRun pins the exact BatchSummaryRevision; admission accepts only latest-at-create under the BatchRow lock.
- A run already created against S(N) remains pinned when S(N+1) is later persisted.
- ResultContext loads the exact pinned summary and exposes selected valid evidence, exclusions, and Trial conditions.
- Raw file contents do not enter ResultContext; later raw MetricObservation mutation does not override the pinned summary.
- Follow-up Proposal stays in the same Experiment; R2 and V2 numbers increment.
- Proposal crash/retry returns the same R2 and does not create R3.
- Agent cannot auto-verify, auto-approve, or auto-create/start a Batch.
- The initial Agent path remains compatible and remote provider local-only egress remains blocked.

## 4. Test and CI evidence

- Task 7 deterministic acceptance: PASS.
- Task 8 focused regression: PASS.
- Task 8 concurrency barrier repeated 5x: PASS.
- Task 8 GitHub CI #312: SUCCESS.
- `./scripts/check_ci.sh`: repository-equivalent full gate PASS.
- Ruff: PASS; Mypy: PASS; unit/architecture: PASS; integration: PASS; dependency shell guard: PASS.

## 5. Not established

This closeout does NOT establish:

- autonomous research Agent;
- automatic approval or next-Batch execution;
- controlled/autonomous code modification or editable Agent Worktree;
- Multi-Agent or cross-Experiment historical search;
- global exactly-once execution;
- Slice 2 production acceptance or production-scale scheduling/readiness;
- measured researcher productivity improvement.

The product claim remains: bounded Agent-assisted Experiment Runtime / Harness.
45 changes: 45 additions & 0 deletions migrations/versions/0013_result_aware_agent_runs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
"""add result-aware AgentRun provenance

Revision ID: 0013_result_aware_agent_runs
Revises: 0012_version_identity_nonunique
Create Date: 2026-09-20
"""

from collections.abc import Sequence

import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql

revision: str = "0013_result_aware_agent_runs"
down_revision: str | None = "0012_version_identity_nonunique"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None


def upgrade() -> None:
op.add_column(
"agent_runs",
sa.Column(
"source_batch_summary_revision_id",
postgresql.UUID(as_uuid=True),
nullable=True,
),
)
op.create_foreign_key(
"fk_agent_runs_source_summary_revision",
"agent_runs",
"batch_summary_revisions",
["source_batch_summary_revision_id"],
["batch_summary_revision_id"],
ondelete="RESTRICT",
)


def downgrade() -> None:
op.drop_constraint(
"fk_agent_runs_source_summary_revision",
"agent_runs",
type_="foreignkey",
)
op.drop_column("agent_runs", "source_batch_summary_revision_id")
41 changes: 39 additions & 2 deletions src/labpilot/agent_runtime/context_compiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from pydantic import TypeAdapter

from labpilot.agent_runtime.provider import ModelRequest
from labpilot.agent_runtime.result_context import ResultContext
from labpilot.agent_runtime.types import AgentDecision
from labpilot.common.canonical_json import CanonicalValue, canonical_json_bytes
from labpilot.common.hashing import namespaced_sha256
Expand All @@ -23,6 +24,7 @@
_MAX_PRIOR_RESULTS = 20
_MAX_TOOL_RESULT_BYTES = 64 * 1024
_MAX_HUMAN_INPUT_BYTES = 4000
_MAX_RESULT_CONTEXT_BYTES = 128 * 1024
_MAX_MESSAGE_BYTES = 256 * 1024


Expand Down Expand Up @@ -54,6 +56,7 @@ def compile(
definitions: tuple[ToolDefinition, ...],
prior_tool_results: tuple[PriorToolResult, ...],
prior_human_inputs: tuple[str, ...],
result_context: ResultContext | None = None,
) -> CompiledContext:
_require_utf8(goal, field="goal", max_bytes=_MAX_GOAL_BYTES, non_blank=True)
if len(definitions) > _MAX_TOOL_COUNT:
Expand All @@ -65,7 +68,7 @@ def compile(
output_schema_value = _canonical_value(output_schema)
output_schema_hash = namespaced_sha256(_OUTPUT_SCHEMA_NAMESPACE, output_schema_value)
tools = [_tool_contract(definition) for definition in definitions]
system_value: CanonicalValue = {
system_value: dict[str, CanonicalValue] = {
"instruction": (
"Return exactly one JSON object matching agent_decision_schema. "
"Do not include rationale, reasoning, or chain_of_thought."
Expand Down Expand Up @@ -120,6 +123,23 @@ def compile(
"agent_decision_schema": output_schema_value,
"tools": tools,
}
if result_context is not None:
system_value["result_context_policy"] = [
(
"Treat selected and eligible evidence in result_context as authoritative "
"for the deterministic facts it contains."
),
(
"Use exclusion reason codes to diagnose rejected Trials, but do not "
"promote invalid raw results into accepted scientific evidence."
),
"If the declared research goal is already satisfied, stop is a valid decision.",
(
"If another experiment is warranted, propose must include the full "
"ProposalPayload required by the existing proposal contract."
),
"Never start or approve the next experiment from the Agent decision.",
]
prior_results: list[CanonicalValue] = []
for item in prior_tool_results:
value: CanonicalValue = {
Expand All @@ -138,7 +158,7 @@ def compile(
non_blank=True,
)
human_inputs.append(value)
user_value: CanonicalValue = {
user_value: dict[str, CanonicalValue] = {
"goal": goal,
"snapshot": {
"repository_snapshot_id": str(snapshot.repository_snapshot_id),
Expand All @@ -153,6 +173,17 @@ def compile(
"prior_tool_results": prior_results,
"prior_human_inputs": human_inputs,
}
if result_context is not None:
result_value = _canonical_value(
result_context.model_dump(
mode="json",
warnings="none",
fallback=_serialization_fallback,
)
)
if len(canonical_json_bytes(result_value)) > _MAX_RESULT_CONTEXT_BYTES:
raise ContextCompilationError("result_context_too_large")
user_value["result_context"] = result_value
messages = (
{"role": "system", "content": _render(system_value)},
{"role": "user", "content": _render(user_value)},
Expand Down Expand Up @@ -214,6 +245,12 @@ def _render(value: CanonicalValue) -> str:
return canonical_json_bytes(value).decode("utf-8")


def _serialization_fallback(value: object) -> object:
if isinstance(value, Mapping):
return dict(value)
raise TypeError(f"unsupported result context value: {type(value).__name__}")


def _require_utf8(value: str, *, field: str, max_bytes: int, non_blank: bool) -> None:
if non_blank and not value.strip():
raise ContextCompilationError(f"{field}_blank")
Expand Down
9 changes: 9 additions & 0 deletions src/labpilot/agent_runtime/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ class AgentRunRow(Base):
name="fk_agent_runs_produced_proposal_revision_id_proposal_revisions",
ondelete="RESTRICT",
),
ForeignKeyConstraint(
["source_batch_summary_revision_id"],
["batch_summary_revisions.batch_summary_revision_id"],
name="fk_agent_runs_source_summary_revision",
ondelete="RESTRICT",
),
PrimaryKeyConstraint("agent_run_id", name="pk_agent_runs"),
CheckConstraint(
"status IN ('ready','running','awaiting_human','completed',"
Expand Down Expand Up @@ -73,6 +79,9 @@ class AgentRunRow(Base):
produced_proposal_revision_id: Mapped[UUID | None] = mapped_column(
PGUUID(as_uuid=True), nullable=True
)
source_batch_summary_revision_id: Mapped[UUID | None] = mapped_column(
PGUUID(as_uuid=True), nullable=True
)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
Expand Down
12 changes: 9 additions & 3 deletions src/labpilot/agent_runtime/repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,15 @@ def __init__(self, session_context_factory: SessionContextFactory) -> None:
self._session_context_factory = session_context_factory

async def add_run(self, run: AgentRunDetails) -> AgentRunDetails:
row = _run_row(run)
async with UnitOfWork(self._session_context_factory) as uow:
uow.session.add(row)
await uow.session.flush()
details = await self.add_run_in_uow(uow, run)
await uow.commit()
return details

async def add_run_in_uow(self, uow: UnitOfWork, run: AgentRunDetails) -> AgentRunDetails:
row = _run_row(run)
uow.session.add(row)
await uow.session.flush()
return _run_details(row)

async def get_run(self, agent_run_id: UUID) -> AgentRunDetails | None:
Expand Down Expand Up @@ -285,6 +289,7 @@ def _run_row(run: AgentRunDetails) -> AgentRunRow:
total_tokens=run.total_tokens,
next_step_ordinal=run.next_step_ordinal,
produced_proposal_revision_id=run.produced_proposal_revision_id,
source_batch_summary_revision_id=run.source_batch_summary_revision_id,
created_at=run.created_at,
started_at=run.started_at,
completed_at=run.completed_at,
Expand Down Expand Up @@ -337,6 +342,7 @@ def _run_details(row: AgentRunRow) -> AgentRunDetails:
next_step_ordinal=row.next_step_ordinal,
produced_proposal_revision_id=row.produced_proposal_revision_id,
created_at=row.created_at,
source_batch_summary_revision_id=row.source_batch_summary_revision_id,
started_at=row.started_at,
completed_at=row.completed_at,
row_version=row.row_version,
Expand Down
45 changes: 45 additions & 0 deletions src/labpilot/agent_runtime/result_context.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
from __future__ import annotations

from typing import Literal, Protocol
from uuid import UUID

from pydantic import BaseModel, ConfigDict, Field, JsonValue

from labpilot.agent_runtime.types import AgentRunDetails
from labpilot.experiments.proposal_types import ProposalPayload


class ResultTrialCondition(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)

trial_id: UUID
trial_ordinal: int = Field(gt=0)
resolved_parameters: dict[str, JsonValue]
seed: int


class ResultContext(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)

experiment_id: UUID
source_experiment_version_id: UUID
source_version_number: int = Field(gt=0)
source_proposal_revision_id: UUID
source_batch_id: UUID
source_batch_summary_revision_id: UUID
source_summary_revision_number: int = Field(gt=0)
presentation_status: Literal["complete", "complete_with_invalid_results"]
source_proposal: ProposalPayload
trials: tuple[ResultTrialCondition, ...]
included_results: tuple[dict[str, JsonValue], ...]
excluded_results: tuple[dict[str, JsonValue], ...]
warnings: tuple[dict[str, JsonValue], ...]
aggregates: dict[str, JsonValue]


class FollowUpRunStore(Protocol):
async def create_bound_run(self, run: AgentRunDetails) -> AgentRunDetails: ...


class ResultContextSource(Protocol):
async def load(self, source_batch_summary_revision_id: UUID) -> ResultContext: ...
Loading
Loading