Skip to content

fix(harness): close the turn's event source when delivery stops early - #541

Open
michaelxu2288 wants to merge 2 commits into
scaleapi:mainfrom
michaelxu2288:fix/harness-close-source-on-cancel
Open

michaelxu2288 wants to merge 2 commits into
scaleapi:mainfrom
michaelxu2288:fix/harness-close-source-on-cancel

Conversation

@michaelxu2288

@michaelxu2288 michaelxu2288 commented Oct 1, 2026 •

Copy link
Copy Markdown

Problem

The harness delivery adapters never close the event source they iterate:

  • auto_send, the async push path behind UnifiedEmitter.auto_send_turn;
  • yield_events and UnifiedEmitter.yield_turn, the sync HTTP yield path.

Their finally blocks close streaming contexts and flush span derivation, but leave turn.events alone. Every CLI tap depends on being closed. The Claude Code and Codex taps close their raw line iterator in a finally, and that close is what terminates the CLI subprocess in the scaffolds.

It goes wrong in two cases:

  • a turn is cancelled (an interrupt, or an activity cancel) while delivery is awaiting the backend (ctx.stream_update or a context close) rather than the source;
  • a sync client disconnects mid-stream.

In both, the source stays suspended at a yield. The turn object keeps a reference to its event generator, and agents hold the turn to read turn.session_id. So garbage collection never finalizes it, and the CLI subprocess and its stdout pipe leak.

Fix

  • auto_send and yield_events close events in their finally, after the existing cleanup. The close is guarded with contextlib.suppress(Exception), so a failing aclose cannot mask the original exception.
  • UnifiedEmitter.yield_turn closes its delivery generator explicitly instead of leaving it to async-generator finalization.
  • Plain async iterators without aclose, and generators that are already exhausted, behave exactly as before.

Verification

  • New tests:
    • test_auto_send.py: cancel auto_send while stream_update blocks, with the source held by a local and no gc.collect();
    • test_yield_delivery.py: close yield_events early;
    • test_emitter.py: close yield_turn early, with a turn that pins its generator the way the CLI taps do.
  • All three fail on main with assert [] == [True] (the source's finally never ran) and pass here.
  • An extra test checks that a source without aclose still delivers normally.
  • uv run pytest -n 0 tests/lib/core/harness tests/lib/adk: 571 passed, 1 skipped.
  • ruff check and pyright (1.1.399) are clean.

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no actionable issue remains established.

Summary

Harness delivery now closes event sources when delivery stops early, so CLI tap cleanup runs on cancellation and client disconnects. The ACP response stream also closes its handler generator, carrying disconnect cleanup upstream.

  • Async push delivery closes its source after its existing cleanup.
  • Sync harness delivery passes early closure through to the event source.
  • ACP response streaming closes the handler generator when the stream ends.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[ACP response body closes] --> B[Handler generator closes]
  B --> C[yield_turn closes delivery]
  C --> D[yield_events closes turn events]
Loading

Reviews (2) · Last reviewed commit: "fix(acp): close the handler generator wh..."

The delivery adapters (auto_send for the async push path, yield_events
and UnifiedEmitter.yield_turn for the sync yield path) closed their
streaming contexts and flushed span derivation in `finally`, but never
closed the event source they were iterating. Every tap relies on that
close: the Claude Code and Codex taps close their raw line iterator in a
`finally`, which is what terminates the CLI subprocess in the scaffolds.

When a turn is cancelled while delivery is awaiting the backend
(stream_update or a context close) rather than the source, or a sync
client disconnects mid-stream, the source stays suspended at a yield.
The turn object keeps a reference to its event generator, so garbage
collection does not finalize it while the caller still holds the turn,
and the CLI subprocess and its stdout handle leak.

Both adapters now close the source in `finally`, after the existing
cleanup and guarded so a failing aclose cannot mask the original
exception. yield_turn closes its delivery generator explicitly instead
of leaving it to async-generator finalization. Plain async iterators
without aclose and already-exhausted generators are unaffected.

Three new tests (cancel during a blocked stream_update; early close of
yield_events; early close of yield_turn with a turn that pins its
generator) fail on main with `assert [] == [True]` and pass here.
tests/lib/core/harness and tests/lib/adk: 571 passed, 1 skipped.
Comment thread src/agentex/lib/core/harness/emitter.py
_handle_streaming_response iterated the handler's async generator inside
generate_json_rpc_stream but never closed it. When a client disconnects,
Starlette closes the body iterator, and the handler (with the harness
turn and CLI subprocess under it) stayed suspended until garbage
collection. Close the handler generator in a finally, guarded so a
failing aclose cannot mask the original error.

New test: take one chunk from the streaming response, close the body
iterator, and assert the handler generator's finally ran. It fails on
main (assert [] == [True]) and passes here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant