Skip to content

fix: replay no longer dispatches a second EXECUTION_END hook - #8422

Open
Wilson0511 wants to merge 3 commits into
crewAIInc:mainfrom
Wilson0511:fix/replay-execution-end-pairing
Open

Wilson0511 wants to merge 3 commits into
crewAIInc:mainfrom
Wilson0511:fix/replay-execution-end-pairing

Conversation

@Wilson0511

@Wilson0511 Wilson0511 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #8421

Problem

Crew.replay() skips prepare_kickoff, so no EXECUTION_START dispatches for the replayed run — but its execution tail (_create_crew_output) dispatched InterceptionPoint.EXECUTION_END unconditionally. After kickoff() + replay(), every @on(InterceptionPoint.EXECUTION_END) hook had fired twice for a single EXECUTION_START (observed dispatch sequence: start, end, end).

This violates the pairing contract documented in prepare_kickoff:

Pairing flags: EXECUTION_END fires (once) only for executions whose EXECUTION_START actually dispatched, including on the failure path.

which _dispatch_execution_end_failure already enforces on the failure path.

Change

  • src/crewai/crew.py (_create_crew_output): dispatch EXECUTION_END only when _execution_start_dispatched and not _execution_end_dispatched — the same guard the failure path uses. The flag is still set before dispatch so an END hook that raises HookAborted cannot trigger a second (failure) dispatch upstream.

Verification

  • New regression test tests/test_crew.py::test_replay_does_not_redispatch_execution_end_hook: fails on main (dispatches == ['start', 'end', 'end']), passes with the fix.
  • tests/test_crew.py -k replay: 15 passed.
  • tests/hooks/: 200 passed, 1 pre-existing failure (test_a_structured_litellm_call_is_seen_once, missing optional litellm dep — identical on clean main).
  • Full tests/test_crew.py: 137 passed, 1 pre-existing litellm/vcr env failure identical on clean main.
  • mypy src/crewai/crew.py: clean; ruff error count unchanged vs main.

Risk

Low: kickoff() always sets _execution_start_dispatched (via prepare_kickoff) before reaching _execute_tasks, so the guard is a no-op on the normal path; only replay's tail changes behavior.

🤖 Generated with ZCode (AI agent) on behalf of @Wilson0511


Note

Low Risk
Narrows when EXECUTION_END runs; standard kickoff still pairs start/end, and only replay’s extra end dispatch is skipped.

Overview
Fixes duplicate EXECUTION_END interception hooks after kickoff() + replay(). replay() does not run prepare_kickoff, so it never dispatches EXECUTION_START, but _create_crew_output still fired EXECUTION_END on every successful run—listeners saw start, end, end for one start.

In _create_crew_output, EXECUTION_END is dispatched only when _execution_start_dispatched is true and _execution_end_dispatched is false, matching the pairing logic already used in _dispatch_execution_end_failure. Normal kickoff() is unchanged because it always sets the start flag via prepare_kickoff.

Adds regression test test_replay_does_not_redispatch_execution_end_hook asserting exactly one start and one end after kickoff and replay.

Reviewed by Cursor Bugbot for commit e9bc5fb. Bugbot is set up for automated code reviews on this repo. Configure here.

Crew.replay() skips prepare_kickoff, so no EXECUTION_START dispatches
for the replayed run, but its execution tail (_create_crew_output)
dispatched InterceptionPoint.EXECUTION_END unconditionally. After
kickoff + replay an EXECUTION_END hook fired twice for a single
EXECUTION_START (observed dispatch sequence: start, end, end).

Change:
- src/crewai/crew.py (_create_crew_output): dispatch EXECUTION_END only
  when _execution_start_dispatched and not _execution_end_dispatched.
  The flag is still set before dispatch so an END hook that raises
  HookAborted cannot trigger a second (failure) dispatch upstream.

Why:
- prepare_kickoff documents the pairing contract: EXECUTION_END fires
  (once) only for executions whose EXECUTION_START actually dispatched,
  and _dispatch_execution_end_failure already enforces it on the failure
  path. replay() broke it: END hooks (finalizers, per-run aggregation)
  ran a second time with no START to pair against.

Proof:
- tests/test_crew.py::test_replay_does_not_redispatch_execution_end_hook
  fails on main (dispatches == [start, end, end]) and passes with the
  fix. Full tests/test_crew.py: 137 passed, 1 pre-existing litellm/vcr
  env failure (test_crew_kickoff_streaming_usage_metrics) identical on
  clean main.

Risk:
- Low. kickoff/prepare_kickoff always sets _execution_start_dispatched
  before _execute_tasks, so the guard is a no-op on the normal path;
  only replay's tail changes behavior.

🤖 Generated with ZCode (AI agent) on behalf of Wilson0511

Co-Authored-By: ZCode <[email protected]>
@strix-security

strix-security Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 31 commits after the last Strix review (42b1b54). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: This PR makes `Crew.replay()` stop dispatching a second `EXECUTION_END` interception hook: the success-path tail now dispatches that hook only when a matching `EXECUTION_START` fired and no end hook has fired yet, mirroring the guard already used on the failure path. It adds a regression test asserting `kickoff()` followed by `replay()` fires each boundary hook exactly once.

Review summary

Reviewed the full diff: a single 5-line guard in Crew._create_crew_output that dispatches EXECUTION_END only when _execution_start_dispatched is set and _execution_end_dispatched is not, plus a regression test. The change is a hook-pairing idempotency fix and introduces no security weakness. The guard is a no-op on the normal kickoff path because prepare_kickoff resets both flags immediately before dispatching EXECUTION_START; on the replay tail it skips only the observational, fail-open EXECUTION_END dispatch, while the OUTPUT hook that the documentation designates for output sanitization still fires and its payload replacement still reaches the returned CrewOutput. crew_output is bound before the guard, so the new conditional cannot raise an unbound-variable error on any path. Static analysis of the changed file produced no diff-attributable findings.

Ruled out (2)
  • Does skipping EXECUTION_END for replayed executions let an attacker bypass an audit or output-sanitization control?
    EXECUTION_END is a documented fail-open observational hook (non-HookAborted errors are swallowed), the OUTPUT point the docs designate for redaction is still dispatched unconditionally on the replay path and its payload replacement reaches the returned CrewOutput, and replay() is an in-process SDK/…
  • Can stale per-instance pairing flags suppress an EXECUTION_END hook during normal, fan-out, or concurrent executions?
    prepare_kickoff resets both flags immediately before dispatching EXECUTION_START on every kickoff, kickoff_for_each and checkpoint restore operate on copies whose flags default to false, and overlapping kickoffs on one instance are already documented as unsupported and pre-broken for other instance…

Review 1 on this PR · 42b1b54 · requested Oct 22 16:16 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@Wilson0511 Wilson0511 closed this Oct 23, 2026
@Wilson0511 Wilson0511 reopened this Oct 23, 2026
@Vidit-Ostwal

Copy link
Copy Markdown
Member

Can you resolve the conflict once?

Resolve test_crew.py conflict by keeping both sides: this branch's
test_replay_does_not_redispatch_execution_end_hook and main's two
replay-failure tests from crewAIInc#8418 (test_replay_failure_emits_kickoff_failed_event,
test_replay_failure_does_not_leak_kickoff_scope_into_next_kickoff),
plus the CrewKickoffFailedEvent import the latter needs.

Verified: tests/test_crew.py -k replay -> 17 passed on the merged tree;
full file 138 passed with the pre-existing local-env vcr failure in
test_crew_kickoff_streaming_usage_metrics (fails identically on clean
origin/main).

🤖 Generated with ZCode (AI agent) on behalf of Wilson0511

Co-Authored-By: ZCode <[email protected]>
@Wilson0511

Copy link
Copy Markdown
Contributor Author

Conflict resolved — merged latest main and kept both sides' replay tests (the two replay-failure tests from #8418 plus this PR's test_replay_does_not_redispatch_execution_end_hook). -k replay is 17 passed on the merged tree.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b1e4494. Configure here.

Comment thread lib/crewai/tests/test_crew.py Resolved
The merge of origin/main into fix/replay-execution-end-pairing kept this
PR's replay tests but dropped two unrelated regression tests plus their
ScriptedReActLLM helper from the main side:

- test_hierarchical_crew_delegates_to_workers_on_every_kickoff
- test_crew_test_setup_failure_pairs_events
- _ScriptedReActLLM and the asyncio / BaseLLM / CrewTestFailedEvent imports

Restore them verbatim from main. Flagged by Cursor Bugbot on the PR.

Change: re-add the three blocks and imports, no source changes.
Why: conflict resolution must not silently drop main's regression coverage.
Proof: the five restored tests pass; -k 'replay or <restored>' is 19 passed
with the 3 known OPENAI_API_KEY-env failures unchanged.
Risk: none — additive test-file changes only.

🤖 Generated with ZCode (AI agent) on behalf of Wilson0511

Co-Authored-By: ZCode <[email protected]>
@Wilson0511

Copy link
Copy Markdown
Contributor Author

Good catch by Bugbot — the conflict resolution had indeed dropped those two regression tests (plus the _ScriptedReActLLM helper they use) from the main side. Restored verbatim from main in e9bc5fb; the five restored tests pass locally (19 passed with -k 'replay or hierarchical_crew_delegates or crew_test_setup', the 3 remaining failures are the known OPENAI_API_KEY-env ones, unchanged).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Crew.replay() dispatches a second EXECUTION_END hook without a matching EXECUTION_START

2 participants