Repository navigation
fix: emit LLMGuardrailCompletedEvent for a malformed guardrail result - #8648
Conversation
process_guardrail emits LLMGuardrailStartedEvent, then converts the guardrail's return value with GuardrailResult.from_tuple. That conversion ran outside the try block crewAIInc#8403 added, so a guardrail returning the wrong shape, such as (False, 42) or a three-item tuple, raised without a LLMGuardrailCompletedEvent. The started scope stayed on the event stack: Agent.kickoff and Crew.kickoff then logged event pairing mismatches and later events in the same context got the wrong parent. Convert the result inside the try block so these errors take the same terminal path as a guardrail that raises. The caller still gets the same exception.
Strix Security Review✅ No open security findings. PR summary: Moves the conversion of a guardrail's return value into a `GuardrailResult` inside the existing error-handling `try` block of `process_guardrail`, so a guardrail that returns the wrong shape now emits a matching `LLMGuardrailCompletedEvent` instead of leaving the started event unpaired. Adds tests for malformed returns and a well-formed control case. Review summaryThe only functional change moves Review 1 on this PR · Reviewed by Strix |
Vidit-Ostwal
left a comment
There was a problem hiding this comment.
Reviewed the malformed guardrail-result path and its regression coverage. The conversion now correctly remains inside the exception boundary, ensuring the started guardrail event receives a failed completion event while preserving the original exception.
Related issue
Fixes #8647
Summary
Follow-up to #8403.
process_guardrailconverts the guardrail's return value withGuardrailResult.from_tupleafter the
tryblock that #8403 added, so a guardrail returning the wrong shape (for example(False, 42)or athree-item tuple) raised without a
LLMGuardrailCompletedEvent. The started scope stayed on the event stack:Agent.kickoffandCrew.kickofflogged event pairing mismatches, and later events in the same context got thewrong parent.
The conversion now runs inside the
try, so these errors take the same terminal path as a guardrail that raises.The caller still gets the same exception, and a well-formed result is handled exactly as before. All four callers
(
Agent,LiteAgent, and the sync and asyncTaskpaths) go throughprocess_guardrail, so this one changecovers them.
Verification
Added to
test_guardrail_events_on_exception.py, next to the #8403 tests:test_a_malformed_guardrail_result_still_reports_the_validation_it_started(non-string-error,three-tuple):both fail on
mainwithassert 0 == 1(no completed event) and pass with the fix.test_a_valid_guardrail_result_reports_one_completed_event: control, a well-formed(False, "too short")stillemits exactly one completed event. Passes on
mainand with the fix.The issue repro through
Agent.kickoff()andCrew.kickoff()now emits both events, leaves the scope stack empty,and logs no pairing mismatch.
lib/crewai/tests/: 6349 passed, 44 skipped (Python 3.13)ruff check,ruff format,mypycleanAdditional context
None
Note
Low Risk
Small control-flow change in event emission only; behavior for valid guardrails and raised errors stays the same.
Overview
Moves
GuardrailResult.from_tupleinside thetryinprocess_guardrailso malformed guardrail return values (wrong tuple length or non-string error payload) no longer skip the terminalLLMGuardrailCompletedEventthat #8403 added for raised exceptions.Those failures now follow the same path as guardrail raises: one completed event with
success=Falseand an error string, then the original exception is re-raised. Valid(bool, …)results are unchanged.Tests cover malformed returns (
(False, 42), three-tuples) and confirm well-formed failures still emit a single completed event.Reviewed by Cursor Bugbot for commit 59ef43c. Bugbot is set up for automated code reviews on this repo. Configure here.