You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BUG: Crescendo passes the sent prompt to the refusal scorer as the attack objective #3561
CrescendoAttack._perform_backtrack_if_refused_async passes prompt_sent, the full adversarial prompt that was just sent to the target, as the objective argument of _check_refusal_async. That parameter is documented as the attack objective and is rendered into the refusal judge's prompt as the conversation objective (conversation_objective: {objective} in SelfAskRefusalScorer).
With the default SelfAskRefusalScorer in an OBJECTIVE mode, the judge therefore evaluates deflection and redirection against the adversarial prompt itself instead of against the attack objective. The refusal score's persisted expectation also records the prompt as the objective, so expectation fingerprints and replay identity carry the wrong reference text.
Expected Behavior
The refusal check should receive context.objective, matching the parameter's contract and the other scoring calls in the same class, which all pass context.objective.
Reproduction Steps
Run a Crescendo attack with a SelfAskRefusalScorer in OBJECTIVE_STRICT mode.
Inspect the refusal judge's rendered request or the persisted scored_expectation.
The conversation objective shown to the judge is the prompt that was sent, not the attack objective.
Minimal Reproduction
pyrit/executor/attack/multi_turn/crescendo.py line 817: self._check_refusal_async(context, prompt_sent) where the method signature names the second parameter objective.
I'd like to work on this. The Crescendo orchestrator is incorrectly passing the adversarial prompt as the objective to the refusal scorer. I'll investigate the context propagation and submit a PR
Hi 🐦[T3T]Berserker 🐦 🐍 (@TrinityBerserker), just a heads-up
this issue has already been assigned to me by a maintainer, and I've implemented the fix in PR #3562, including regression tests.
I'd recommend checking existing assignments and linked PRs before picking up issues, as working on already-assigned and actively addressed issues can lead to unnecessary duplication and wasted effort.
I'd appreciate it if you could respect the existing assignment and focus on issues that aren't already being worked on. Thanks for understanding!
Activity
TrinityBerserker commentedon Oct 27, 2026
I'd like to work on this. The Crescendo orchestrator is incorrectly passing the adversarial prompt as the objective to the refusal scorer. I'll investigate the context propagation and submit a PR
ManoharPaturi commentedon Oct 27, 2026
Hi 🐦[T3T]Berserker 🐦 🐍 (@TrinityBerserker), just a heads-up
this issue has already been assigned to me by a maintainer, and I've implemented the fix in PR #3562, including regression tests.
I'd recommend checking existing assignments and linked PRs before picking up issues, as working on already-assigned and actively addressed issues can lead to unnecessary duplication and wasted effort.
I'd appreciate it if you could respect the existing assignment and focus on issues that aren't already being worked on. Thanks for understanding!
TrinityBerserker commentedon Oct 27, 2026
No problem at all, I didn't see PR #3562 linked to this issue. Closing this PR now. Thanks!