Skip to content

BUG: Crescendo passes the sent prompt to the refusal scorer as the attack objective #3561

Open

Description

Description

CrescendoAttack._perform_backtrack_if_refused_async passes prompt_sent, the full adversarial prompt that was just sent to the target, as the objective argument of _check_refusal_async. That parameter is documented as the attack objective and is rendered into the refusal judge's prompt as the conversation objective (conversation_objective: {objective} in SelfAskRefusalScorer).

With the default SelfAskRefusalScorer in an OBJECTIVE mode, the judge therefore evaluates deflection and redirection against the adversarial prompt itself instead of against the attack objective. The refusal score's persisted expectation also records the prompt as the objective, so expectation fingerprints and replay identity carry the wrong reference text.

Expected Behavior

The refusal check should receive context.objective, matching the parameter's contract and the other scoring calls in the same class, which all pass context.objective.

Reproduction Steps

  1. Run a Crescendo attack with a SelfAskRefusalScorer in OBJECTIVE_STRICT mode.
  2. Inspect the refusal judge's rendered request or the persisted scored_expectation.
  3. The conversation objective shown to the judge is the prompt that was sent, not the attack objective.

Minimal Reproduction

pyrit/executor/attack/multi_turn/crescendo.py line 817: self._check_refusal_async(context, prompt_sent) where the method signature names the second parameter objective.

Environment

PyRIT main. Python 3.11.

Activity

  1. TrinityBerserker commented on Oct 27, 2026

    @TrinityBerserker

    I'd like to work on this. The Crescendo orchestrator is incorrectly passing the adversarial prompt as the objective to the refusal scorer. I'll investigate the context propagation and submit a PR

  2. ManoharPaturi commented on Oct 27, 2026

    @ManoharPaturi
    ContributorAuthor

    Hi 🐦[T3T]Berserker 🐦 🐍 (@TrinityBerserker), just a heads-up
    this issue has already been assigned to me by a maintainer, and I've implemented the fix in PR #3562, including regression tests.

    I'd recommend checking existing assignments and linked PRs before picking up issues, as working on already-assigned and actively addressed issues can lead to unnecessary duplication and wasted effort.

    I'd appreciate it if you could respect the existing assignment and focus on issues that aren't already being worked on. Thanks for understanding!

  3. TrinityBerserker commented on Oct 27, 2026

    @TrinityBerserker

    No problem at all, I didn't see PR #3562 linked to this issue. Closing this PR now. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions