Skip to content

fix: pass grader_payload's lang, hide_output, and skip_grader through - #327

Draft
ormsbee wants to merge 1 commit into
mainfrom
ormsbee/preserve-xqueue-body
Draft

ormsbee wants to merge 1 commit into
mainfrom
ormsbee/preserve-xqueue-body

Conversation

@ormsbee

@ormsbee ormsbee commented Nov 12, 2026 •

Copy link
Copy Markdown

Description

Part of openedx/edx-submissions#390. Companion to openedx/edx-submissions#391.

When a course sends submissions to edx-submissions instead of XQueue (the send_to_submission_course.enable waffle flag), XQueueInterfaceSubmission parses the problem's <grader_payload>, keeps only the grader key, and drops the rest. xqueue-watcher's built-in graders read three other keys: skip_grader, hide_output, and lang. This PR:

  1. Passes those three options through. We read them from <grader_payload> and pass them to create_external_grader_detail. edx-submissions stores them and sends them back to the grader in grader_payload.
    • hide_output and skip_grader are read by truthiness, the way xqueue-watcher reads them, so the string "false" still counts as true.
    • A lang that isn't a string of at most 32 characters is dropped with a warning, and the grader uses its default language. A non-string lang can break the grader, and edx-submissions stores lang in a 32-character field.
  2. Stops rejecting payloads that aren't JSON objects. Before, a <grader_payload> that wasn't a JSON object raised an error. That includes a problem with no <grader_payload> at all. Now we read grader and the options when we can, use the defaults when we can't, and queue the submission either way, as legacy XQueue did.
  3. Surfaces failures to the learner. _send_to_queue threw away the error that send_to_submission returned and always reported success. So when a submission couldn't be created, no row was written and the learner sat on "queued" forever. Now it returns the error, and the learner sees the same "Unable to deliver your submission to grader" message as an XQueue delivery failure.

get_submission_params now returns the options as a sixth value. Nothing outside this module calls it.

MissingKeyError isn't raised anywhere now, but I left the class in errors.py in case anything else imports it.

Release order

This is safe to merge and release before the edx-submissions PR. edx-submissions releases without these arguments ignore them; I checked this end to end against 4.1.0. The fix takes effect once both are released and pinned in openedx-platform.

Testing

  • Unit tests for:

    • reading the options, including truthiness and invalid lang values
    • passing them to edx-submissions
    • the different grader_payload shapes
    • the error now reaching send_to_queue's caller

    The src/xblock_problem suite passes, and ruff and pylint are clean.

  • End-to-end check with no mocks: send_to_submission → real create_external_grader_detail → the grader_payload edx-submissions' get_submission sends.

    • Against the companion branch, the grader got {"grader": "output/grader.py", "lang": "es", "hide_output": true, "skip_grader": true}.
    • Against released edx-submissions 4.1.0, the submission is still created, and the options are ignored.
    • A body with "grader_payload": "" is queued in both.

Manual testing (not yet done)

  1. Run an LMS with this branch and the edx-submissions branch, and turn on send_to_submission_course.enable for a course.
  2. Point xqueue-watcher at the LMS's xqueue endpoints.
  3. Add a <coderesponse> problem with <grader_payload>{"grader": "grader.py", "hide_output": true}</grader_payload> and submit a wrong answer. The feedback should show the overall result without per-test output.
  4. Change it to "skip_grader": true. Submitting should give full credit without running the code.
  5. Submit to a problem with no <grader_payload>. It should show up in the ExternalGraderDetail admin instead of hanging on "queued".

Merge checklist:
Check off if complete or not applicable:

  • Version bumped (N/A: the version comes from the release tag)
  • Changelog record added (N/A: this repo has no changelog)
  • Documentation updated (not only docstrings) (N/A)
  • Fixup commits are squashed away
  • Unit tests added/updated
  • Manual testing instructions provided
  • Noted any: Concerns, dependencies, migration issues, deadlines, tickets

🤖 Generated with Claude Code

When a course uses the edx-submissions queue instead of XQueue, we parsed
the problem's <grader_payload>, kept only its "grader" key, and dropped
the rest. xqueue-watcher's built-in graders read three other keys:

* skip_grader: give full credit without running the learner's code
* hide_output: don't show per-test results, including the expected
  output from the staff answer
* lang: which translation catalog to use for feedback

This passes those to create_external_grader_detail, which stores them
and sends them back to the grader in grader_payload. We read hide_output
and skip_grader by truthiness, the way xqueue-watcher does, and drop a
lang that isn't a string of at most 32 characters. Releases of
edx-submissions that don't know about these arguments ignore them, so
this is safe to release first.

It also fixes two ways a submission could be lost without the learner
knowing:

* A <grader_payload> that wasn't a JSON object (including a problem with
  no <grader_payload> at all) raised an error, so no submission was
  created. We now read the options only when we can, and queue the
  submission either way, as legacy XQueue did.
* _send_to_queue ignored the error that send_to_submission returned and
  always reported success, so the learner sat on "queued" forever. It
  now returns the error, and the problem shows it to the learner the
  same way it shows XQueue delivery errors.

Part of: openedx/edx-submissions#390

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@ormsbee ormsbee changed the title fix: pass the full xqueue body through to edx-submissions fix: pass grader_payload's lang, hide_output, and skip_grader through Nov 13, 2026
@ormsbee
ormsbee force-pushed the ormsbee/preserve-xqueue-body branch from ce21a38 to d9189f0 Compare November 13, 2026 19:50
@ormsbee

ormsbee commented Nov 13, 2026

Copy link
Copy Markdown
Author

I'm going to wait on the result of the conversation in openedx/edx-submissions#390 (comment) before fixing this PR up and marking it ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant