Repository navigation
Conversation
When a course uses the edx-submissions queue instead of XQueue, we parsed the problem's <grader_payload>, kept only its "grader" key, and dropped the rest. xqueue-watcher's built-in graders read three other keys: * skip_grader: give full credit without running the learner's code * hide_output: don't show per-test results, including the expected output from the staff answer * lang: which translation catalog to use for feedback This passes those to create_external_grader_detail, which stores them and sends them back to the grader in grader_payload. We read hide_output and skip_grader by truthiness, the way xqueue-watcher does, and drop a lang that isn't a string of at most 32 characters. Releases of edx-submissions that don't know about these arguments ignore them, so this is safe to release first. It also fixes two ways a submission could be lost without the learner knowing: * A <grader_payload> that wasn't a JSON object (including a problem with no <grader_payload> at all) raised an error, so no submission was created. We now read the options only when we can, and queue the submission either way, as legacy XQueue did. * _send_to_queue ignored the error that send_to_submission returned and always reported success, so the learner sat on "queued" forever. It now returns the error, and the problem shows it to the learner the same way it shows XQueue delivery errors. Part of: openedx/edx-submissions#390 Co-Authored-By: Claude Opus 5.5 <[email protected]>
ormsbee
force-pushed
the
ormsbee/preserve-xqueue-body
branch
from
November 13, 2026 19:50
ce21a38 to
d9189f0
Compare
Author
|
I'm going to wait on the result of the conversation in openedx/edx-submissions#390 (comment) before fixing this PR up and marking it ready for review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Part of openedx/edx-submissions#390. Companion to openedx/edx-submissions#391.
When a course sends submissions to edx-submissions instead of XQueue (the
send_to_submission_course.enablewaffle flag),XQueueInterfaceSubmissionparses the problem's<grader_payload>, keeps only thegraderkey, and drops the rest. xqueue-watcher's built-in graders read three other keys:skip_grader,hide_output, andlang. This PR:<grader_payload>and pass them tocreate_external_grader_detail. edx-submissions stores them and sends them back to the grader ingrader_payload.hide_outputandskip_graderare read by truthiness, the way xqueue-watcher reads them, so the string"false"still counts as true.langthat isn't a string of at most 32 characters is dropped with a warning, and the grader uses its default language. A non-stringlangcan break the grader, and edx-submissions storeslangin a 32-character field.<grader_payload>that wasn't a JSON object raised an error. That includes a problem with no<grader_payload>at all. Now we readgraderand the options when we can, use the defaults when we can't, and queue the submission either way, as legacy XQueue did._send_to_queuethrew away the error thatsend_to_submissionreturned and always reported success. So when a submission couldn't be created, no row was written and the learner sat on "queued" forever. Now it returns the error, and the learner sees the same "Unable to deliver your submission to grader" message as an XQueue delivery failure.get_submission_paramsnow returns the options as a sixth value. Nothing outside this module calls it.MissingKeyErrorisn't raised anywhere now, but I left the class inerrors.pyin case anything else imports it.Release order
This is safe to merge and release before the edx-submissions PR. edx-submissions releases without these arguments ignore them; I checked this end to end against 4.1.0. The fix takes effect once both are released and pinned in openedx-platform.
Testing
Unit tests for:
langvaluesgrader_payloadshapessend_to_queue's callerThe
src/xblock_problemsuite passes, and ruff and pylint are clean.End-to-end check with no mocks:
send_to_submission→ realcreate_external_grader_detail→ thegrader_payloadedx-submissions'get_submissionsends.{"grader": "output/grader.py", "lang": "es", "hide_output": true, "skip_grader": true}."grader_payload": ""is queued in both.Manual testing (not yet done)
send_to_submission_course.enablefor a course.<coderesponse>problem with<grader_payload>{"grader": "grader.py", "hide_output": true}</grader_payload>and submit a wrong answer. The feedback should show the overall result without per-test output."skip_grader": true. Submitting should give full credit without running the code.<grader_payload>. It should show up in theExternalGraderDetailadmin instead of hanging on "queued".Merge checklist:
Check off if complete or not applicable:
🤖 Generated with Claude Code