Skip to content

[DEPR]: XQueue #286

Open
Open

Description

@ormsbee

RFC Start Date

2025-01-29

Target Plan Accepted Date

2025-02-12

Target Transition Unblocked Date

(currently unblocked)

Earliest Breaking Changes Unblocked Date

2026-04-09 -- After Verawood is cut.

Earliest Open edX Named Release Without This Functionality

Willow - 2026-12

Rationale

The XQueue service adds a lot of operational complexity for the capability that it provides. While it did originally do more (e.g. push grading), that functionality is no longer used because it was prone to operational issues. The current form of XQueue mostly serves as a holding area for submissions, waiting for xqueue-watcher to poll for them.

A larger deprecation was proposed in #214, but we concluded that the off-site grading functionality that XQueue represented was still valuable to preserve.

Removal

The XQueue service will be discontinued and the xqueue repo will be archived.

The following settings will be removed from edx-platform:

  • XQUEUE_INTERFACE
  • XQUEUE_WAITTIME_BETWEEN_REQUESTS

We will maintain support for xqueue-watcher and ProblemBlock use of code graders.

When xqueue references are eventually completely removed from edx-platform (post-Teak), we will also execute #192 (MATLAB removal). The two systems are tightly coupled enough where it makes sense to remove them at the same time.

Replacement

Instead of having a separate XQueue service, the edx-submissions repo will be expanded to cover the same use case while running in-process in the LMS. The LMS will also re-implement the REST API that xqueue-watcher interacts with.

The LonCapaSystem service XQueueService will be replaced with a generic XBlock runtime service that is not ProblemBlock-specific.

Deprecation

The LMS will emit warning log messages in Teak whenever the old xqueue-callback URL for ProblemBlocks is hit.

Migration

  • No content changes will be required for authored problems. Existing content is intended to work exactly as it always has.
  • No code changes will be required for xqueue-watcher.
  • Configuration changes will be required for xqueue-watcher, to point it at the LMS instead of at the XQueue service.

In the Teak release, all code grader submissions will be written to the new backend in edx-submissions, and will be available for xqueue-watcher to pull directly from the LMS. If XQUEUE_INTERFACE is also configured for edx-platform, the LMS will additionally post the request to the XQueue service. If the LMS gets a response from the XQueue service, it will delete the queued submission in edx-submissions.

This should allow us a smooth transition path where operators could make their transition by deploying updated configuration for their xqueue-watcher. Operators that are very downtime sensitive could run multiple xqueue-watcher instances concurrently during the transition, falling back to the old behavior if necessary.

Open question:

  1. Should duplicate grading attempts for the same submission be "first one wins" or "last one wins"?
  2. Do we need to preserve an alias for the old Loncapa XQueueService? This would only need to be done if people have extended their instance ProblemBlocks with custom input/response-types that use that service... which seems unlikely?

Support for posting to the XQueue service would be removed after 2025-08-12, and all code and tests in edx-platform related to xqueue will be removed at that point.

Additional Info

We ultimately want to iterate on this and create a much more flexible off-platform grading capability that can:

  1. Make better use of other communications mechanisms, like the message queue backend that openedx-events can utilize.
  2. Allows for multiple interfaces, and so accepts things like LTI's grade passbacks, and possibly bulk grading.
  3. Understands team-related grading, like ORA does.
  4. Remove some of the contortions we make to allow LTI blocks to authenticate non-user requests for grade passback purposes.

But the first step is to preserve the existing functionality while reducing operational complexity of running this system, which this DEPR represents.

Task List

No response

Activity

  1. added
    deprProposal for deprecation & removal per OEP-21
    on Jan 25, 2025
  2. moved this from Proposed to Communicated in DEPR: Deprecation & Removalon Jan 29, 2025
  3. ormsbee commented on Jan 29, 2025

    @ormsbee
    Author
  4. kdmccormick commented on Feb 12, 2025

    @kdmccormick
    Member

    Excellent plan 👍🏻

  5. moved this from Communicated to Accepted in DEPR: Deprecation & Removalon Feb 14, 2025
  6. robrap commented on May 29, 2025

    @robrap

    [inform/request] This ticket was accepted under the old DEPR process. The updated Process States in OEP-21 help add clarity to communications and timing regarding the introduction of breaking changes. This ticket must move to Transition Unblocked before moving to Breaking Changes Unblocked. The default minimum time to wait in Transition Unblocked is 1 month, so early deployers have time to prepare. If you wish to make breaking changes in less than 1 month, please initiate discussions to negotiate a date as soon as possible. Thank you!

  7. feanil commented on May 30, 2025

    @feanil
    Contributor

    Confirming that after a review, this ticket is currently in the right state. It can move to transition unblocked once we have documented what changes need to be made in edx-platform to switch from using the old xqueue service to using the in-process version. Once that document has been posted and live, we can move this to "Transition Unblocked" and remove support for the old code 1 month after that.

  8. robrap commented on May 30, 2025

    @robrap

    Thanks @feanil.

    1. Consider updating the PR description to match the PR template. It may be not change much in this case though.
    2. Consider using the "Review After Date" to let you and the DEPR WG know when and if we should check in on status. It doesn't mean that the ticket can't move before that. But maybe in a month, or 3 months, or a year, you'd like to see if there is any progress.
  9. 8 remaining items

  10. feanil commented on Mar 5, 2026

    @feanil
    Contributor

    Thanks Dave, I've updated this ticket so that we don't look at it again in DEPR review until a ew weeks after the Verawood cut since I assume, you'll probably close this out around then.

  11. sarina commented on Mar 5, 2026

    @sarina
    Contributor

    @feanil full disclosure we currently don't have a resourcing plan on this DEPR.

  12. feanil commented on Apr 2, 2026

    @feanil
    Contributor

    This work is still not resourced so pushing out the date when we next check in on this to after the conference in May

  13. feanil commented on Jun 12, 2026

    @feanil
    Contributor

    @ormsbee @sarina Aximprovements could take this on potentially, if you could provide more info on what needs to be done? Is there work in openedx-platform or is this mostly archivingthe XQueue repo?

  14. sarina commented on Jun 12, 2026

    @sarina
    Contributor

    Will leave to @ormsbee to detail, but that would be a HUGE assist! Thank you!

  15. moved this from Transition Unblocked to Breaking Changes Unblocked in DEPR: Deprecation & Removalon Nov 12, 2026
  16. kdmccormick commented on Nov 12, 2026

    @kdmccormick
    Member

    @ormsbee has confirmed that all support for the XQueue microservice is safe to be removed!

    XQueue repo archival is happening now: openedx-unsupported/xqueue#991 (comment)

    We don't yet have a specific plan for when XQueue microservice support will be removed from openedx-platform. Aximprovements might be able to pick it up in the future? But if anyone else is interested in doing that part of the removal, please let us know!

    @ormsbee I know we are keeping xqueue-watcher, but do you know if any cleanup should be done in that repo as it no longer needs to talk to the XQueue microservice?

  17. kdmccormick commented on Nov 12, 2026

    @kdmccormick
    Member
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

deprProposal for deprecation & removal per OEP-21

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions