Skip to content

fix: order responses numerically by PK instead of lexicographically - #279

Open
taimoor-ahmed-1 wants to merge 2 commits into
masterfrom
fix/response-sort-numeric-ordering
Open

taimoor-ahmed-1 wants to merge 2 commits into
masterfrom
fix/response-sort-numeric-ordering

Conversation

@taimoor-ahmed-1

Copy link
Copy Markdown
Contributor

Summary

Comment.get_list() sorted responses by sort_key, a CharField that holds the comment's primary key rendered as a string. String comparison gives the lexicographic order \"1\", \"10\", \"11\", \"12\", \"2\", \"3\", … instead of the numeric 1, 2, 3, …, 12. To users this looks as though the response sort dropdown does nothing — picking Newest first or Oldest first produces what appears to be a random shuffle.

This change orders comments at the database layer by integer PK (pk is monotonic with creation time, so PK ascending == oldest first).

Fixes openedx/frontend-app-discussions#823.
Discuss thread (same user reporting): https://discuss.openedx.org/t/discuss-forum-messages-order-not-organized-as-expected-in-teak/18665

What changed

In `forum/backends/mysql/models.py`, `Comment.get_list()`:

  • Before: Pulled all comments into Python and called `sorted(..., key=lambda x: (x.sort_key is None, x.sort_key or ""))` — string sort.
  • After: Uses Django's `queryset.order_by("pk")` / `order_by("-pk")` so MySQL does the ordering numerically.

Behaviour for the two documented inputs is preserved:

  • `sort == 1` (ascending / Oldest first) → `order_by("pk")`
  • `sort == -1` (descending / Newest first) → `order_by("-pk")`
  • `sort is None` (legacy unsorted call) → now also `order_by("pk")`. This is a behaviour change but fixes a latent bug: when `sort=None` and `resp_limit` was set, the old code sliced into an empty `result` list and returned no rows at all. The new code returns a deterministic page.

Why not fix `get_sort_key` to be int-safe instead?

`sort_key` is a denormalised `CharField` of the form `"{pk}"` or `"{parent_pk}-{pk}"`. Sorting by PK avoids the field entirely, removes a Python-level pass over every result, and pushes ordering to the database (so pagination via `[resp_skip:resp_end]` runs against an already-sorted queryset instead of a list slice).

Endorsed/answered responses

The reporter notes that endorsed/answer responses should remain on top regardless of sort. That separation is handled one layer up in edx-platform `lms/djangoapps/discussion/rest_api/api.py` for question threads (`endorsed_responses` + `non_endorsed_responses`). Not in scope for this fix.

Test plan

  • Local Tutor dev (`forum_v2.enable_mysql_backend` on) — created a thread with 12+ responses; sort dropdown now produces correct numeric order in both directions.
  • Suggested follow-up unit test: in `tests/test_backends/test_mysql/test_comments.py` create ≥10 comments and assert `Comment.get_list(sort=1, comment_thread=…)` returns ascending PK order (not lex order). Happy to add it in this PR if reviewers prefer it bundled.

🤖 Generated with Claude Code

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Jun 7, 2026
@openedx-webhooks

Copy link
Copy Markdown

Thanks for the pull request, @taimoor-ahmed-1!

This repository is currently maintained by @openedx/wg-maintainers-forums.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@github-project-automation github-project-automation Bot moved this to Needs Triage in Contributions Jun 7, 2026
@mphilbrick211 mphilbrick211 moved this from Needs Triage to Waiting on Author in Contributions Jun 8, 2026
@mphilbrick211

Copy link
Copy Markdown

Hi @taimoor-ahmed-1! Is this pull request still in progress?

@mphilbrick211 mphilbrick211 added the inactive PR author has been unresponsive for several months label Nov 4, 2026
@mphilbrick211

Copy link
Copy Markdown

Friendly ping on this, @taimoor-ahmed-1

@taimoor-ahmed-1
taimoor-ahmed-1 force-pushed the fix/response-sort-numeric-ordering branch from f4878e9 to 56a1738 Compare November 11, 2026 06:31
Taimoor Ahmed and others added 2 commits November 11, 2026 12:26
Comment.get_list() previously sorted by `sort_key`, a CharField that
holds the comment's primary key rendered as a string. String sort gave
the order "1", "10", "11", ..., "2", "3", ..., which looks to users as
if response sorting is not being applied at all.

Switch to ordering at the database layer by integer PK (monotonic with
creation time): `.order_by("pk")` for ascending (Oldest first) and
`.order_by("-pk")` for descending (Newest first). This also fixes a
latent bug where, with `sort=None` and `resp_limit` set, pagination
sliced into an empty `result` list and returned no rows.

Fixes openedx/frontend-app-discussions#823

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Add an API-level regression test that creates twelve responses on a
thread and checks they come back in numeric id order for both
reverse_order values. Against the previous sort_key string sort this
fails with "1, 10, 11, 12, 2, ...".

Co-Authored-By: Claude Fable 5.1 <[email protected]>
@taimoor-ahmed-1
taimoor-ahmed-1 force-pushed the fix/response-sort-numeric-ordering branch from 56a1738 to 0201b86 Compare November 11, 2026 07:26

@Faraz32123 Faraz32123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix looks good to me, ordering by pk matches the old sort_key order, minus the string sort. Couple of small things inline.

The description needs a small update too: the unit test is already added, and the models.py link 404s (it's under src/).

comments_qs = comments_qs.order_by("pk")

# Paginate in the database: an open-ended slice when no limit is given.
resp_end = None if resp_limit is None else resp_skip + resp_limit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Negative resp_skip now 500s here (Django can't slice a queryset with a negative index), e.g. resp_skip=-1, or page=0 in get_user_comments. Maybe clamp with max(0, ...)?


ascending = ids(sort=1)
assert len(ascending) == 12
assert ascending == sorted(ascending)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only fails on the old code because the ids go from 1 digit to 2. Maybe add assert ascending != sorted(ascending, key=str) so it can't pass by accident?

return [content.to_dict() for content in paginated_comments]
comments_qs = Comment.objects.filter(**kwargs)

# Order by primary key, which is monotonically increasing with creation

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the history here could go in the commit message, one line on why pk is enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inactive PR author has been unresponsive for several months open-source-contribution PR author is not from Axim or 2U

Projects

Status: Waiting on Author

Development

Successfully merging this pull request may close these issues.

[Bug] Response sort option is not working

4 participants