Repository navigation
perf: bulk-prefetch persisted grades in ProblemGradeReport - #38944
blarghmatey wants to merge 4 commits into
Conversation
…arner caches
ProblemGradeReport read persisted grades one learner at a time, unlike
CourseGradeReport, whose _rows_for_users opens with a _CourseGradeBulkContext.
With nothing prefetched, PersistentSubsectionGrade.bulk_read_grades missed the
per-course RequestCache and fell through to a query per learner -- and this
report walks course_grade.problem_scores, so it touches more subsection data per
learner than the course report does.
Measured with CaptureQueriesContext over the report, varying cohort size:
learners 5 10 25 50
before 20 30 60 110 (fits 2N + 10)
after 13 13 13 13
Adds a regression test asserting the flat count at two cohort sizes; a
single-size assertion would pass while a per-learner read crept back in.
_ProblemGradeBulkContext is deliberately narrower than _CourseGradeBulkContext:
the problem report emits no cohort, team, certificate or course-tag columns, so
prefetching those would be wasted work. It loads persisted course and subsection
grades plus enrollment states, which is what the rows actually read.
Also in the same report path:
* Pass the already-loaded block structure to grading_context() instead of
calling grading_context_for_course(), which re-enters get_course_in_cache.
BlockStructureManager.get_collected() deserializes fresh from the cache
backend on every call, so a large course paid that cost, and its peak
allocation, twice. CourseGradeReport already does this correctly.
* Clear the visible-blocks and subsection-override caches between batches.
RequestCache is only flushed when the task ends, and unlike the course-keyed
grade prefetches -- replaced wholesale each batch -- these are keyed per
(user, course) and only ever grow, though nothing reads an entry again once
that learner's row is written. The _clear_caches hook already existed and
was called per batch; the base implementation was empty. Adds the missing
clear_prefetched_data() counterparts to VisibleBlocks and
PersistentSubsectionGradeOverride, and a
clear_prefetched_grade_overrides_and_visible_blocks() api function pairing
with the existing prefetch_grade_overrides_and_visible_blocks().
* Wire USER_BATCH_SIZE through to grouper(). It was defined on
CourseGradeReport but never referenced, so the effective batch size was
grouper's hardcoded default and tuning the constant did nothing. Moved to
GradeReportBase so both reports honour it.
CSV contents and format are unchanged by all of the above.
Refs: openedx#38943
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ki1NYMjSjVB4uz1gdgsjBh
|
Thanks for the pull request, @blarghmatey! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
|
@openedx/wg-maintenance-openedx-platform-oncall — this is ready for engineering review. Checklist from the bot above:
Short version: Also folds in three smaller fixes from the same issue: a double block-structure deserialization, two per-learner RequestCaches that were never evicted across a long task, and a Related: #38946 does the remaining finding from #38943 (streaming the report upload). Independent of this one — they touch different parts of |
There was a problem hiding this comment.
🟡 Changes recommended
Tests do not exercise persisted problem grades or verify cache eviction between batches, leaving the intended N+1 prevention and bounded cache retention without adequate regression coverage.
2 open issues
What changed in this PR
This PR improves asynchronous problem grade reporting by bulk-loading learner data and bounding per-learner caches, addressing four performance issues from #38943. Shared report-upload streaming remains out of scope.
Changes:
- Prefetches grades and enrollment states per batch and reuses the loaded course structure.
- Clears visible-block and override caches between batches and makes both reports honor
USER_BATCH_SIZE. - Adds a query-count regression test at two learner population sizes.
| File | Description |
|---|---|
lms/djangoapps/instructor_task/tests/test_tasks_helper.py |
Adds query-count coverage across two learner populations. |
lms/djangoapps/instructor_task/tasks_helper/grades.py |
Adds bulk prefetching, structure reuse, shared cache cleanup, and configurable batching. |
lms/djangoapps/grades/models.py |
Adds cache-clearing methods for visible blocks and subsection overrides. |
lms/djangoapps/grades/models_api.py |
Exposes a helper to clear both per-learner caches. |
Review effort: Balanced
The grade reports read grades through CourseGradeFactory.read and SubsectionGradeFactory.create(read_only=True). That path never fills the VisibleBlocks or PersistentSubsectionGradeOverride RequestCache entries: only update(force_update_subsections=True), update_or_create_grade and bulk_create_grades do. With persisted grades for every learner and a batch size of 2, both caches held 0 entries before and after every batch for ProblemGradeReport and CourseGradeReport, so the clear between batches removed nothing. Claude-Session: https://claude.ai/code/session_01FGZebF8pmhpsgn5eqD2G83
The query-count test had no graded problems, so the report had no problem columns and never read a subsection grade. With a graded problem every learner has answered, the count is 8N + 16 (56 at 5 learners, 216 at 25) against 11N + 14 (69, 289) before the bulk prefetch, so the test pins both counts instead of asserting a flat one. Claude-Session: https://claude.ai/code/session_01FGZebF8pmhpsgn5eqD2G83

What
Addresses three of the five findings in #38943. Finding 4 (streaming the report upload through
DjangoStorageReportStore.store) is in #38946, because it touches shared storage code used by every instructor-task report, not only grade reports. Finding 3 (per-learner caches) turned out not to apply to the grade reports, so this PR leaves it out; see below.Why it matters
ProblemGradeReportread persisted grades one learner at a time, unlikeCourseGradeReport, whose_rows_for_usersopens with a_CourseGradeBulkContext. With nothing prefetched,PersistentSubsectionGrade.bulk_read_gradesmissed the per-course RequestCache and fell through to a query per learner, and this report walkscourse_grade.problem_scores, so it touches more subsection data per learner than the course report does.Measured with
assertNumQueries(authz tables ignored, as in the file's other query-count tests) around the report, in a course with a graded problem that every learner has answered, so each one has persisted course and subsection grades:Before fits
11N + 14, after fits8N + 16. The prefetch removes three queries per learner; what still scales is read per learner while computing problem scores (courseware state, submission scores and the course access checks), and is out of scope here. At the report's 100-learner batch that is about 816 queries per batch against 1,114 before.Changes
Finding 1, bulk prefetch. New
_ProblemGradeBulkContext, deliberately narrower than_CourseGradeBulkContext: the problem report emits no cohort, team, certificate or course-tag columns, so prefetching those would be wasted work. It loads persisted course and subsection grades plus enrollment states, which is what the rows read.Finding 2, double block-structure load.
graded_scorable_blocks_headernow passes the already-loaded structure tograding_context()instead of callinggrading_context_for_course(), which re-entersget_course_in_cache.BlockStructureManager.get_collected()deserializes fresh from the cache backend on every call, so a large course paid that cost, and its peak allocation, twice.CourseGradeReportalready does this correctly.Finding 5, dead constant.
USER_BATCH_SIZEwas defined onCourseGradeReportbut never referenced, so the effective batch size wasgrouper's hardcoded default and tuning the constant did nothing. Moved toGradeReportBaseand wired through, so both reports honour it.Finding 3 is not addressed. The grade reports read through
CourseGradeFactory.readandSubsectionGradeFactory.create(read_only=True), and that path never fills the visible-blocks or subsection-override RequestCache entries; onlyupdate(force_update_subsections=True),update_or_create_gradeandbulk_create_gradesdo. With persisted grades for every learner and a batch size of 2, both caches held 0 entries before and after every batch for bothProblemGradeReportandCourseGradeReport, so a clear between batches would remove nothing.Testing
lms/djangoapps/instructor_task/tests/test_tasks_helper.py: 104 passed.lms/djangoapps/grades/tests/: 777 passed.TestProblemGradeReport::test_query_count_per_learnerpins 56 queries at 5 learners and 216 at 25. Against the pre-PRgrades.pyit fails with 69 executed, 56 expected. Asserting two cohort sizes means a change in the per-learner cost fails the test, not only a change in the fixed cost.TestInstructorGradeReport::test_query_countsassertion forCourseGradeReport(48 queries) still passes unmodified, so the sharedUSER_BATCH_SIZEchange does not alter that report.Backward compatibility
CSV contents and format are unchanged. No schema or data migration, and no new public API.
Context
Found while investigating a 5,154-learner course whose problem grade report took about 80 minutes end to end, of which the CSV upload was 0.17s, so nearly all of it was in the per-learner compile loop. Related: #38911 proposes bringing the same
CourseGradeReportbulk-prefetch pattern to the synchronous CCX grade report; same defect class, different code path.