Repository navigation
feat: return taxonomy_type on the object tags endpoint - #39259
Conversation
|
Thanks for the pull request, @ufedaseyeuconsultant! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
|
While writing the query-count test for this PR, I found a pre-existing N+1 on the same endpoint, and I'd like to open a follow-up issue for it rather than fix it here. What happens. Measured on Where it comes from. #34200 added the loop over prefetched Proposed fix, for the follow-up. Change the three Why not in this PR. The cost comes from the permission check and the filter backend, not from Does that sound reasonable? If so, I'll open the issue and link it. |
jesperhodge
left a comment
There was a problem hiding this comment.
looks great. Manual testing worked.
I find your observation of the N+1 query very observant. Please open an issue and PR for it in our project.
AI found only one really small issue and a few nits:
"
Must fix before merge
The change depends on an internal detail of another library that an accepted design decision says this project must not depend on.
Details
In ObjectTagOrgView.get_queryset (openedx/core/djangoapps/content_tagging/rest_api/v1/views.py) the queryset gets .select_related("taxonomy__competencytaxonomy"). ADR 0013 says openedx-platform must reach this relation only through the CBE app's public API, and this string is the only place competencytaxonomy appears in openedx-platform; TaxonomyOrgView on master uses select_competency_taxonomies(). Your code comment is accurate that the helper only accepts a Taxonomy queryset and this one holds ObjectTag rows.
Two things soften the risk without removing it. A rename upstream would make Django raise FieldError here, so the platform tests would fail at the pin bump rather than silently. But the join only prevents the per-taxonomy query because is_competency_taxonomy happens to be implemented as hasattr on that accessor; if openedx-core reimplemented the helpers, the join would stop helping and only your new query-count test would notice. The PR body also cites the taxonomy__taxonomyorg_set prefetch as the same pattern, but TaxonomyOrg is openedx-platform's own model, so the precedent does not apply.
Pick one:
| Option | Extra queries per request | Follows ADR 0013 | Work outside this PR |
|---|---|---|---|
B. Batch the check in the serializer: collect the entries' non-null taxonomy_id values, run select_competency_taxonomies(Taxonomy.objects.filter(pk__in=ids)) once, record which pks is_competency_taxonomy says are competency, and delete the view override. |
One, whatever the number of taxonomies. Django skips it for an empty id set. | Yes | None. Expected numbers in test_object_tags_query_count go up by one for the cases with tags. |
C. Add a CBE API variant in openedx-core that takes a lookup prefix or an ObjectTag queryset, and use it here. |
None | Yes, and the ADR expects this growth | An openedx-core release and a pin bump. Correct #783's wording to match the ADR. |
| A. Keep the code. | None | No, an explicit exception | None, but the get_queryset comment should say it is an exception and link a follow-up issue for option C, and the "same pattern" sentence should leave the PR body. |
Option B is the smallest change that follows the ADR. It also removes the hidden dependency the serializer docstring warns about ("costs no extra query only if the taxonomies were loaded with their competency row joined, as ObjectTagOrgView does"). A Prefetch on taxonomy with a custom queryset does not work, because get_object_tags() already uses select_related("taxonomy") and Django skips a prefetch for an already-cached relation.
Improvements worth doing
One new test checks less of the response than the acceptance criterion it covers.
Details
test_get_tags_untagged_object asserts that the helper's return value, response.data[object_id]["taxonomies"], is empty. The criterion says the response "contains no taxonomy type information". The test did fail when the review made the serializer emit an entry for an untagged object, so the criterion counts as covered, but it cannot see anything outside that list. Assert the whole payload: response.data == {str(self.courseA): {"taxonomies": []}}.
A claim in the pull request description has no test behind it.
Details
The PR body says PUT responses include the field because update returns through retrieve. The review confirmed it by reading the base view and by a throwaway PUT that returned taxonomy_type: "competency"; removing the join added exactly one query per taxonomy on PUT too. Add one PUT assertion on taxonomy_type.
A defensive check handles a state this endpoint cannot produce, and its comment gives an incomplete reason.
Details
In the new to_representation, taxonomy is not None and is_competency_taxonomy(taxonomy) carries the comment "A tag whose taxonomy was deleted has no taxonomy, so it can't be a competency". The model facts are right. But get_object_tags() excludes tags with taxonomy=None unless asked for deleted ones, and the view never asks. The review tagged a course, deleted the taxonomy, and fetched the tags as two different users: the response was {"taxonomies": []}. The commit message for e607ed3c1d says the guard also keeps mypy happy, which the comment omits, and the comment sits above the dictionary lookup rather than the guard. Either drop the guard and comment, or keep the guard and reword the comment to its real reason (type narrowing, and safety for a direct serializer call) on the guard's line.
The endpoint's documentation does not mention the new field.
Details
ObjectTagOrgView's docstring says "Refer to ObjectTagView docstring for usage details." That base docstring lists each entry's taxonomy_id, can_tag_object, and tags, and under ADR 0013 it can never list taxonomy_type. Add one sentence naming taxonomy_type and its two values.
Two of the three new tests for the field's values fail only when the third also fails.
Details
Every break that failed test_get_tags_taxonomy_type_competency or test_get_tags_taxonomy_type_ordinary also failed test_get_tags_taxonomy_type_mixed. The three map one-to-one to the ticket's first three scenarios, which has traceability value, so keeping them is defensible. To have fewer tests, keep the mixed one and name it after criteria 1 to 3.
ff9abd3 to
036797c
Compare
|
@jesperhodge ADR 0013 dependency (must fix). Went with option B. Untagged-object test. PUT claim. Added Defensive check on a deleted taxonomy. Gone, together with its comment. The batch only collects non-null taxonomy ids, so a tag without a taxonomy simply reports Endpoint docs. The Three value tests. I kept them, since each maps to one of the ticket's first three scenarios. N+1 on org-owned taxonomies. I'll open the issue. |
jesperhodge
left a comment
There was a problem hiding this comment.
@ufedaseyeuconsultant awesome work!
In the future, if you could make sure that you have a separate commit for the review fixes so I can tell what happened after the review instead of before, that would be amazing.
036797c to
9aefb8b
Compare
mgwozdz-unicon
left a comment
There was a problem hiding this comment.
@ufedaseyeuconsultant the change itself looks good: it follows ADR 0013, every acceptance criterion of openedx/openedx-core#783 has a test behind it, and CI is green. Following are the findings from my and Claude's review:
-
Merge conflict in
rest_api/v1/serializers.py. #39369 changed theCOURSES_MANAGE_TAGSimport onmaster, on the line next to the import this PR extends. Please rebase, keepingfrom openedx_authz.authz import COURSES_MANAGE_TAGSfrommasterand yourfrom openedx_learning.api import is_competency_taxonomy, select_competency_taxonomies. -
test_get_tags_query_count_independent_of_taxonomy_type_checktakes a roundabout path to its assertion. The loop over two courses, the list of counts, andquery_counts[0] == query_counts[1]make the reader work out which course has one taxonomy and which has three. Counting the queries for the same object twice, once with one taxonomy and once after adding two more, and then comparing two named counts, reads more directly. A sketch is below. I have not run it. With it, theuserparameter of_get_taxonomy_entrieshas no caller and can go.
def test_get_tags_query_count_independent_of_taxonomy_type_check(self):
"""
The query count does not grow with the number of taxonomies, competency ones included.
All taxonomies here are all-org on purpose. For org-owned taxonomies, ``can_tag_object`` makes its own
organization lookups, a separate cost that this test leaves out so it isolates the cost of ``taxonomy_type``.
"""
object_id = str(self.courseA)
url = OBJECT_TAGS_URL.format(object_id=object_id)
self.client.force_authenticate(user=self.staff)
tagging_api.tag_object(object_id=object_id, taxonomy=self.t1, tags=["android"])
self.client.get(url) # Warm up the site and waffle flag caches, as test_object_tags_query_count does.
with CaptureQueriesContext(connection) as queries_with_one_taxonomy:
self.client.get(url)
competency = self._create_competency_taxonomy()
extra = tagging_api.create_taxonomy(name="Extra all-org taxonomy", enabled=True)
Tag.objects.create(taxonomy=extra, value="Extra")
TaxonomyOrg.objects.create(taxonomy=extra, org=None, rel_type=TaxonomyOrg.RelType.OWNER)
tagging_api.tag_object(object_id=object_id, taxonomy=competency, tags=["Triage"])
tagging_api.tag_object(object_id=object_id, taxonomy=extra, tags=["Extra"])
with CaptureQueriesContext(connection) as queries_with_three_taxonomies:
response = self.client.get(url)
assert len(response.data[object_id]["taxonomies"]) == 3
assert len(queries_with_three_taxonomies) == len(queries_with_one_taxonomy)-
if obj_tag.taxonomy_id is not NoneinObjectTagOrgByTaxonomySerializer.to_representationguards a state this endpoint cannot produce.get_object_tagsexcludes tags whose taxonomy was deleted unless the caller asks for them, and this view never does. Please drop the condition. -
The
to_representationdocstring says what the code does and leaves out why. The second paragraph could be one sentence giving the reason for the batch:is_competency_taxonomycosts one query per taxonomy unless the taxonomies are loaded throughselect_competency_taxonomies. That is what stops a later reader from simplifying it back into a per-taxonomy check. -
PR description. It says the pinned openedx-core is 1.17.0, and the pin on this branch is 1.21.0.
Each taxonomy entry returned by GET /api/content_tagging/v1/object_tags/{object_id}/
now carries taxonomy_type ("competency" or "tags"), computed with the CBE
public API is_competency_taxonomy, the same way TaxonomyOrgSerializer does.
ObjectTagOrgView joins each taxonomy's competency row so the check adds no
per-taxonomy query.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
ObjectTag.taxonomy is nullable, since deleting a taxonomy leaves its tags with no taxonomy. Report such entries as "tags", which also satisfies mypy. Co-Authored-By: Claude Opus 5.5 <[email protected]>
ObjectTagOrgView joined taxonomy__competencytaxonomy into its queryset, reaching a CBE model relation directly, which ADR 0013 rules out. The serializer now loads the object tags' taxonomies once through select_competency_taxonomies() and asks is_competency_taxonomy() which of them are competency taxonomies, so the check is one query per response, and none for an untagged object. The view override is gone. Also document taxonomy_type on ObjectTagOrgView, assert the whole payload for an untagged object, and cover taxonomy_type on PUT. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The query-count test now requests the same object before and after adding a competency taxonomy and an all-org taxonomy to it, and asserts the query count is unchanged. The earlier version compared two different objects, so differences between the objects could hide a per-taxonomy query. The serializer no longer skips tags without a taxonomy. The object tags endpoint never returns them, because the object-tag lookup excludes tags whose taxonomy was deleted and this view does not ask for them. The to_representation docstring now gives the reason for the batch: is_competency_taxonomy costs one query per taxonomy unless the taxonomies are loaded through select_competency_taxonomies. Co-authored-by: Mary Gwozdz <[email protected]> Co-Authored-By: Claude Fable 5 <[email protected]>
9aefb8b to
87ade87
Compare
|
@mgwozdz-unicon Done |
Description
Closes openedx/openedx-core#783.
GET /api/content_tagging/v1/object_tags/{object_id}/returns a content object's tags grouped by taxonomy. Each taxonomy entry in that response now has ataxonomy_typefield. Its value is"competency"when the taxonomy is a competency taxonomy and"tags"otherwise. The same field already exists on the taxonomy endpoints (openedx/openedx-core#618), and it uses the same values. The Course Outline and Libraries pages read this endpoint through the sharedcontent-tags-drawermodule offrontend-app-authoring. They need the field to hide the remove control on competency tags (openedx/openedx-core#784).{"block-v1:...": {"taxonomies": [ {"name": "Nursing Competencies", "taxonomy_id": 7, "export_id": "nursing-competencies", "can_tag_object": true, "taxonomy_type": "competency", "tags": [...]} ]}}How it works
ObjectTagOrgByTaxonomySerializer.to_representation()calls the baseopenedx_taggingserializer. It then addstaxonomy_typeto each taxonomy entry, using the CBE public APIis_competency_taxonomyand theTaxonomyTypeenum, exactly asTaxonomyOrgSerializerdoes. The rule appears in two places, so I called the same check directly rather than extracting a shared helper. A third caller would justify the helper.is_competency_taxonomychecks whether the taxonomy has a competency row, and without a join that check costs one query per taxonomy. So the serializer collects the taxonomy ids of the object tags, loads those taxonomies once through the CBE public helperselect_competency_taxonomies(), and records which idsis_competency_taxonomyreports as competency. The check costs one query per response, whatever the number of taxonomies. For an untagged object the id set is empty, and Django runs no query.PUTon the same endpoint returns its result through the same retrieve path, so its response includes the field too.ObjectTagOrgViewdocstring namestaxonomy_typeand its two values.openedx_taggingor the CBE app changes, per openedx-core ADR 0013.Who is affected
Supporting information
taxonomy_typeon the taxonomy endpoints. This PR reuses its computation.docs/openedx_tagging/decisions/0013-competency-taxonomy-detection.rst: why competency detection lives in openedx-platform and the CBE app, not inopenedx_tagging.Testing instructions
Automated, from a CMS test environment:
The new tests in
TestObjectTagViewSet:"competency". An ordinary taxonomy's entry reports"tags".{"<object_id>": {"taxonomies": []}}.PUTthat tags an object from a competency taxonomy returns"competency"on that entry.test_get_tagsandtest_get_copied_tagsnow expect the extra key. Every other existing key and value is unchanged. Every case oftest_object_tags_query_countexpects one more query, the competency check.Manual:
GET /api/content_tagging/v1/object_tags/<usage_key>/as a course author.taxonomy_type. It reads"competency"for the competency taxonomy and"tags"for the other one. All other fields are as before.Deadline
None.
Other information
is_competency_taxonomyandselect_competency_taxonomies). The pinned openedx-core 1.23.0 already has both.can_tag_objectpermission check runs once per taxonomy, and the prefetch loads the taxonomy's ownership rows but not their organizations, so each org-owned taxonomy fetches its organization separately. The count is the same with and without this PR. So the new query-count test uses taxonomies available to all orgs, which measures only the cost oftaxonomy_type. I propose a follow-up issue for the org-owned cost, described in a comment below.🤖 Generated with Claude Code