Repository navigation
Reference publications by their identifier so citations can be written under RLS #945
Description
Summary
A publication is a shared entity. Score sets, experiments, experiment sets and calibrations cite it, and the ui renders it on its own page. Under RLS a publication will be visible only when the caller can see a record that cites it (terms in the glossary on #808).
Today a citation stores the publication's numeric id, and getting that id means reading the publication row. Under the policy, that read fails in two cases:
- The publication is cited only by records the caller can't see. Find-or-create misses the hidden row and inserts a duplicate. Once both rows become visible,
.one_or_none()raises, and the request fails with a 500. - The publication is brand new. The ORM's
INSERT … RETURNING idmust see the row it just wrote, and nothing cites that row yet, so the insert fails.
This issue makes citations reference the publication by its natural key, (db_name, identifier), so recording one never needs to read the publication row.
Scope
-
Unique key. Add a unique constraint on
publication_identifiers (db_name, identifier). There are no duplicates today. Crossref rows already haveidentifierequal todoi, so the key also covers the Crossref lookup bydoi. -
Citation tables. Replace
publication_identifier_idwith(db_name, identifier), a foreign key to the new unique key. Keep theprimaryflag. The tables:scoreset_publication_identifiersexperiment_publication_identifiersexperiment_set_publication_identifiersscore_calibration_publication_identifiers
publication_identifiers.idstays as the primary key, and stays in api responses. -
"Ensure, then cite" replaces find-or-create.
find_or_create_publication_identifierbecomes:- Resolve the article externally, as now.
- Insert it with
INSERT … ON CONFLICT DO NOTHING. Use a Core insert with no named conflict target and noRETURNING, neversession.add(). - Write the citation rows by natural key.
Update the callers in
routers/experiments.py,routers/score_sets.pyandlib/score_calibrations.py. The calibration code maps associations by(relation, publication_identifier_id)and moves to the natural key. -
Internal lookups. The internal lookups in
find_generic_articletreat a hidden row as absent and fall back to the external fetch. The response for a hidden publication must match the response for one MaveDB doesn't have. -
Unlinked rows. Add a worker task that deletes publication rows nothing cites. Write RLS policies for every guarded table as migrations #809's policy has no "or unlinked" clause, so these rows are invisible to the api.
The policy itself is written in #809: a publication is visible if a citation the caller can see exists in any of the four citation tables. It has no NOT EXISTS clause, because a caller who can't see the private citations would read them as "no citations".
Acceptance criteria
-
publication_identifiershas a unique(db_name, identifier), and the four citation tables reference it in place ofpublication_identifier_id. - The migration replays from an empty database (
alembic/README.md), and its downgrade restores the id references. - Citing a publication writes no
RETURNINGonpublication_identifiersand reads no publication row. - With Write RLS policies for every guarded table as migrations #809's policy enabled on
publication_identifiers, a test running as a non-owner role shows:- citing a publication already cited only by another user's private record creates no duplicate, and the citation resolves to the existing row;
- citing a brand-new publication succeeds;
/publication-identifiers/search-externalgives the same response for a hidden publication as for one not in MaveDB.
- The worker task removes uncited publication rows.
- Existing publication routes and their api responses keep their shape.
Background
Behaviour checked on PG 15, as a role that can't see an existing publication-like row:
| Operation | Result |
|---|---|
| A citation referencing a hidden row by natural key | works: foreign-key checks ignore row security |
| A citation of an identifier that doesn't exist | rejected by the foreign key |
INSERT … ON CONFLICT DO NOTHING, no conflict target, row exists but hidden |
silent no-op |
INSERT … ON CONFLICT (identifier) DO NOTHING |
fails: a named target needs SELECT, so the new row is checked against the read policy |
Plain insert of a new row, no RETURNING |
works |
INSERT … RETURNING id of a new, uncited row |
fails |
On the 2026.3.0 dump there are 174 publications, and 3 of them are cited by nothing.
The alternatives considered, and why they were rejected:
| Alternative | Why not |
|---|---|
| Filter the publication routes in Python and leave the table without a policy | Every new route has to remember the filter |
| Store a copy of the publication on each citation | Loses the publication entity the ui renders |
A SECURITY DEFINER id lookup |
#808 rules out definer functions, and it would reveal that the row exists |
| Deterministic UUID ids | Changes the api's id type and hides the same normalization problem in a hash |
| Let the worker resolve citations | Makes citing asynchronous |
The DOI, raw read and gene identifier tables aren't rendered as entities. #944 folds them into their link rows instead.
Activity
feat(db): store DOI, raw read and target gene identifiers on the rows…