Skip to content

Reference publications by their identifier so citations can be written under RLS #945

Open

Description

@bencap

Summary

A publication is a shared entity. Score sets, experiments, experiment sets and calibrations cite it, and the ui renders it on its own page. Under RLS a publication will be visible only when the caller can see a record that cites it (terms in the glossary on #808).

Today a citation stores the publication's numeric id, and getting that id means reading the publication row. Under the policy, that read fails in two cases:

  • The publication is cited only by records the caller can't see. Find-or-create misses the hidden row and inserts a duplicate. Once both rows become visible, .one_or_none() raises, and the request fails with a 500.
  • The publication is brand new. The ORM's INSERT … RETURNING id must see the row it just wrote, and nothing cites that row yet, so the insert fails.

This issue makes citations reference the publication by its natural key, (db_name, identifier), so recording one never needs to read the publication row.

Scope

  1. Unique key. Add a unique constraint on publication_identifiers (db_name, identifier). There are no duplicates today. Crossref rows already have identifier equal to doi, so the key also covers the Crossref lookup by doi.

  2. Citation tables. Replace publication_identifier_id with (db_name, identifier), a foreign key to the new unique key. Keep the primary flag. The tables:

    • scoreset_publication_identifiers
    • experiment_publication_identifiers
    • experiment_set_publication_identifiers
    • score_calibration_publication_identifiers

    publication_identifiers.id stays as the primary key, and stays in api responses.

  3. "Ensure, then cite" replaces find-or-create. find_or_create_publication_identifier becomes:

    1. Resolve the article externally, as now.
    2. Insert it with INSERT … ON CONFLICT DO NOTHING. Use a Core insert with no named conflict target and no RETURNING, never session.add().
    3. Write the citation rows by natural key.

    Update the callers in routers/experiments.py, routers/score_sets.py and lib/score_calibrations.py. The calibration code maps associations by (relation, publication_identifier_id) and moves to the natural key.

  4. Internal lookups. The internal lookups in find_generic_article treat a hidden row as absent and fall back to the external fetch. The response for a hidden publication must match the response for one MaveDB doesn't have.

  5. Unlinked rows. Add a worker task that deletes publication rows nothing cites. Write RLS policies for every guarded table as migrations #809's policy has no "or unlinked" clause, so these rows are invisible to the api.

The policy itself is written in #809: a publication is visible if a citation the caller can see exists in any of the four citation tables. It has no NOT EXISTS clause, because a caller who can't see the private citations would read them as "no citations".

Acceptance criteria

  • publication_identifiers has a unique (db_name, identifier), and the four citation tables reference it in place of publication_identifier_id.
  • The migration replays from an empty database (alembic/README.md), and its downgrade restores the id references.
  • Citing a publication writes no RETURNING on publication_identifiers and reads no publication row.
  • With Write RLS policies for every guarded table as migrations #809's policy enabled on publication_identifiers, a test running as a non-owner role shows:
    • citing a publication already cited only by another user's private record creates no duplicate, and the citation resolves to the existing row;
    • citing a brand-new publication succeeds;
    • /publication-identifiers/search-external gives the same response for a hidden publication as for one not in MaveDB.
  • The worker task removes uncited publication rows.
  • Existing publication routes and their api responses keep their shape.
Background

Behaviour checked on PG 15, as a role that can't see an existing publication-like row:

Operation Result
A citation referencing a hidden row by natural key works: foreign-key checks ignore row security
A citation of an identifier that doesn't exist rejected by the foreign key
INSERT … ON CONFLICT DO NOTHING, no conflict target, row exists but hidden silent no-op
INSERT … ON CONFLICT (identifier) DO NOTHING fails: a named target needs SELECT, so the new row is checked against the read policy
Plain insert of a new row, no RETURNING works
INSERT … RETURNING id of a new, uncited row fails

On the 2026.3.0 dump there are 174 publications, and 3 of them are cited by nothing.

The alternatives considered, and why they were rejected:

Alternative Why not
Filter the publication routes in Python and leave the table without a policy Every new route has to remember the filter
Store a copy of the publication on each citation Loses the publication entity the ui renders
A SECURITY DEFINER id lookup #808 rules out definer functions, and it would reveal that the row exists
Deterministic UUID ids Changes the api's id type and hides the same normalization problem in a hash
Let the worker resolve citations Makes citing asynchronous

The DOI, raw read and gene identifier tables aren't rendered as entities. #944 folds them into their link rows instead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    app: backendTask implementation touches the backendapp: databaseTask implementation requires database changes

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions