Skip to content

Add control_quantile_mapping: control-anchored per-population quantile maps - #152

Open
Ephemeral6 wants to merge 2 commits into
openproblems-bio:mainfrom
Ephemeral6:method/control-quantile-mapping
Open

Ephemeral6 wants to merge 2 commits into
openproblems-bio:mainfrom
Ephemeral6:method/control-quantile-mapping

Conversation

@Ephemeral6

Copy link
Copy Markdown

Add new method: control_quantile_mapping

Method

The control samples (is_control > 0) are aliquots of the same donor measured in every batch. The differences
between their per-marker distributions across batches are therefore technical. For every batch and every marker
with to_correct:

  1. Global map. A monotone quantile map F_ref^-1(F_b(x)) sends the control cells of batch b onto the donor's
    reference distribution: the 1D Wasserstein barycenter over batches, i.e. a "mid" target. Maps are averaged over
    control donors.
  2. Population maps. The same maps are learned per cell population (k-means, k = 40, on the globally corrected
    lineage markers) and shrunk towards the global map with weight n / (n + 200).
  3. All-cells blend. Each population map is averaged 50/50 with the quantile map computed from all cells of that
    population. This keeps the correction stable when the controls are in a different biological state than the
    other samples.

The maps are applied to every cell of the batch; other markers are returned unchanged. The method uses only batch,
is_control, the preprocessed layer and the var to_correct / marker_type columns of the censored input.
Python only (numpy, scipy, scikit-learn); a few minutes per split on a laptop CPU.

Local benchmark

I scored the method with the metric components of the published run (commit 3321360, 9 metrics) and scaled the
scores with the control methods as on the website (mean(clip(scaled, 0, 1))). Published values were taken from
that run's score_uns.yaml.

Pipeline check.

  • no_integration reproduces all 27 published dataset × metric values (max difference 4.8e-7).
  • cycombine_no_controls_to_mid re-run locally reproduces its published values exactly: all metrics on
    human_blood and lille; batch R² and EMD on mouse_spleen (the remaining mouse metrics were not re-run).
method human_blood lille mouse_spleen mean
control_quantile_mapping (this PR, mean of 3 seeds) 0.7310 0.7858 0.7664 0.7611 (sd 0.0034)
cycombine_no_controls_to_mid (published best) 0.7315 0.7609 0.7383 0.7436
cycombine_all_controls_to_mid 0.7407
  • Per seed (k-means seed): 0.7579 / 0.7647 / 0.7606, i.e. +0.014 / +0.021 / +0.017 over the published best.
  • Paired bootstrap of the official horizontal EMD against the locally reproduced cyCombine: 95% CI
    [0.0075, 0.0133].
  • Per metric: the gain comes from same-donor cross-batch agreement (batch R², horizontal EMD, Cohen's d), on every
    dataset.

Caveats (please read)

  • Gain is concentrated in two datasets. lille (+0.023 to +0.030) and mouse_spleen (+0.016 to +0.040). On
    human_blood it ties with cyCombine, and cytonorm_no_controls_to_mid is better there (0.7506).
  • Metric losses. It loses to cyCombine on mouse iLISI (scaled 0.69–0.71 vs 0.77), human ratio of consistent
    peaks (about 0.70 vs 0.76) and human FlowSOM mapping similarity (0.67–0.68 vs 0.70).
  • Below the pre-registered margin. Before running the final evaluation I set myself a bar of +0.02 over the
    published best. The result, +0.0175, is just below it; I'm submitting because it beats every published method on
    the leaderboard metric, consistently across seeds.
  • Seed sensitivity. Mouse_spleen varies with the k-means seed (0.754–0.779); only 3 seeds were run.
  • Development data. Hyperparameters were chosen on folds built from the control donors of the same three
    datasets. Non-control donors were relabelled so the metrics ignored them, and their labels were never used. The
    configuration was frozen before the final runs. Still, this is not an independent dataset.
  • Needs controls. The method needs control donors present in every batch (guaranteed by this task's API).
  • Not compared on main. The comparison is against the published run (9 metrics). Since then main has added
    abundance_preservation (Add DA metric #143), the R sva ComBat (Adapt combat #149) and parameter-optimisation ranges (Note on parameter optimization added in config files #151), which may change
    the ranking. abundance_preservation was not scored here.
  • Not run locally. viash test was not run (no viash/Docker on this machine); the script was run with the
    config's default arguments by injecting par the way viash does. I'm happy to fix anything CI flags.

Changes

  • src/methods/control_quantile_mapping/ (config.vsh.yaml, script.py).
  • Registered in src/workflows/run_benchmark/config.vsh.yaml and main.nf.
  • CHANGELOG entry.

AI assistance disclosure

An AI agent (Anthropic Claude, via Claude Code) designed, implemented, evaluated and wrote up this method, operating
the Ephemeral6 account. All numbers come from logged runs. A separate agent independently recomputed the scores from
the raw metric outputs, re-ran the official metrics and the method, and checked for label leakage.

🤖 Generated with Claude Code

panguancheng and others added 2 commits October 30, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant