diff --git a/CHANGELOG.md b/CHANGELOG.md index da37338..f723656 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,55 +1,49 @@ -# task_template x.y.z - -## BREAKING CHANGES - - - -* `run_benchmark`: replaced `--method_ids` with `--methods_include`/`--methods_exclude` and added `--metrics_include`/`--metrics_exclude` (PR #20). - -## NEW FUNCTIONALITY - -* Added `control_methods/true_labels` component (PR #5). - -* Added `methods/logistic_regression` component (PR #5). - -* Added `metrics/accuracy` component (PR #5). - -* Added `control_methods/random_labels` component (PR #22). - -* `run_benchmark`: run parameterised methods once per named paramset from `info.variants` or the new `--paramsets` file, tag scores with `paramset_name`/`paramset`, and allow `--methods_include`/`--methods_exclude` to target `.`, using the new paramset helpers in `common/nextflow_helpers/helper.nf` (PR #23). - -* `methods/logistic_regression`: added an `--inverse_regularization_strength` argument with three `info.variants` as an example of a parameterised method (PR #23). - -## MAJOR CHANGES - -* Updated `api` files (PR #5). - -* Updated configs, components and CI to the latest Viash version (PR #8). - -* Updated to Viash 0.9.4 (PR #12). - -* Use dependencies in `openproblems-bio/openproblems` (PR #12). - -## MINOR CHANGES - -* Updated `README.md` (PR #5). - -* `run_benchmark`: write the commit the workflow ran from and the launch time into - `task_info.yaml`, instead of publishing `_viash.yaml` verbatim (PR #18). - -* Updated to Viash 0.9.7 (PR #19). - -* Updated the `common` submodule (PR #19). - -* Fixed broken documentation links in `README.md` and `CONTRIBUTING.md`, and added a quickstart to the README (PR #21). - -* `run_test_local.sh` now runs as-is on the template's test resources (PR #21). - -* Added a placeholder `thumbnail.svg` and scoped the common test resources to `cxg_mouse_pancreas_atlas` (PR #21). - -## BUGFIXES - -* `process_dataset`: also seed `np.random` when `--seed` is set (PR #22). - -* `accuracy`: write `metric_values` as a list to match `metric_ids` (PR #22). - +# task_spatial_proteomics_preprocessing + +## Unreleased + +Initial port of the spatial proteomics preprocessing benchmark onto the +OpenProblems task structure. Developed in +`juliusstein77/task_isp_preprocessing`; only the proteomics surface was +carried across. + +### Components + +* Five method namespaces, each with interchangeable alternatives sharing one + argument contract: `methods_image_correction`, `methods_proteomics_segmentation`, + `methods_protein_aggregation`, `methods_proteomics_qc`, + `methods_proteomics_normalization`. +* Six metric families in `metrics_proteomics`, wrapping + [`sprotmetrics`](https://github.com/theislab/sprotmetrics) `0.1.0`. + +### Conformance to the OpenProblems structure + +* `src/api/file_score.yaml` now requires `dataset_id`, `normalization_id` and + `method_id` in `uns`. These were commented out in the source repository, so + scores carried no provenance and could not be attributed to a run. +* Every metric component declares the `info.metrics` block that + `common/schemas/task_metric.yaml` requires - 66 metrics across six families, + generated from the `sprotmetrics` registry by + `scripts/create_component/generate_metric_info.py` rather than maintained by + hand. Metrics with no direction are deliberately omitted from that block: the + schema requires `maximize`, and has no way to express "descriptive", so + declaring them would hand the scoring stage a ranking criterion they cannot + support. They are still computed and still appear in the score file. +* The workflow moves from `src/workflows/run_benchmark_isp/main_isp.nf` to the + conventional `src/workflows/run_benchmark/main.nf`. + +### Known gaps + +* `run_benchmark/main.nf` is still the hand-written form inherited from the + source repository - one process per stage with a `case` block - rather than + the `runEach` / `fromState` idiom the template uses. Porting it is what will + supply the `method_id` and `dataset_id` that `file_score.yaml` now requires. +* Test resources are not yet on S3; `_viash.yaml` points at the intended + location. `scripts/create_resources/isp/` builds the input locally from the + public MCMICRO download in the meantime. +* There is no `workflows/process_datasets`. The template's version implements a + train/test/solution split for label projection, which is not the shape of this + task - the input is a SpatialData store, and preprocessing is what is being + benchmarked rather than prediction. It was removed rather than left dangling + against deleted API files. A real dataset processor belongs with the S3 + resources work above. diff --git a/DEVELOPER.md b/DEVELOPER.md new file mode 100644 index 0000000..2798949 --- /dev/null +++ b/DEVELOPER.md @@ -0,0 +1,242 @@ +# Developer guide + +How to work on the spatial proteomics (ISP) preprocessing benchmark: the layout, +how to add a method, and how to run the tests. + +> This repository began as a fork of the OpenProblems iST preprocessing task. +> A good deal of the original imaging-based spatial *transcriptomics* code is +> still present under `src/` (`methods_transcript_assignment/`, +> `methods_segmentation/`, `methods_cell_type_annotation/`, +> `methods_expression_correction/`, `metrics/`, `src/workflows/run_benchmark/`). +> None of it is used by the ISP workflow. When in doubt, the ISP surface is +> exactly what `src/workflows/run_benchmark/config.vsh.yaml` lists under +> `dependencies`. + +## Tooling + +| Tool | Version | Purpose | +|---|---|---| +| [Viash](https://viash.io) | 0.9.4 (pinned in `_viash.yaml`) | Turns `config.vsh.yaml` + `script.py` into a standalone executable | +| Nextflow | tested with 24.04.3 | Runs the workflow | +| Docker | any recent | Default execution engine for components | + +Every component also declares a `native` engine, so the suite can be built and +tested against a local Python environment when Docker is unavailable. See +[Running the tests](#running-the-tests). + +## Layout + +```text +src/ + api/ Shared component contracts (arguments + test wiring) + methods_image_correction/ + methods_proteomics_segmentation/ + methods_protein_aggregation/ + methods_proteomics_qc/ + methods_proteomics_normalization/ + metrics_proteomics/ + testing/ Synthetic fixtures and the shared test scripts + workflows/run_benchmark_isp/ + main.nf, config.vsh.yaml, test.sh +scripts/thesis_benchmark/ Drivers for the 15 published runs, plus figure/table generation +docs/metrics_catalogue.csv What every metric measures, its direction, and its origin +``` + +A component is a directory with two files: + +- `config.vsh.yaml` — merges an API contract via `__merge__`, then adds the + component's name, description, and engine dependencies. +- `script.py` — reads a `par` dict that Viash injects, writes the declared + outputs. + +The API contract in `src/api/comp_*.yaml` owns the argument list, so every +method in a namespace accepts the same flags. That is what lets the workflow +swap one method for another without changing its call. + +## The ISP pipeline + +```text +SpatialData .zarr + -> image correction corrected_image.npy (C x Y x X float32) + corrected_markers.json + -> segmentation segmentation_labels.npy (Y x X int32) + segmentation_metadata.json + -> aggregation cell_by_protein.h5ad + -> QC cell_by_protein_qc.h5ad + -> normalization cell_by_protein_norm.h5ad + -> metrics metrics_*.h5ad +``` + +Two contracts are worth knowing before you change anything: + +- **Segmentation labels are a gapless `1..n` integer range**, so + `max(labels) == n_objects`. Image-derived methods must renumber after any + step that drops objects (`relabel_sequential`). Reference segmentation is the + deliberate exception: dataset-provided label ids are passed through untouched + so they stay comparable to the dataset's own cell ids. +- **QC annotates, it does not filter.** QC components add `obs['passed_QC']`, + `obs['qc_filter_reason']` and `uns['qc']`, and keep every cell. Normalization + runs on all of them, which is harmless because every normalization method is + per-cell: which cells are present cannot change the values of the cells that + remain. Metrics are the only population-level statistics in the pipeline, so + they are where QC has to be accounted for, and each metric component reports + its values over all cells and again over the QC-passing subset with a + `_qc_pass` suffix. A new metric component should do the same, or the QC module + will look inert through it. + +## Adding a method + +Say you are adding a segmentation method. + +1. **Create the component.** + + ```bash + mkdir -p src/methods_proteomics_segmentation/my_method + ``` + + `config.vsh.yaml`: + + ```yaml + __merge__: /src/api/comp_method_proteomics_segmentation.yaml + + name: my_method + label: "My method" + summary: "One line." + description: | + What it does and what it is a baseline for. + + resources: + - type: python_script + path: script.py + + engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - numpy + - scikit-image + - type: native + + runners: + - type: executable + ``` + + Keep the `native` engine: without it the component is invisible to + `viash ns test --engine native` and silently drops out of the test suite. + +2. **Write `script.py`.** Read from `par`, write the declared outputs, and end + with the `## VIASH START` / `## VIASH END` block giving defaults for direct + execution. Prefer raising on an unrecognised option over falling back to a + default — a silently ignored argument is reported as a successful run. + +3. **Add any new arguments to the API contract**, not just to your component. + The workflow calls every method in a namespace with the same flags, so an + argument that only one component declares will be dropped for the others. + Viash 0.9 does *not* reject unknown arguments; it warns and treats them as + positional, so this failure mode is silent. + +4. **Register it with the workflow** in three places: + - `src/workflows/run_benchmark/config.vsh.yaml` under `dependencies` + - `main.nf`: an `*_EXE` constant and a branch in the stage's `case` + - `src/workflows/run_benchmark/test.sh`: the build list + +5. **Run the tests** (below). The shared test script for the namespace picks up + the new component automatically; extend it if the method needs a new + assertion. + +## Running the tests + +Each API contract wires in a shared test script from `src/testing/`, so every +component in a namespace is covered by one file: + +| Namespace | Test | +|---|---| +| `methods_image_correction` | `test_image_correction.py` | +| `methods_proteomics_segmentation` | `test_segmentation.py` | +| `methods_protein_aggregation` (arrays) | `test_aggregation.py` | +| `methods_protein_aggregation` (zarr) | `test_aggregation_zarr.py` | +| `methods_proteomics_qc` | `test_qc.py` | +| `methods_proteomics_normalization` | `test_normalization.py` | +| `metrics_proteomics` (matrix-based) | `test_metrics.py` | +| `metrics_proteomics/signal_background` (pixel-based) | `test_signal_background.py` | + +The tests build their own inputs with `src/testing/isp_fixtures.py` — a 64x64, +3-channel synthetic image, a matching label mask, and small AnnData objects. +Nothing is downloaded, so the whole suite runs in well under a minute and needs +no access to the MCMICRO dataset. + +With Docker: + +```bash +viash ns test --query "methods_proteomics_segmentation" +``` + +Without Docker, against a local environment that provides the packages the +components declare (`numpy`, `scipy`, `scikit-image`, `pandas`, `anndata`, +`spatialdata`, `zarr`): + +```bash +viash ns test --engine native --query "methods_proteomics_segmentation" +``` + +Everything at once: + +```bash +for ns in methods_image_correction methods_proteomics_segmentation \ + methods_protein_aggregation methods_proteomics_qc \ + methods_proteomics_normalization metrics_proteomics; do + viash ns test --query "$ns" +done +``` + +Note that `viash ns test` exits 0 even when individual tests fail, so read the +`result` column rather than the exit status. A namespace that reports no rows +at all usually means no component in it declares the engine you asked for. + +## Running the workflow + +```bash +ZARR_PATH=/path/to/exemplar-001.zarr \ +bash src/workflows/run_benchmark/test.sh +``` + +`test.sh` rebuilds every component and then runs `main.nf`. It must be +launched from the repository root: `main.nf` resolves component executables +relative to `workflow.launchDir`. + +`VIASH_ENGINE` (default `docker`) selects the engine both for the build and for +the run; the two must agree, which is why one variable drives both. Set it to +`native` to run without Docker: + +```bash +VIASH_ENGINE=native ZARR_PATH=/path/to/exemplar-001.zarr \ +bash src/workflows/run_benchmark/test.sh +``` + +The dataset itself is not in the repository and the public download is not in +the form the workflow wants; `scripts/create_resources/isp/prepare_exemplar_001.sh` +builds it. See **Data availability** in [README.md](README.md). + +Method choice and parameters come from environment variables — see the table in +[README.md](README.md). To vary one module at a time across the published +comparison set, use `scripts/thesis_benchmark/run_15_benchmark_runs.sh`. + +## Known rough edges + +Worth knowing before you plan larger changes: + +- `main.nf` is hand-written Nextflow with a `case` block per stage, rather + than the OpenProblems `runEach` / `fromState` idiom used by the legacy iST + workflow in `src/workflows/run_benchmark/main.nf`. One run covers exactly one + method combination; sweeps are done by an outer bash loop. +- There is no scoring stage in the workflow. Result tables are assembled + afterwards by `scripts/thesis_benchmark/collect_benchmark_results.py`. +- Image components load the whole image with `np.asarray(...)`, with no dask or + multiscale handling, and infer channel order from a shape heuristic. The + corrected image is written as a dense float32 `.npy`, which for exemplar-001 + is 371 MB per run. +- Several components are superseded but retained: `normalize_total_intensity`, + `basic_protein_aggregation`, `basic_protein_aggregation_arrays`. + `metrics_proteomics/cluster_separability` and the two + `methods_proteomics_cell_type_annotation` components are implemented but not + wired into the workflow. diff --git a/README.md b/README.md index 20b9134..b51bd29 100644 --- a/README.md +++ b/README.md @@ -1,53 +1,116 @@ -# Task Template +# Spatial Proteomics Preprocessing + +Benchmarking preprocessing workflows that turn multiplexed protein images into +cell-by-protein matrices. + +> [!NOTE] +> This README is normally generated by `scripts/create_readme.sh`, which renders +> the API in `src/api/` through a container. It is written by hand here because +> that generator has not yet been run against this repository; running it will +> overwrite this file with the canonical rendering. + +## Description + +Imaging-based spatial proteomics platforms — CyCIF, CODEX, MIBI, IMC — produce +multiplexed images in which each channel is an antibody-stained protein marker. +Turning those images into cell-level protein profiles takes a chain of +preprocessing choices: correcting the image, segmenting cells, aggregating pixel +intensities per cell, applying quality control, and normalizing. Each choice +changes the cell-by-protein matrix that every downstream analysis is built on, +there is no established consensus on which to make, and the effect of any single +one is rarely reported in isolation. + +This task benchmarks those steps as interchangeable modules: + +```text +SpatialData .zarr + -> image correction corrected_image.npy (C x Y x X float32) + corrected_markers.json + -> segmentation segmentation_labels.npy (Y x X int32) + segmentation_metadata.json + -> aggregation cell_by_protein.h5ad + -> QC cell_by_protein_qc.h5ad + -> normalization cell_by_protein_norm.h5ad + -> metrics metrics_*.h5ad +``` + +Methods within a module share an argument contract, so a run can swap one module +while holding the rest fixed. -This repo is a template to create a new task for the OpenProblems v2. This repo contains several example files and components that can be used when updated with the task info. +The task stops at preprocessing. Cell-type annotation and biological +interpretation are treated as downstream consumers of the output rather than as +benchmarked steps. -> [!WARNING] -> This README will be overwritten when performing the `create_task_readme` script. +## Metrics -## Create a repository from this template +Metric implementations live in [`sprotmetrics`](https://github.com/theislab/sprotmetrics); +the components in `src/metrics_proteomics/` are the Viash contracts around them. +`scripts/create_component/generate_metric_info.py` regenerates the `info.metrics` +blocks from that package's registry, so the direction and bounds declared here +cannot drift from the implementations. -> [!IMPORTANT] -> Before creating a new repository, make sure you are part of the OpenProblems task team. This will be done when you create an issue for the task and you get the go ahead to create the task. -> For more information on how to create a new task, check out the [Create a new task](https://openproblems.bio/documentation/create_task/getting-started/) documentation. +The metrics are deliberately **reference-free** wherever possible — they score a +result against properties the data must satisfy if preprocessing is correct, +rather than against a ground truth: -The instructions below will guide you through creating a new repository from this template ([creating-a-repository-from-a-template](https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template#creating-a-repository-from-a-template)). +| family | what it scores | +|---|---| +| `basic` | cell counts, morphology, intensity, marker coverage | +| `marker_distribution` | per-marker sparsity, coverage, dynamic range | +| `normalization_diagnostics` | raw-versus-normalized totals, variance, rank preservation | +| `segmentation_quality` | nuclear positivity, agreement between repeated nuclear stains, mutually exclusive co-expression, neighbour spillover | +| `technical_covariates` | residual dependence on cell area; variance stabilisation | +| `signal_background` | pixel-level signal inside masks against a locally proximal background | +This matters because ground-truth segmentation masks are unavailable for most +real datasets, including the public exemplar this task is developed against. -* Click the "Use this template" button on the top right of the repository. -* Use the Owner dropdown menu to select the `openproblems-bio` account. -* Type a name for your repository (task_...), and a description. -* Set the repository visibility to public. -* Click "Create repository from template". +Every metric is additionally reported over the QC-passing subset, suffixed +`_qc_pass`. QC components annotate rather than filter, and every normalization +method is per-cell, so without that second pass the choice of QC method would +leave every other metric untouched. -## Clone the repository +## Input data -To clone the repository with the submodule files, you can use the following command: +The workflow takes a **SpatialData Zarr store**: at least one `Images` element +with dims `(c, y, x)` and a channel coordinate carrying marker names, and +optionally a `Labels` element used by the `reference` segmentation method. + +The public MCMICRO `exemplar-001` download is raw acquisition data — unstitched +tiles for three imaging cycles — so two steps bridge the gap, both scripted: ```bash -git clone --recursive git@github.com:openproblems-bio/.git +bash scripts/create_resources/isp/prepare_exemplar_001.sh resources/isp ``` ->[!NOTE] -> If somehow there are no files visible in the submodule after cloning using the above command. Check the instructions [here](common/README.md). - -## What to do next -Check out the [task instructions](https://github.com/openproblems-bio/common_resources/blob/main/TASK_INSTRUCTIONS.md) for more information on how to update the example files and components. These instructions also contain information on how to build out the task and basic commands. +This downloads the dataset, registers the cycles into one mosaic with `ashlar`, +and converts the result to `exemplar-001.zarr`. Pass that path as `ZARR_PATH`. -To verify that everything works as-is before making changes, run: +## Running the benchmark ```bash -# download the test resources -scripts/sync_resources.sh +ZARR_PATH=/path/to/exemplar-001.zarr \ +NUCLEAR_MARKERS="DNA_6,DNA_7,DNA_8" \ +EXCLUSIVE_PAIRS="ECAD:CD45,ECAD:CD11B,SMA:CD45" \ +bash src/workflows/run_benchmark/test.sh +``` -# build the components and docker images -viash ns build --parallel --setup cachedbuild +Panel-dependent metrics return `NaN` when their configuration is absent, rather +than guessing for an unknown panel. See [DEVELOPER.md](DEVELOPER.md) for the +component layout, how to add a method, and how to run the tests. -# run the unit tests -viash ns test --parallel +## Authors & contributors -# run the benchmark on the test resources -scripts/run_benchmark/run_test_local.sh -``` +| name | roles | +|---|---| +| Julius Stein | author | +| Sergio Marco Salas | author, maintainer | + +## Component types -For more information on the OpenProblems v2, check out the [documentation](https://openproblems.bio/documentation/). \ No newline at end of file +| namespace | purpose | +|---|---| +| `methods_image_correction` | background and illumination correction | +| `methods_proteomics_segmentation` | cell label masks | +| `methods_protein_aggregation` | pixel intensities to a cell-by-protein matrix | +| `methods_proteomics_qc` | per-cell QC annotation | +| `methods_proteomics_normalization` | intensity normalization | +| `metrics_proteomics` | preprocessing quality metrics | diff --git a/_viash.yaml b/_viash.yaml index b3a3a42..7cfe9df 100644 --- a/_viash.yaml +++ b/_viash.yaml @@ -1,83 +1,73 @@ viash_version: 0.9.7 -# Step 1: Change the name of the task. -# example: task_name_of_this_task -name: task_template +name: task_spatial_proteomics_preprocessing organization: openproblems-bio version: dev license: MIT -# Step 2: Add keywords to describe the task. -keywords: [single-cell, openproblems, benchmark] -# Step 3: Update the `task_template` to the name of the task from step 1. +keywords: [single-cell, openproblems, benchmark, spatial proteomics, multiplexed imaging, preprocessing] links: - issue_tracker: https://github.com/openproblems-bio/task_template/issues - repository: https://github.com/openproblems-bio/task_template + issue_tracker: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing/issues + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing docker_registry: ghcr.io - -# Step 4: Update the label, summary and description. -# A unique, human-readable, short label. Used for creating summary tables and visualisations. -label: Template -summary: A one sentence summary of purpose and methodology. Used for creating an overview tables. +label: Spatial Proteomics Preprocessing +summary: Benchmarking preprocessing workflows that turn multiplexed protein images into cell-by-protein matrices. description: | - Provide a clear and concise description of your task, detailing the specific problem it aims - to solve. Outline the input data types, the expected output, and any assumptions or constraints. - Be sure to explain any terminology or concepts that are essential for understanding the task. + Imaging-based spatial proteomics platforms - CyCIF, CODEX, MIBI, IMC - produce + multiplexed images in which each channel is an antibody-stained protein marker. + Turning those images into cell-level protein profiles takes a chain of + preprocessing choices: correcting the image, segmenting cells, aggregating + pixel intensities per cell, applying quality control, and normalizing. Each + choice changes the cell-by-protein matrix that every downstream analysis is + built on, there is no established consensus on which to make, and the effect + of any single one is rarely reported in isolation. + + This task benchmarks those steps as interchangeable modules. The input is a + SpatialData Zarr store holding a multichannel image and, optionally, reference + segmentation labels. The output is a standardized cell-by-protein AnnData + object together with preprocessing quality metrics. Methods within a module + share an argument contract, so a run can swap one module while holding the + rest fixed. + + The metrics are deliberately reference-free wherever possible: they score a + result against properties the data must satisfy if preprocessing is correct - + every segmented object should contain nuclear signal, nuclear stains repeated + across imaging cycles are the same physical nuclei and must agree within a + cell, markers confined to incompatible lineages should rarely be co-positive, + and expression should not be explained by how large a cell happened to be. + This matters because ground-truth segmentation masks are unavailable for most + real datasets, including the public exemplar this task is developed against. + + The task stops at preprocessing. Cell-type annotation and biological + interpretation are treated as downstream consumers of the output rather than + as benchmarked steps. - Explain the motivation behind your proposed task. Describe the biological or computational - problem you aim to address and why it's important. Discuss the current state of research in - this area and any gaps or challenges that your task could help address. This section - should convince readers of the significance and relevance of your task. + The metric implementations live in the `sprotmetrics` package + (https://github.com/theislab/sprotmetrics); the components here are the Viash + contracts around them. -# A list of references to relevant literature. Each reference should be a DOI or a bibtex entry references: doi: - - 10.21203/rs.3.rs-4181617/v1 - # bibtex: - # - | - # @article{doe_2021_template, - # doi = {10.21203/rs.3.rs-4181617/v1}, - # url = {https://doi.org/10.21203/rs.3.rs-4181617/v1}, - # author = {Doe, John}, - # title = {A template for creating new tasks}, - # publisher = {Research Square}, - # year = {2021}, - # } - + # MCMICRO, the source of the exemplar-001 dataset this task is developed against. + - 10.1038/s41592-021-01308-y + info: - # Step 5: Replace the thumbnail with an image of your task. Should be an - # *.svg file renderable to 450x150 px. image: thumbnail.svg - # Step 6: Replace the task_template to the name of the task, and the common - # dataset to the dataset(s) your task uses. test_resources: - type: s3 - path: s3://openproblems-data/resources_test/common/cxg_mouse_pancreas_atlas/ - dest: resources_test/common/cxg_mouse_pancreas_atlas - - type: s3 - path: s3://openproblems-data/resources_test/task_template/ - dest: resources_test/task_template + path: s3://openproblems-data/resources_test/task_spatial_proteomics_preprocessing/ + dest: resources_test/task_spatial_proteomics_preprocessing -# Step 7: Update the authors of the task. -authors: - # Full name of the author, usually in the name of FirstName MiddleName LastName. - - name: John Doe - # Role of the author. Possible values: - # - # * `"author"`: Authors who have made substantial contributions to the component. - # * `"maintainer"`: The maintainer of the component. - # * `"contributor"`: Authors who have made smaller contributions (such as code patches etc.). +authors: + - name: Julius Stein + roles: [ "author" ] + info: + github: juliusstein77 + - name: Sergio Marco Salas roles: [ "author", "maintainer" ] - # Additional information on the author - info: - github: johndoe - orcid: 0000-0000-0000-0000 - email: john@doe.me - twitter: johndoe - linkedin: johndoe - -# Step 8: Remove all of the comments of the steps you completed + info: + github: sergiomarco25 config_mods: | .runners[.type == "nextflow"].config.labels := { lowmem : "memory = 20.Gb", midmem : "memory = 50.Gb", highmem : "memory = 100.Gb", lowcpu : "cpus = 5", midcpu : "cpus = 15", highcpu : "cpus = 30", lowtime : "time = 1.h", midtime : "time = 4.h", hightime : "time = 8.h", veryhightime : "time = 24.h" } @@ -86,6 +76,4 @@ repositories: - name: openproblems type: github repo: openproblems-bio/openproblems - tag: build/main # Step 9: Set this to the latest release - -# Step 10: High five yourself! + tag: build/main diff --git a/docs/make_metrics_xlsx.py b/docs/make_metrics_xlsx.py new file mode 100644 index 0000000..91a5994 --- /dev/null +++ b/docs/make_metrics_xlsx.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""Render docs/metrics_catalogue.csv as a colour-coded Excel workbook. + +The CSV is the source of truth; this only styles it. Run from the repo root: + + python docs/make_metrics_xlsx.py [OUTPUT.xlsx] +""" + +from __future__ import annotations + +import csv +import sys +from collections import Counter, defaultdict +from pathlib import Path + +from openpyxl import Workbook +from openpyxl.styles import Alignment, Border, Font, PatternFill, Side +from openpyxl.utils import get_column_letter + +CSV_PATH = Path("docs/metrics_catalogue.csv") + +# Status drives the row's identity colour: what exists, what is new, what is +# still open, what was rejected. +STATUS_STYLE = { + "IMPLEMENTED NOW": ("C8E6C9", "1B5E20", True), + "pre-existing": ("DCE9F7", "13395E", False), + "pre-existing (structurally dead)": ("E2E2E2", "5A5A5A", False), + "pre-existing (not wired into the workflow)": ("FFE7BA", "7A4E00", False), + "CANDIDATE (not implemented)": ("E6DAF2", "4A2C6B", False), + "BLOCKED": ("FFD5C2", "8A3A12", False), + "NOT ADAPTING": ("F7D7DA", "8A1C24", False), +} + +# Order matters twice over. The explicit non-directions come first, because +# their parentheticals can mention "high" or "low" in passing - "none +# (descriptive: high suits a lineage marker...)" is not a high-is-better metric. +# Then "weakly high" before "high", or it takes the firmer colour. +DIRECTION_STYLE = [ + ("none", "EDEDED", "6B6B6B"), + ("n/a", "EDEDED", "6B6B6B"), + ("constant", "EDEDED", "6B6B6B"), + ("ambiguous", "FFF0D0", "7A5A00"), + ("weakly high", "EAF6EB", "3E7B44"), + ("weakly low", "EAF1FB", "3A5F80"), + ("high", "D6EFD8", "1B5E20"), + ("low", "D9E7FA", "13395E"), +] + +# Values in the "separates" column that mean "no signal / not measured". +NO_SIGNAL = {"none", "n/a", "unknown", "not run", "cannot run", ""} + +HEADER_FILL = PatternFill("solid", fgColor="21354B") +HEADER_FONT = Font(bold=True, color="FFFFFF", size=11) +THIN = Side(style="thin", color="C9CDD2") +BORDER = Border(bottom=THIN, right=THIN) + +WIDTHS = { + "metric_id": 46, "component": 24, "task_axis": 26, "what_it_measures": 58, + "range": 14, "desired_direction": 30, "status": 34, "origin": 46, + "inputs_needed": 18, "prior_knowledge_needed": 30, + "separates_methods_on_exemplar_001": 24, "notes": 88, +} +WRAP = {"what_it_measures", "notes", "origin", "desired_direction", + "prior_knowledge_needed", "status", "task_axis"} + + +def direction_style(value: str): + """Fill and font for a direction, matching the most specific label first.""" + v = value.lower().strip() + for key, fill, font in DIRECTION_STYLE: + if key in v: + return fill, font + return None, None + + +def main() -> None: + out_path = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("metrics_catalogue.xlsx") + rows = list(csv.DictReader(CSV_PATH.open())) + cols = list(rows[0].keys()) + + wb = Workbook() + ws = wb.active + ws.title = "Metrics" + + ws.append([c.replace("_", " ") for c in cols]) + for cell in ws[1]: + cell.fill = HEADER_FILL + cell.font = HEADER_FONT + cell.alignment = Alignment(vertical="center", wrap_text=True) + ws.row_dimensions[1].height = 34 + + for r in rows: + ws.append([r[c] for c in cols]) + i = ws.max_row + fill, font_colour, bold = STATUS_STYLE.get(r["status"], ("FFFFFF", "000000", False)) + + for j, col in enumerate(cols, start=1): + cell = ws.cell(row=i, column=j) + cell.border = BORDER + cell.alignment = Alignment( + vertical="top", wrap_text=col in WRAP, + horizontal="center" if col in {"range", "inputs_needed"} else "left", + ) + # Row-wide tint from the status, so a glance groups the table. + cell.fill = PatternFill("solid", fgColor=fill) + cell.font = Font(color=font_colour, size=10, + bold=bold and col == "metric_id") + + # Columns that carry their own signal get a stronger, local colour. + ws.cell(row=i, column=cols.index("metric_id") + 1).font = Font( + color=font_colour, size=10, bold=True + ) + ws.cell(row=i, column=cols.index("status") + 1).font = Font( + color=font_colour, size=10, bold=True + ) + + d_cell = ws.cell(row=i, column=cols.index("desired_direction") + 1) + d_fill, d_font = direction_style(r["desired_direction"]) + if d_fill: + d_cell.fill = PatternFill("solid", fgColor=d_fill) + muted = d_font in {"6B6B6B", "7A5A00"} + d_cell.font = Font(color=d_font, size=10, bold=not muted, italic=muted) + + s_cell = ws.cell(row=i, column=cols.index("separates_methods_on_exemplar_001") + 1) + sep = r["separates_methods_on_exemplar_001"].strip().lower() + if sep in NO_SIGNAL: + s_cell.fill = PatternFill("solid", fgColor="EDEDED") + s_cell.font = Font(color="6B6B6B", size=10, italic=True) + else: + s_cell.fill = PatternFill("solid", fgColor="D6EFD8") + s_cell.font = Font(color="1B5E20", size=10, bold=True) + + for j, col in enumerate(cols, start=1): + ws.column_dimensions[get_column_letter(j)].width = WIDTHS.get(col, 20) + ws.freeze_panes = "B2" + ws.auto_filter.ref = f"A1:{get_column_letter(len(cols))}{ws.max_row}" + + _summary_sheet(wb, rows) + wb.save(out_path) + print(f"wrote {out_path} ({len(rows)} metrics)") + + +def _summary_sheet(wb: Workbook, rows: list[dict]) -> None: + ws = wb.create_sheet("Summary") + + def header(text, row, span=3): + c = ws.cell(row=row, column=1, value=text) + c.font = Font(bold=True, color="FFFFFF", size=11) + c.fill = HEADER_FILL + for j in range(2, span + 1): + ws.cell(row=row, column=j).fill = HEADER_FILL + + ws.column_dimensions["A"].width = 46 + ws.column_dimensions["B"].width = 12 + ws.column_dimensions["C"].width = 74 + + header("Status", 1) + ws.cell(row=1, column=2, value="Count").font = Font(bold=True, color="FFFFFF") + ws.cell(row=1, column=3, value="Meaning").font = Font(bold=True, color="FFFFFF") + + meaning = { + "IMPLEMENTED NOW": "Added in this change; adapted from the sprot prototype or written from the catalogue", + "pre-existing": "Already in the benchmark before this work", + "pre-existing (structurally dead)": "Cannot vary given the methods implemented, so carries no signal", + "pre-existing (not wired into the workflow)": "Component exists but the workflow never calls it", + "CANDIDATE (not implemented)": "Worth adding; not done yet", + "BLOCKED": "Cannot be implemented until missing input is available", + "NOT ADAPTING": "Deliberately rejected; see notes for the reason", + } + r = 2 + for status, n in sorted(Counter(x["status"] for x in rows).items(), key=lambda kv: -kv[1]): + fill, font, _ = STATUS_STYLE.get(status, ("FFFFFF", "000000", False)) + for j, v in enumerate([status, n, meaning.get(status, "")], start=1): + c = ws.cell(row=r, column=j, value=v) + c.fill = PatternFill("solid", fgColor=fill) + c.font = Font(color=font, size=10, bold=j == 1) + c.alignment = Alignment(vertical="top", wrap_text=j == 3) + c.border = BORDER + r += 1 + + r += 1 + header("Directional metrics per pipeline axis", r) + ws.cell(row=r, column=2, value="Before").font = Font(bold=True, color="FFFFFF") + ws.cell(row=r, column=3, value="Now").font = Font(bold=True, color="FFFFFF") + r += 1 + + # A metric only helps rank a method if it has a direction and is live. + def directional(x): + d = x["desired_direction"].lower() + return d.startswith(("high", "low")) and "dead" not in x["status"] + + before, now = defaultdict(int), defaultdict(int) + for x in rows: + if not directional(x): + continue + for axis in [a.strip() for a in x["task_axis"].split(",")]: + if x["status"].startswith("pre-existing"): + before[axis] += 1 + now[axis] += 1 + elif x["status"] == "IMPLEMENTED NOW": + now[axis] += 1 + + for axis in ["segmentation", "image correction", "aggregation", "normalization", "QC"]: + b, n = before.get(axis, 0), now.get(axis, 0) + ws.cell(row=r, column=1, value=axis).font = Font(size=10, bold=True) + for j, v in ((2, b), (3, n)): + c = ws.cell(row=r, column=j, value=v) + c.alignment = Alignment(horizontal="center") + # Red where an axis had nothing to rank on, green where it now does. + c.fill = PatternFill("solid", fgColor="F7D7DA" if v == 0 else "C8E6C9") + c.font = Font(color="8A1C24" if v == 0 else "1B5E20", size=10, bold=True) + for j in range(1, 4): + ws.cell(row=r, column=j).border = BORDER + r += 1 + + r += 1 + note = ws.cell( + row=r, column=1, + value=("A metric counts as directional only if it has a known good direction " + "(high or low) and can actually vary. Descriptive metrics such as " + "median_area, and metrics that are structurally constant such as the " + "rank-correlation pair, are excluded."), + ) + note.font = Font(italic=True, size=9, color="5A5A5A") + note.alignment = Alignment(wrap_text=True, vertical="top") + ws.merge_cells(start_row=r, start_column=1, end_row=r + 2, end_column=3) + + +if __name__ == "__main__": + main() diff --git a/docs/metrics_catalogue.csv b/docs/metrics_catalogue.csv new file mode 100644 index 0000000..29a7aef --- /dev/null +++ b/docs/metrics_catalogue.csv @@ -0,0 +1,58 @@ +metric_id,component,task_axis,what_it_measures,range,desired_direction,status,origin,inputs_needed,prior_knowledge_needed,separates_methods_on_exemplar_001,notes +n_cells,basic,segmentation,Number of segmented objects,0..inf,none (more is not better; over-segmentation inflates it),pre-existing,this repo,h5ad,,segmentation,Also listed in Metrics-2.xlsx quality_metrics as 'Number of total cells' +n_markers,basic,dataset statistics,Panel size,0..inf,none (constant),pre-existing,this repo,h5ad,,none,Invariant by construction; carries no benchmark signal +qc_pass_rate,basic,QC,Fraction of cells passing QC,0-1,none (no_qc trivially scores 1.0),pre-existing,this repo,h5ad,,QC, +median_area,basic,segmentation,Median object area,0..inf,none (no biological target encoded),pre-existing,this repo,h5ad,,segmentation, +mean_area,basic,segmentation,Mean object area,0..inf,none (no biological target encoded),pre-existing,this repo,h5ad,,segmentation, +median_total_intensity,basic,aggregation,Median per-cell summed intensity,0..inf,none,pre-existing,this repo,h5ad,,"segmentation, aggregation", +mean_total_intensity,basic,aggregation,Mean per-cell summed intensity,0..inf,none,pre-existing,this repo,h5ad,,"segmentation, aggregation", +median_marker_coverage,basic + marker_distribution,image correction,Median fraction of cells above the coverage threshold,0-1,weakly high,pre-existing,this repo,h5ad,,segmentation,DUPLICATE: emitted by two families from identical code; saturated near 0.995 +min_marker_coverage,basic,image correction,Worst marker's coverage,0-1,weakly high,pre-existing,this repo,h5ad,,segmentation,Saturated near 0.992 +median_marker_mean,marker_distribution,image correction,Median per-marker mean intensity,0..inf,none,pre-existing,this repo,h5ad,,segmentation, +median_marker_cv,marker_distribution,normalization,Median per-marker coefficient of variation,0..inf,ambiguous (biology or noise),pre-existing,this repo,h5ad,,segmentation,Prior sprot cv_per_protein is the same quantity +median_marker_zero_fraction,marker_distribution,image correction,Fraction of entries at or below the zero threshold,0-1,weakly low,pre-existing,this repo,h5ad,,segmentation,Prior sprot sparsity_per_protein is the same quantity +mean_marker_zero_fraction,marker_distribution,image correction,Fraction of entries at or below the zero threshold,0-1,weakly low,pre-existing,this repo,h5ad,,segmentation,Prior sprot sparsity_per_protein is the same quantity +mean_marker_coverage,marker_distribution,image correction,Mean per-marker detection rate,0-1,weakly high,pre-existing,this repo,h5ad,,segmentation, +median_marker_dynamic_range,marker_distribution,image correction,Median per-marker q99 minus q01,0..inf,weakly high,pre-existing,this repo,h5ad,,segmentation,More robust than the prior sprot max-minus-min version +raw_total_cv,normalization_diagnostics,normalization,CV of per-cell total intensity before normalization,0..inf,none (a property of the input),pre-existing,this repo,h5ad,,segmentation, +normalized_total_cv,normalization_diagnostics,normalization,CV of per-cell total intensity after normalization,0..inf,low,pre-existing,this repo,h5ad,,"segmentation, QC",One of only two directional metrics that existed before this work +total_cv_ratio_norm_over_raw,normalization_diagnostics,normalization,Compression of total-intensity spread achieved,0..inf,low,pre-existing,this repo,h5ad,,"segmentation, QC",The other pre-existing directional metric +median_raw_total_intensity,normalization_diagnostics,normalization,Median raw per-cell total,0..inf,none,pre-existing,this repo,h5ad,,segmentation,median_raw_total_intensity DUPLICATES basic::median_total_intensity exactly +median_normalized_total_intensity,normalization_diagnostics,normalization,Median normalized per-cell total,0..inf,none,pre-existing,this repo,h5ad,,segmentation, +median_marker_variance_ratio_norm_over_raw,normalization_diagnostics,normalization,Per-marker variance change under normalization,0..inf,none,pre-existing,this repo,h5ad,,segmentation, +median_cell_marker_rank_correlation,normalization_diagnostics,normalization,Preservation of within-cell marker ranks,-1..1,high,pre-existing (structurally dead),this repo,h5ad,,none,"Exactly 1.0 in every run: all implemented normalizations are monotonic per cell, so ranks cannot change" +mean_cell_marker_rank_correlation,normalization_diagnostics,normalization,Preservation of within-cell marker ranks,-1..1,high,pre-existing (structurally dead),this repo,h5ad,,none,"Exactly 1.0 in every run: all implemented normalizations are monotonic per cell, so ranks cannot change" +silhouette_cell_type,cluster_separability,normalization,Cluster separability of annotated cell types,-1..1,high,pre-existing (not wired into the workflow),this repo,h5ad,obs['cell_type'],not run,Nothing in the pipeline produces cell_type; needs the annotation components wired first +dapi_positive_ratio,segmentation_quality,segmentation,Fraction of objects containing nuclear signal (Otsu on the nuclear channels),0-1,high,IMPLEMENTED NOW,Metrics-2.xlsx row 8 (no prior code),h5ad,nuclear marker names (auto-detected from DNA/DAPI/Hoechst),"segmentation, aggregation",threshold 0.535 vs watershed 0.441. Every segmented cell should contain a nucleus +replicate_correlation,segmentation_quality,"segmentation, normalization",Agreement between nuclear stains repeated across imaging cycles,-1..1,high,IMPLEMENTED NOW,sprot protein_contribution.equivalent_proteins_correlation (adapted),h5ad,which markers are replicates,"segmentation, aggregation",threshold 0.837 vs watershed 0.889. CAUTION: sum aggregation inflates it to 0.999 because every channel then scales with area +median_replicate_cv,segmentation_quality,segmentation,Per-cell spread across the repeated nuclear stains,0..inf,low,IMPLEMENTED NOW,new (per-cell variant of the above),h5ad,which markers are replicates,segmentation, +mutually_exclusive_coexpression_ratio,segmentation_quality,"segmentation, image correction","Cells co-positive for incompatible lineage markers, over cells positive for either",0-1,low,IMPLEMENTED NOW,Metrics-2.xlsx row 6 (no prior code),h5ad,mutually exclusive marker pairs (NaN when unset),"segmentation, aggregation","threshold 0.685 vs watershed 0.709. Both high on exemplar-001, worth investigating" +spillover_ratio,segmentation_quality,segmentation,Intensity in negative cells adjacent to a positive cell vs non-adjacent negative cells,0..inf,low (1.0 means no spillover),IMPLEMENTED NOW,Metrics-2.xlsx row 7 (no prior code),h5ad (centroids),,"segmentation, aggregation","threshold 1.087 vs watershed 1.213, consistent with watershed over-expanding masks. Needs >~200 cells to be defined" +variance_explained_by_covariate_counts,technical_covariates,aggregation,"Mean R2 of expression on cell area, counts layer",0-1,low,IMPLEMENTED NOW,sprot technical_correction.variance_explained_by_covariates (adapted to cell area),h5ad,obs['area'],segmentation,Only the _counts variant can see the aggregation axis; per-cell normalization divides out the area scaling that sum aggregation creates +max_marker_variance_explained_by_covariate_counts,technical_covariates,aggregation,"Worst single marker's R2 on cell area, counts layer",0-1,low,IMPLEMENTED NOW,new (worst-case companion),h5ad,obs['area'],segmentation,Stops one area-driven channel being hidden by the panel average +covariate_pc_correlation_counts,technical_covariates,aggregation,"Mean |correlation| of leading PCs with cell area, counts layer",0-1,low,IMPLEMENTED NOW,sprot technical_correction.library_size_pc_correlation (adapted to cell area),h5ad,obs['area'],segmentation,Metrics-2.xlsx row 11. PCA by SVD to avoid a scikit-learn dependency +variance_explained_by_covariate_normalized,technical_covariates,normalization,"Mean R2 of expression on cell area, normalized layer",0-1,low,IMPLEMENTED NOW,sprot technical_correction.variance_explained_by_covariates (adapted to cell area),h5ad,obs['area'],segmentation,Only the _counts variant can see the aggregation axis; per-cell normalization divides out the area scaling that sum aggregation creates +max_marker_variance_explained_by_covariate_normalized,technical_covariates,normalization,"Worst single marker's R2 on cell area, normalized layer",0-1,low,IMPLEMENTED NOW,new (worst-case companion),h5ad,obs['area'],segmentation,Stops one area-driven channel being hidden by the panel average +covariate_pc_correlation_normalized,technical_covariates,normalization,"Mean |correlation| of leading PCs with cell area, normalized layer",0-1,low,IMPLEMENTED NOW,sprot technical_correction.library_size_pc_correlation (adapted to cell area),h5ad,obs['area'],segmentation,Metrics-2.xlsx row 11. PCA by SVD to avoid a scikit-learn dependency +mean_variance_correlation,technical_covariates,normalization,Dependence of per-marker variance on per-marker mean,-1..1,low,IMPLEMENTED NOW,sprot technical_correction.mean_variance_correlation (direct),h5ad,,segmentation,Metrics-2.xlsx row 3. Tests whether the transform stabilised variance +marker_variance_evenness,technical_covariates,normalization,Shannon entropy of per-marker variance shares,0-1,high,IMPLEMENTED NOW,sprot protein_contribution.gene_variance_evenness (direct),h5ad,,segmentation,Metrics-2.xlsx row 1. Low means a few bright markers dominate the variance +median_rel_sni,signal_background,"image correction, segmentation",Area between the sorted per-cell intensity curve and a rank-matched proximal-background curve,-inf..inf,high,IMPLEMENTED NOW,sprot protein_quality_metrics.metric_rel_sni (adapted),image + labels,,segmentation,threshold -0.029 vs watershed +0.023. Negative means cells are dimmer than the adjacent ring; worth investigating on this dataset +min_rel_sni,signal_background,image correction,Worst marker's signal integral,-inf..inf,high,IMPLEMENTED NOW,new (worst-case companion),image + labels,,segmentation, +median_intracell_coverage,signal_background,image correction,Fraction of own pixels above threshold in the brightest 5% of cells,0-1,high,IMPLEMENTED NOW,sprot protein_quality_metrics.metric_intracell_coverage (adapted),image + labels,,segmentation,Separates real staining (bright and extensive) from punctate artefact (bright and sparse). Prototype returned 1.0 identically because prop_positive short-circuits at thresh=0 +median_signal_to_background_ratio,signal_background,image correction,Mean intensity inside masks over mean in the surrounding ring,0..inf,high,IMPLEMENTED NOW,Metrics-2.xlsx row 4 (no prior code),image + labels,,segmentation,Only 1.02-1.04 on exemplar-001: a 3px ring is mostly cell-adjacent membrane. Tune --dilation_iterations +median_cellular_specificity,signal_background,image correction,Positive pixels inside masks as a share of positive pixels inside plus in the ring,0-1,high,IMPLEMENTED NOW,Metrics-2.xlsx row 14 (no prior code),image + labels,,segmentation, +median_marker_gini,signal_background,marker characterisation,"Gini coefficient of per-cell intensity, per marker",0-1,"none (descriptive: high suits a lineage marker, low a housekeeper)",IMPLEMENTED NOW,sprot protein_quality_metrics.metric_sparsity_gini (adapted),image + labels,,segmentation, +signal_drop_on_aggregation,(would extend signal_background),aggregation,Cells with positive pixels whose aggregated value falls to zero,0-1,low,CANDIDATE (not implemented),Metrics-2.xlsx row 13,image + labels,,unknown,"Directly targets aggregation, which still has the weakest metric coverage. Machinery now exists, so this is cheap" +nuclear_signal_containment,(would extend signal_background),segmentation,Fraction of total nuclear signal falling inside any mask,0-1,high,CANDIDATE (not implemented),proposed in the roadmap,image + labels,,unknown,The only way to catch cells missed entirely; every cell-level metric is blind to them because they are simply absent from the table +marker_bimodality,(new),"image correction, normalization",Between-class over within-class variance of the Otsu split per marker,0..inf,high,CANDIDATE (not implemented),proposed in the roadmap,h5ad,,unknown,Asks whether a marker can be called positive at all after preprocessing +tile_cv,(new),image correction,Variation of intensity across spatial tiles,0..inf,low,CANDIDATE (not implemented),Metrics-2.xlsx row 10,h5ad (centroids),,unknown,"Would test flat-field correction directly, which nothing currently does" +clustering_stability,(new),all axes,Stability of clustering under resampling,0-1,high,CANDIDATE (not implemented),Metrics-2.xlsx row 5,h5ad,,unknown,Needs repeated perturbed clustering; the most expensive candidate here +hpa_localisation_agreement,(new),marker characterisation,Match between observed subcellular pattern and Human Protein Atlas priors,0-1,high,CANDIDATE (not implemented),Metrics-2.xlsx quality_metrics row 7 (implemented by Bence at the scverse hackathon),image + labels,HPA localisation priors (external),unknown,Needs an external reference the repo does not carry; code lives outside these archives +"segmentation_vs_reference (dice, IoU, object F1, split/merge)",(would be a new metric component),segmentation,Direct agreement with reference masks,0-1,high,BLOCKED,compare_segmentation_to_gt.py in this repo,labels + reference labels,reference segmentation masks,cannot run,Exists as a standalone script outside the workflow. The public exemplar-001 download has no masks; they are MCMICRO segmentation output +snr_per_protein,-,normalization,Per-marker mean over standard deviation,0..inf,high,NOT ADAPTING,sprot protein_properties.snr_per_protein,h5ad,,n/a,"REDUNDANT: mean/std is exactly 1/CV, so it carries no information beyond median_marker_cv" +cv_per_protein,-,normalization,Per-marker coefficient of variation,0..inf,ambiguous,NOT ADAPTING,sprot protein_properties.cv_per_protein,h5ad,,n/a,ALREADY PRESENT as median_marker_cv +dynamic_range_per_protein,-,image correction,Per-marker max minus min,0..inf,high,NOT ADAPTING,sprot protein_properties.dynamic_range_per_protein,h5ad,,n/a,"ALREADY PRESENT, and ours uses q99-q01 which is robust to single outlier pixels" +sparsity_per_protein,-,image correction,Per-marker fraction of exact zeros,0-1,low,NOT ADAPTING,sprot protein_properties.sparsity_per_protein,h5ad,,n/a,"ALREADY PRESENT as median_marker_zero_fraction, which additionally takes a threshold" +gene_pair_cosine_similarity / kl_compare_adata,-,aggregation,Similarity between two aggregation methods' outputs,0-1 / 0..inf,"n/a (pairwise, not a score)",NOT ADAPTING,sprot aggregating/compute.py,two h5ad files,,n/a,"ARCHITECTURAL MISMATCH: compares two runs against each other, whereas a metric component scores one run. Would need a cross-run comparison stage the workflow does not have" +"variance_explained_by_covariates (total_counts, pct_mito, batch)",-,normalization,Variance explained by single-cell technical covariates,0-1,low,NOT ADAPTING,sprot technical_correction.variance_explained_by_covariates,h5ad,"obs total_counts, pct_mito, batch",n/a,"ADAPTED INSTEAD: those covariates do not exist in imaging data. Reimplemented against cell area, which is the analogous technical driver here" +new_segmentation.py,-,segmentation,-,,,NOT ADAPTING,sprot metrics/new_segmentation.py,,,,EMPTY FILE (0 bytes) in both archives; nothing to adapt diff --git a/scripts/create_component/generate_metric_info.py b/scripts/create_component/generate_metric_info.py new file mode 100644 index 0000000..08deee7 --- /dev/null +++ b/scripts/create_component/generate_metric_info.py @@ -0,0 +1,112 @@ +#!/usr/bin/env python3 +"""Write the ``info.metrics`` block into every metric component's config. + +OpenProblems requires each metric component to declare, per metric it reports: +``label, summary, description, references, links, min, max, maximize`` +(``common/schemas/task_metric.yaml``). Maintaining that by hand across ~70 +metric ids would guarantee drift from the implementations, so it is generated +from the registry in ``sprotmetrics``, which is where direction and bounds are +already recorded. + +Two deliberate behaviours: + +* Metrics with no direction are omitted. The schema has no way to express + "descriptive", and ``maximize`` is required, so including them would hand the + scoring stage a ranking criterion the metric cannot support. They are still + computed and still appear in the score file - they are just not OpenProblems + metrics. +* ``references`` and ``links`` are backfilled here. ``MetricInfo`` carries both + fields, but ``to_openproblems_info()`` in sprotmetrics 0.1.0 does not emit + them, and the schema requires both keys to be present. Where a metric records + no reference of its own, the task's own links are used, which is honest: the + metric originates with this benchmark rather than with a paper. + +Run from the repo root: + + python scripts/create_component/generate_metric_info.py [--check] + +``--check`` exits non-zero if any config is out of date, for CI. +""" + +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +import sprotmetrics as sm +import yaml + +REPO = "https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing" +TASK_LINKS = {"repository": REPO, "docs": f"{REPO}#readme"} + +# Component directory -> the sprotmetrics family it reports. +FAMILIES = { + "basic": "basic", + "marker_distribution": "marker_distribution", + "normalization_diagnostics": "normalization_diagnostics", + "segmentation_quality": "segmentation_quality", + "technical_covariates": "technical_covariates", + "signal_background": "signal_background", +} + + +def entries_for(family: str) -> list[dict]: + out = [] + for metric_id in sm.list_metrics(family=family, include_qc_variants=True).index: + info = sm.get_metric(metric_id) + if info.maximize is None: + continue # descriptive or ambiguous; see the module docstring + entry = dict(info.to_openproblems()) + entry["references"] = ( + {"doi": list(info.references)} if info.references else {} + ) + entry["links"] = dict(info.links) if info.links else dict(TASK_LINKS) + # Schema order, so a regenerated file diffs cleanly against the last one. + out.append({ + k: entry[k] for k in + ["name", "label", "summary", "description", "references", "links", + "min", "max", "maximize"] if k in entry + }) + return out + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("--check", action="store_true", + help="exit non-zero if any config would change") + args = ap.parse_args() + + stale = [] + for component, family in FAMILIES.items(): + path = Path("src/metrics_proteomics") / component / "config.vsh.yaml" + if not path.exists(): + print(f"!! missing {path}", file=sys.stderr) + return 1 + + config = yaml.safe_load(path.read_text()) + wanted = entries_for(family) + current = (config.get("info") or {}).get("metrics") + + if current == wanted: + print(f" {component}: up to date ({len(wanted)} metrics)") + continue + stale.append(component) + if args.check: + print(f"!! {component}: out of date", file=sys.stderr) + continue + + config.setdefault("info", {})["metrics"] = wanted + path.write_text(yaml.safe_dump(config, sort_keys=False, width=100, + default_flow_style=False, allow_unicode=True)) + print(f" {component}: wrote {len(wanted)} metrics") + + if args.check and stale: + print(f"\n{len(stale)} config(s) out of date; run this script without " + f"--check", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/create_resources/isp/ometiff_to_spatialdata.py b/scripts/create_resources/isp/ometiff_to_spatialdata.py new file mode 100755 index 0000000..1a4dcc4 --- /dev/null +++ b/scripts/create_resources/isp/ometiff_to_spatialdata.py @@ -0,0 +1,132 @@ +#!/usr/bin/env python3 +"""Convert a registered multiplexed OME-TIFF into a SpatialData Zarr store. + +The ISP benchmark workflow takes a SpatialData Zarr store as its only input, +but nothing in the repository produces one: the published run used a local +conversion of the MCMICRO exemplar-001 dataset that was never committed. This +script closes that gap for any MCMICRO-style dataset. + +Typical use, starting from the exemplar-001 download: + + # 1. register the raw cycles into one mosaic (what mcmicro's ashlar step does) + ashlar raw/*.ome.tiff --ffp illumination/*-ffp.tif --dfp illumination/*-dfp.tif \ + --align-channel 0 -o exemplar-001-registered.ome.tif + + # 2. convert to the SpatialData store the workflow expects + python ometiff_to_spatialdata.py \ + --image exemplar-001-registered.ome.tif \ + --markers exemplar-001/markers.csv \ + --output exemplar-001.zarr \ + --image-key exemplar-001_image + +Optionally attach a segmentation mask as a labels element, so that the +`reference` segmentation method has something to read: + + ... --labels segmentation/cell.tif --labels-key exemplar-001_cell +""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +import numpy as np +import pandas as pd +import tifffile +import spatialdata as sd +from spatialdata.models import Image2DModel, Labels2DModel + + +def read_markers(path: Path, n_channels: int) -> list[str]: + """Read marker names from an MCMICRO-style markers.csv.""" + df = pd.read_csv(path) + for col in ["marker_name", "Marker", "marker", "name"]: + if col in df.columns: + names = [str(x) for x in df[col].tolist()] + break + else: + raise ValueError( + f"No marker-name column in {path}. Columns present: {list(df.columns)}" + ) + + if len(names) != n_channels: + raise ValueError( + f"{path} lists {len(names)} markers but the image has {n_channels} channels." + ) + return names + + +def read_image_cyx(path: Path) -> np.ndarray: + """Read an OME-TIFF as a (channels, y, x) array.""" + with tifffile.TiffFile(path) as tif: + arr = tif.series[0].asarray() + + if arr.ndim == 2: + arr = arr[None, :, :] + if arr.ndim != 3: + raise ValueError(f"Expected a 2D or 3D image, got shape {arr.shape}.") + + # Registered mosaics are written channel-first, but be explicit about it + # rather than trusting axis order silently: the channel axis is the short one. + if arr.shape[0] > arr.shape[-1] and arr.shape[-1] <= 256: + arr = np.moveaxis(arr, -1, 0) + return arr + + +def read_labels_yx(path: Path) -> np.ndarray: + with tifffile.TiffFile(path) as tif: + arr = tif.series[0].asarray() + arr = np.squeeze(arr) + if arr.ndim != 2: + raise ValueError(f"Expected a 2D label mask, got shape {arr.shape}.") + if arr.min() < 0: + raise ValueError("Label masks must not contain negative values.") + return arr.astype(np.int32, copy=False) + + +def main() -> None: + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--image", required=True, type=Path, help="Registered multichannel OME-TIFF.") + ap.add_argument("--markers", required=True, type=Path, help="markers.csv listing one marker per channel.") + ap.add_argument("--output", required=True, type=Path, help="Destination .zarr store.") + ap.add_argument("--image-key", default="image", help="Key for the image element.") + ap.add_argument("--labels", type=Path, default=None, help="Optional segmentation mask TIFF.") + ap.add_argument("--labels-key", default="cell", help="Key for the labels element.") + args = ap.parse_args() + + print(f">> Reading {args.image}", flush=True) + image = read_image_cyx(args.image) + print(f" shape {image.shape}, dtype {image.dtype}", flush=True) + + markers = read_markers(args.markers, image.shape[0]) + print(f">> {len(markers)} markers: {', '.join(markers)}", flush=True) + + elements = { + "images": { + args.image_key: Image2DModel.parse(image, dims=("c", "y", "x"), c_coords=markers) + } + } + + if args.labels is not None: + print(f">> Reading labels from {args.labels}", flush=True) + labels = read_labels_yx(args.labels) + if labels.shape != image.shape[1:]: + raise ValueError( + f"Label shape {labels.shape} does not match image spatial shape {image.shape[1:]}." + ) + n_objects = int((np.unique(labels) > 0).sum()) + print(f" {n_objects} objects", flush=True) + elements["labels"] = {args.labels_key: Labels2DModel.parse(labels, dims=("y", "x"))} + + print(f">> Writing {args.output}", flush=True) + if args.output.exists(): + raise SystemExit(f"{args.output} already exists; remove it first.") + sdata = sd.SpatialData(**elements) + sdata.write(args.output) + + print(">> Done", flush=True) + print(sdata, flush=True) + + +if __name__ == "__main__": + main() diff --git a/scripts/create_resources/isp/prepare_exemplar_001.sh b/scripts/create_resources/isp/prepare_exemplar_001.sh new file mode 100755 index 0000000..e7e047f --- /dev/null +++ b/scripts/create_resources/isp/prepare_exemplar_001.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash + +# Turn the public MCMICRO exemplar-001 download into the SpatialData Zarr store +# that the ISP benchmark workflow expects. +# +# The workflow takes a .zarr as its only input, but the published exemplar-001 +# download is *raw* acquisition data: unstitched tiles for three imaging cycles, +# plus illumination profiles. Two steps bridge the gap: +# +# 1. registration - stitch and align the cycles into one 12-channel mosaic, +# with illumination correction. This is what mcmicro's registration step +# does, using the same tool (ashlar). +# 2. conversion - wrap the mosaic and its marker names in a SpatialData store. +# +# Usage: +# bash scripts/create_resources/isp/prepare_exemplar_001.sh [OUTPUT_DIR] +# +# The resulting store is what you pass as ZARR_PATH: +# ZARR_PATH=/exemplar-001.zarr \ +# bash src/workflows/run_benchmark/test.sh +# +# Requirements: curl, unzip, and a Python environment with ashlar, tifffile, +# spatialdata, numpy and pandas. ashlar reads OME-TIFF through BioFormats, so a +# real JDK must be on PATH (javac included, not just a JRE). +# +# Note on reference segmentation: the public download contains no segmentation +# masks, so the `reference` segmentation method has nothing to read. Those +# labels are mcmicro segmentation output. If you have them, attach them with +# --labels/--labels-key when running ometiff_to_spatialdata.py, and pass the key +# to the workflow as LABELS_KEY. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +OUTPUT_DIR="${1:-${OUTPUT_DIR:-resources/isp}}" +URL="https://mcmicro.s3.amazonaws.com/exemplars/exemplar-001.zip" + +mkdir -p "$OUTPUT_DIR" +cd "$OUTPUT_DIR" + +echo ">> Output directory: $(pwd)" + +if ! command -v javac >/dev/null 2>&1; then + echo "ERROR: no javac on PATH. ashlar reads OME-TIFF via BioFormats and needs a" >&2 + echo " full JDK, not just a JRE. Install one and set JAVA_HOME." >&2 + exit 1 +fi + +if [ ! -f exemplar-001.zip ]; then + echo ">> Downloading exemplar-001 (~240 MB)" + curl -fL --retry 3 --retry-delay 5 -o exemplar-001.zip.part "$URL" + mv exemplar-001.zip.part exemplar-001.zip +else + echo ">> exemplar-001.zip already present, skipping download" +fi + +if [ ! -d exemplar-001 ]; then + echo ">> Extracting" + unzip -q exemplar-001.zip +fi + +if [ ! -f exemplar-001-registered.ome.tif ]; then + echo ">> Registering the three cycles into one mosaic (this takes a few minutes)" + ( cd exemplar-001 && ashlar \ + raw/exemplar-001-cycle-06.ome.tiff \ + raw/exemplar-001-cycle-07.ome.tiff \ + raw/exemplar-001-cycle-08.ome.tiff \ + --ffp illumination/exemplar-001-cycle-06-ffp.tif \ + illumination/exemplar-001-cycle-07-ffp.tif \ + illumination/exemplar-001-cycle-08-ffp.tif \ + --dfp illumination/exemplar-001-cycle-06-dfp.tif \ + illumination/exemplar-001-cycle-07-dfp.tif \ + illumination/exemplar-001-cycle-08-dfp.tif \ + --align-channel 0 \ + -o ../exemplar-001-registered.ome.tif ) +else + echo ">> exemplar-001-registered.ome.tif already present, skipping registration" +fi + +if [ -d exemplar-001.zarr ]; then + echo ">> exemplar-001.zarr already exists; remove it to rebuild" +else + echo ">> Converting to SpatialData" + python "$SCRIPT_DIR/ometiff_to_spatialdata.py" \ + --image exemplar-001-registered.ome.tif \ + --markers exemplar-001/markers.csv \ + --output exemplar-001.zarr \ + --image-key exemplar-001_image +fi + +echo +echo ">> Done. Run the benchmark with:" +echo " ZARR_PATH=$(pwd)/exemplar-001.zarr bash src/workflows/run_benchmark/test.sh" diff --git a/scripts/create_resources/resources.sh b/scripts/create_resources/resources.sh index 57f4d68..127df99 100755 --- a/scripts/create_resources/resources.sh +++ b/scripts/create_resources/resources.sh @@ -8,7 +8,7 @@ cd "$REPO_ROOT" # remove this when you have implemented the script echo "TODO: once the 'process_datasets' workflow is implemented, update this script to use it." -echo " Step 1: replace 'task_template' with the name of the task in the following command." +echo " Step 1: replace 'task_spatial_proteomics_preprocessing' with the name of the task in the following command." echo " Step 2: replace the rename keys parameters to fit your process_dataset inputs" echo " Step 3: replace the settings parameter to fit your process_dataset outputs" echo " Step 4: remove this message" @@ -19,10 +19,10 @@ input_states: s3://openproblems-data/resources/datasets/**/state.yaml rename_keys: 'input:output_dataset' output_state: '$id/state.yaml' settings: '{"output_train": "$id/train.h5ad", "output_test": "$id/test.h5ad", "output_solution": "$id/solution.h5ad"}' -publish_dir: s3://openproblems-data/resources/task_template/datasets/ +publish_dir: s3://openproblems-data/resources/task_spatial_proteomics_preprocessing/datasets/ HERE -tw launch https://github.com/openproblems-bio/task_template.git \ +tw launch https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing.git \ --revision build/main \ --pull-latest \ --main-script target/nextflow/workflows/process_datasets/main.nf \ @@ -31,4 +31,4 @@ tw launch https://github.com/openproblems-bio/task_template.git \ --params-file /tmp/params.yaml \ --entry-name auto \ --config common/nextflow_helpers/labels_tw.config \ - --labels task_template,process_datasets + --labels task_spatial_proteomics_preprocessing,process_datasets diff --git a/scripts/create_resources/test_resources.sh b/scripts/create_resources/test_resources.sh index 9cb372a..30e6e3d 100755 --- a/scripts/create_resources/test_resources.sh +++ b/scripts/create_resources/test_resources.sh @@ -14,7 +14,7 @@ cd "$REPO_ROOT" set -e RAW_DATA=resources_test/common -DATASET_DIR=resources_test/task_template +DATASET_DIR=resources_test/task_spatial_proteomics_preprocessing mkdir -p $DATASET_DIR @@ -49,5 +49,5 @@ HERE # only run this if you have access to the openproblems-data bucket aws s3 sync --profile op \ - "$DATASET_DIR" s3://openproblems-data/resources_test/task_template \ + "$DATASET_DIR" s3://openproblems-data/resources_test/task_spatial_proteomics_preprocessing \ --delete --dryrun diff --git a/scripts/run_benchmark/run_full_local.sh b/scripts/run_benchmark/run_full_local.sh index bca3d7d..7b0f609 100755 --- a/scripts/run_benchmark/run_full_local.sh +++ b/scripts/run_benchmark/run_full_local.sh @@ -13,7 +13,7 @@ cd "$REPO_ROOT" # remove this when you have updated the script for your task echo "TODO: update this script to fit your task." -echo " Step 1: replace 'task_template' with the name of the task in the commands below." +echo " Step 1: replace 'task_spatial_proteomics_preprocessing' with the name of the task in the commands below." echo " Step 2: update the rename keys to match the run_benchmark workflow inputs." echo " Step 3: remove this message." exit 1 @@ -36,7 +36,7 @@ publish_dir: "$publish_dir" HERE # run the benchmark -nextflow run openproblems-bio/task_template \ +nextflow run openproblems-bio/task_spatial_proteomics_preprocessing \ -r build/main \ -main-script target/nextflow/workflows/run_benchmark/main.nf \ -profile docker \ diff --git a/scripts/run_benchmark/run_full_seqeracloud.sh b/scripts/run_benchmark/run_full_seqeracloud.sh index 45ee247..29f5492 100755 --- a/scripts/run_benchmark/run_full_seqeracloud.sh +++ b/scripts/run_benchmark/run_full_seqeracloud.sh @@ -8,7 +8,7 @@ cd "$REPO_ROOT" # remove this when you have updated the script for your task echo "TODO: update this script to fit your task." -echo " Step 1: replace 'task_template' with the name of the task in the commands below." +echo " Step 1: replace 'task_spatial_proteomics_preprocessing' with the name of the task in the commands below." echo " Step 2: update the rename keys to match the run_benchmark workflow inputs." echo " Step 3: remove this message." exit 1 @@ -17,17 +17,17 @@ set -e # generate a unique id RUN_ID="run_$(date +%Y-%m-%d_%H-%M-%S)" -publish_dir="s3://openproblems-data/resources/task_template/results/${RUN_ID}" +publish_dir="s3://openproblems-data/resources/task_spatial_proteomics_preprocessing/results/${RUN_ID}" # write the parameters to file cat > /tmp/params.yaml << HERE -input_states: s3://openproblems-data/resources/task_template/datasets/**/state.yaml +input_states: s3://openproblems-data/resources/task_spatial_proteomics_preprocessing/datasets/**/state.yaml rename_keys: 'input_train:output_train;input_test:output_test;input_solution:output_solution' output_state: "state.yaml" publish_dir: "$publish_dir" HERE -tw launch https://github.com/openproblems-bio/task_template.git \ +tw launch https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing.git \ --revision build/main \ --pull-latest \ --main-script target/nextflow/workflows/run_benchmark/main.nf \ @@ -36,4 +36,4 @@ tw launch https://github.com/openproblems-bio/task_template.git \ --params-file /tmp/params.yaml \ --entry-name auto \ --config common/nextflow_helpers/labels_tw.config \ - --labels task_template,full \ No newline at end of file + --labels task_spatial_proteomics_preprocessing,full \ No newline at end of file diff --git a/scripts/run_benchmark/run_test_local.sh b/scripts/run_benchmark/run_test_local.sh index 4ab9f25..6f15bba 100755 --- a/scripts/run_benchmark/run_test_local.sh +++ b/scripts/run_benchmark/run_test_local.sh @@ -7,7 +7,7 @@ REPO_ROOT=$(git rev-parse --show-toplevel) cd "$REPO_ROOT" # NOTE: this script works as-is on the template's test resources. -# When adapting the template to a new task, replace 'task_template' in the +# When adapting the template to a new task, replace 'task_spatial_proteomics_preprocessing' in the # paths below and update the arguments to match your run_benchmark workflow. set -e @@ -25,8 +25,8 @@ nextflow run . \ -resume \ -c common/nextflow_helpers/labels_ci.config \ --id cxg_mouse_pancreas_atlas \ - --input_train resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad \ - --input_test resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad \ - --input_solution resources_test/task_template/cxg_mouse_pancreas_atlas/solution.h5ad \ + --input_train resources_test/task_spatial_proteomics_preprocessing/cxg_mouse_pancreas_atlas/train.h5ad \ + --input_test resources_test/task_spatial_proteomics_preprocessing/cxg_mouse_pancreas_atlas/test.h5ad \ + --input_solution resources_test/task_spatial_proteomics_preprocessing/cxg_mouse_pancreas_atlas/solution.h5ad \ --output_state state.yaml \ --publish_dir "$publish_dir" diff --git a/scripts/run_benchmark/run_test_seqeracloud.sh b/scripts/run_benchmark/run_test_seqeracloud.sh index 1bb81c4..7960108 100755 --- a/scripts/run_benchmark/run_test_seqeracloud.sh +++ b/scripts/run_benchmark/run_test_seqeracloud.sh @@ -8,14 +8,14 @@ cd "$REPO_ROOT" # remove this when you have updated the script for your task echo "TODO: update this script to fit your task." -echo " Step 1: replace 'task_template' with the name of the task in the commands below." +echo " Step 1: replace 'task_spatial_proteomics_preprocessing' with the name of the task in the commands below." echo " Step 2: update the parameters to match the run_benchmark workflow inputs." echo " Step 3: remove this message." exit 1 set -e -resources_test_s3=s3://openproblems-data/resources_test/task_template +resources_test_s3=s3://openproblems-data/resources_test/task_spatial_proteomics_preprocessing publish_dir_s3="s3://openproblems-nextflow/temp/results/$(date +%Y-%m-%d_%H-%M-%S)" # write the parameters to file @@ -28,7 +28,7 @@ output_state: "state.yaml" publish_dir: $publish_dir_s3 HERE -tw launch https://github.com/openproblems-bio/task_template.git \ +tw launch https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing.git \ --revision build/main \ --pull-latest \ --main-script target/nextflow/workflows/run_benchmark/main.nf \ @@ -36,4 +36,4 @@ tw launch https://github.com/openproblems-bio/task_template.git \ --compute-env 6TeIFgV5OY4pJCk8I0bfOh \ --params-file /tmp/params.yaml \ --config common/nextflow_helpers/labels_tw.config \ - --labels task_template,test + --labels task_spatial_proteomics_preprocessing,test diff --git a/src/api/comp_control_method.yaml b/src/api/comp_control_method.yaml deleted file mode 100644 index f637aed..0000000 --- a/src/api/comp_control_method.yaml +++ /dev/null @@ -1,37 +0,0 @@ -namespace: control_methods -info: - type: control_method - type_info: - label: Control Method - summary: Quality control methods for verifying the pipeline. - description: | - This folder contains control components for the task. - These components have the same interface as the regular methods - but also receive the solution object as input. It serves as a - starting point to test the relative accuracy of new methods in - the task, and also as a quality control for the metrics defined - in the task. -arguments: - - name: --input_train - __merge__: file_train.yaml - required: true - direction: input - - name: --input_test - __merge__: file_test.yaml - required: true - direction: input - - name: "--input_solution" - __merge__: file_solution.yaml - direction: input - required: true - - name: --output - __merge__: file_prediction.yaml - required: true - direction: output -test_resources: - - type: python_script - path: /common/component_tests/run_and_check_output.py - - type: python_script - path: /common/component_tests/check_config.py - - path: /resources_test/task_template/cxg_mouse_pancreas_atlas - dest: resources_test/task_template/cxg_mouse_pancreas_atlas \ No newline at end of file diff --git a/src/api/comp_data_processor.yaml b/src/api/comp_data_processor.yaml deleted file mode 100644 index 1ed53bd..0000000 --- a/src/api/comp_data_processor.yaml +++ /dev/null @@ -1,31 +0,0 @@ -namespace: "data_processors" -info: - type: data_processor - type_info: - label: Data processor - summary: A data processor. - description: | - A component for processing a Common Dataset into a task-specific dataset. -arguments: - - name: "--input" - __merge__: file_common_dataset.yaml - direction: input - required: true - - name: "--output_train" - __merge__: file_train.yaml - direction: output - required: true - - name: "--output_test" - __merge__: file_test.yaml - direction: output - required: true - - name: "--output_solution" - __merge__: file_solution.yaml - direction: output - required: true -test_resources: - - path: /resources_test/common/cxg_mouse_pancreas_atlas - dest: resources_test/common/cxg_mouse_pancreas_atlas - - type: python_script - path: /common/component_tests/run_and_check_output.py - diff --git a/src/api/comp_method.yaml b/src/api/comp_method.yaml deleted file mode 100644 index 3a93846..0000000 --- a/src/api/comp_method.yaml +++ /dev/null @@ -1,28 +0,0 @@ -namespace: "methods" -info: - type: method - type_info: - label: Method - summary: A method. - description: | - A method to predict the task effects. -arguments: - - name: --input_train - __merge__: file_train.yaml - required: true - direction: input - - name: "--input_test" - __merge__: file_test.yaml - direction: input - required: true - - name: --output - __merge__: file_prediction.yaml - required: true - direction: output -test_resources: - - type: python_script - path: /common/component_tests/run_and_check_output.py - - type: python_script - path: /common/component_tests/check_config.py - - path: /resources_test/task_template/cxg_mouse_pancreas_atlas - dest: resources_test/task_template/cxg_mouse_pancreas_atlas \ No newline at end of file diff --git a/src/api/comp_method_image_correction.yaml b/src/api/comp_method_image_correction.yaml new file mode 100644 index 0000000..c73779d --- /dev/null +++ b/src/api/comp_method_image_correction.yaml @@ -0,0 +1,48 @@ +namespace: methods_image_correction +info: + type: method + subtype: image_correction + type_info: + label: Image correction + summary: Correct raw multiplexed protein images + description: | + Performs minimal image correction on a multichannel image extracted from a SpatialData zarr. +arguments: + - name: --input_zarr + type: file + direction: input + required: true + description: Input SpatialData zarr. + - name: --image_key + type: string + direction: input + required: false + default: null + description: Image element key. If null, the first image is used. + - name: --background_percentile + type: double + direction: input + required: false + default: 1.0 + description: Percentile used as per-channel background estimate. + - name: --rescale_max + type: boolean + direction: input + required: false + default: false + description: Whether to rescale every channel to max 1 after correction. + - name: --output_image + type: file + direction: output + required: true + description: Corrected image as numpy array in channel-y-x format. + - name: --output_markers + type: file + direction: output + required: true + description: JSON file containing marker names. + +test_resources: + - type: python_script + path: /src/testing/test_image_correction.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_method_protein_aggregation_arrays.yaml b/src/api/comp_method_protein_aggregation_arrays.yaml new file mode 100644 index 0000000..f840356 --- /dev/null +++ b/src/api/comp_method_protein_aggregation_arrays.yaml @@ -0,0 +1,51 @@ +namespace: methods_protein_aggregation + +info: + label: "Protein aggregation method from image arrays" + summary: "Shared interface for aggregating multichannel image intensities into cell-by-protein matrices." + description: | + Components implementing this interface take a corrected multichannel image array, a segmentation label mask, + and marker metadata, and produce an AnnData object containing one row per segmented object and one column per + protein marker. The interface is designed for benchmarking alternative aggregation strategies such as mean, + sum, and robust median aggregation. + +argument_groups: + - name: Inputs + arguments: + - name: "--input_image" + type: file + direction: input + required: true + description: "Corrected multichannel image stored as a NumPy array with shape channels by height by width." + - name: "--input_labels" + type: file + direction: input + required: true + description: "Segmentation label mask stored as a two-dimensional NumPy array with shape height by width." + - name: "--input_markers" + type: file + direction: input + required: true + description: "JSON file containing marker names, either as a list or under the key markers." + + - name: Method parameters + arguments: + - name: "--stat" + type: string + direction: input + required: false + default: "mean" + description: "Legacy configurable aggregation statistic. Dedicated benchmark methods ignore this argument." + + - name: Outputs + arguments: + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData file containing the cell-by-protein intensity matrix." + +test_resources: + - type: python_script + path: /src/testing/test_aggregation.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_method_proteomics_normalization.yaml b/src/api/comp_method_proteomics_normalization.yaml new file mode 100644 index 0000000..ccc8a3c --- /dev/null +++ b/src/api/comp_method_proteomics_normalization.yaml @@ -0,0 +1,45 @@ +namespace: methods_proteomics_normalization +info: + type: method + subtype: normalization + type_info: + label: "Proteomics normalization" + summary: "Normalize cell-by-protein intensity matrices" + description: | + Shared interface for spatial proteomics normalization methods. + Components read an AnnData object with raw intensities in layers['counts'] + or X and write an AnnData object containing layers['normalized']. +arguments: + - name: "--input" + type: file + direction: input + required: true + description: "Input AnnData file with cell-by-protein intensities." + - name: "--scale" + type: double + direction: input + required: false + default: 10000.0 + description: "Target total intensity used by total-log1p normalization." + - name: "--cofactor" + type: double + direction: input + required: false + default: 5.0 + description: "Cofactor used by arcsinh normalization." + - name: "--transform" + type: string + direction: input + required: false + default: "log1p" + description: "Legacy transform argument kept for backward compatibility." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData file with layers['normalized']." + +test_resources: + - type: python_script + path: /src/testing/test_normalization.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_method_proteomics_qc.yaml b/src/api/comp_method_proteomics_qc.yaml new file mode 100644 index 0000000..68ee288 --- /dev/null +++ b/src/api/comp_method_proteomics_qc.yaml @@ -0,0 +1,50 @@ +namespace: methods_proteomics_qc +info: + type: method + subtype: qc_filter + type_info: + label: Proteomics QC filter + summary: Compute QC annotations for spatial proteomics cell-by-protein matrices + description: | + Adds quality-control annotations to an AnnData object generated from spatial proteomics images. + Implementations must write obs['passed_QC'] and store method metadata in uns['qc']. +arguments: + - name: --input + type: file + direction: input + required: true + description: "Input AnnData h5ad with per-cell protein intensities." + - name: --min_area + type: integer + direction: input + required: false + default: 50 + description: "Minimum cell area in pixels for fixed-threshold QC." + - name: --min_total + type: double + direction: input + required: false + default: 0.0 + description: "Minimum total protein intensity for fixed-threshold QC." + - name: --area_quantile + type: double + direction: input + required: false + default: 0.05 + description: "Lower area quantile used by adaptive quantile QC." + - name: --total_quantile + type: double + direction: input + required: false + default: 0.05 + description: "Lower total-intensity quantile used by adaptive quantile QC." + - name: --output + type: file + direction: output + required: true + description: "Output AnnData h5ad with QC annotations." + +test_resources: + - type: python_script + path: /src/testing/test_qc.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_method_proteomics_segmentation.yaml b/src/api/comp_method_proteomics_segmentation.yaml new file mode 100644 index 0000000..830f1c6 --- /dev/null +++ b/src/api/comp_method_proteomics_segmentation.yaml @@ -0,0 +1,90 @@ +namespace: methods_proteomics_segmentation +info: + type: method + subtype: segmentation + type_info: + label: "Proteomics segmentation" + summary: "Segment cells from corrected spatial proteomics images" + description: | + Produces a cell label image from corrected multichannel protein images or from reference labels stored in a SpatialData object. + The API is intentionally shared by multiple segmentation methods so that benchmark workflows can swap methods while preserving standardized outputs. +arguments: + - name: "--input_image" + type: file + direction: input + required: false + description: "Corrected image as numpy array in channel-y-x format. Required for image-derived segmentation methods." + - name: "--input_zarr" + type: file + direction: input + required: false + description: "SpatialData Zarr store. Required for reference-label segmentation." + - name: "--labels_key" + type: string + direction: input + required: false + default: null + description: "SpatialData labels key to use for reference segmentation. If null, the first labels element is used." + - name: "--nuclei_channel" + type: integer + direction: input + required: false + default: 0 + description: "Channel index used as nuclear marker for image-derived segmentation." + - name: "--threshold_method" + type: string + direction: input + required: false + default: "quantile" + description: "Strategy used to threshold the nuclear channel. One of quantile or otsu." + - name: "--threshold_quantile" + type: double + direction: input + required: false + default: 0.90 + description: "Quantile used to threshold the nuclear channel." + - name: "--min_area" + type: integer + direction: input + required: false + default: 30 + description: "Minimum object size in pixels." + - name: "--smoothing_sigma" + type: double + direction: input + required: false + default: 1.0 + description: "Gaussian smoothing sigma used by watershed-based segmentation." + - name: "--hole_area" + type: integer + direction: input + required: false + default: 64 + description: "Maximum hole area removed from the nuclear foreground mask." + - name: "--seed_min_distance" + type: integer + direction: input + required: false + default: 5 + description: "Minimum distance between watershed seed points." + - name: "--expansion_distance" + type: integer + direction: input + required: false + default: 5 + description: "Pixel distance used to expand nuclear labels toward approximate cell regions." + - name: "--output_labels" + type: file + direction: output + required: true + description: "Output segmentation label image as a numpy array." + - name: "--output_metadata" + type: file + direction: output + required: false + description: "Optional JSON file describing the segmentation method, its parameters, and the resulting object count." + +test_resources: + - type: python_script + path: /src/testing/test_segmentation.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_metric.yaml b/src/api/comp_metric.yaml deleted file mode 100644 index 1c76a3d..0000000 --- a/src/api/comp_metric.yaml +++ /dev/null @@ -1,28 +0,0 @@ -namespace: "metrics" -info: - type: metric - type_info: - label: Metric - summary: A task template metric. - description: | - A metric for evaluating method predictions. -arguments: - - name: "--input_solution" - __merge__: file_solution.yaml - direction: input - required: true - - name: "--input_prediction" - __merge__: file_prediction.yaml - direction: input - required: true - - name: "--output" - __merge__: file_score.yaml - direction: output - required: true -test_resources: - - type: python_script - path: /common/component_tests/run_and_check_output.py - - type: python_script - path: /common/component_tests/check_config.py - - path: /resources_test/task_template/cxg_mouse_pancreas_atlas - dest: resources_test/task_template/cxg_mouse_pancreas_atlas diff --git a/src/api/comp_metric_proteomics_basic.yaml b/src/api/comp_metric_proteomics_basic.yaml new file mode 100644 index 0000000..c4e7c2e --- /dev/null +++ b/src/api/comp_metric_proteomics_basic.yaml @@ -0,0 +1,40 @@ +namespace: metrics_proteomics +info: + type: metric + subtype: proteomics_basic + type_info: + label: "Proteomics basic metrics" + summary: "Compute basic cell-level QC, morphology, intensity, and marker coverage metrics" + description: | + Produces an AnnData object with benchmark metrics stored in uns['metric_ids'] + and uns['metric_values']. This metric family summarizes core preprocessing + outputs such as cell count, QC pass rate, cell area, total intensity, and + marker coverage. +arguments: + - name: "--input" + type: file + direction: input + required: true + description: "Input AnnData file with cell-by-protein intensities and QC annotations." + - name: "--cov_thr" + type: double + direction: input + required: false + default: 0.1 + description: "Intensity threshold used for marker coverage computation." + - name: "--coverage_threshold" + type: double + direction: input + required: false + default: 0.1 + description: "Backward-compatible alias for --cov_thr." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData file with uns['metric_ids'] and uns['metric_values']." + +test_resources: + - type: python_script + path: /src/testing/test_metrics.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_metric_proteomics_marker_distribution.yaml b/src/api/comp_metric_proteomics_marker_distribution.yaml new file mode 100644 index 0000000..e04f81f --- /dev/null +++ b/src/api/comp_metric_proteomics_marker_distribution.yaml @@ -0,0 +1,57 @@ +namespace: metrics_proteomics +info: + type: metric + subtype: proteomics_marker_distribution + type_info: + label: "Proteomics marker distribution metrics" + summary: "Compute marker-level sparsity, coverage, intensity, and dynamic-range metrics" + description: | + Produces an AnnData object with marker distribution metrics stored in + uns['metric_ids'] and uns['metric_values']. These metrics are useful for + comparing preprocessing choices that alter marker intensity distributions. +arguments: + - name: "--input" + type: file + direction: input + required: true + description: "Input AnnData file with cell-by-protein intensities." + - name: "--layer" + type: string + direction: input + required: false + default: "counts" + description: "Layer to evaluate. If missing, X is used. Use counts for raw marker distributions." + - name: "--zero_threshold" + type: double + direction: input + required: false + default: 0.0 + description: "Values less than or equal to this threshold are counted as zero or absent." + - name: "--coverage_threshold" + type: double + direction: input + required: false + default: 0.1 + description: "Values above this threshold are counted as marker-positive for coverage." + - name: "--low_quantile" + type: double + direction: input + required: false + default: 0.01 + description: "Lower quantile for robust dynamic-range estimation." + - name: "--high_quantile" + type: double + direction: input + required: false + default: 0.99 + description: "Upper quantile for robust dynamic-range estimation." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData file with uns['metric_ids'] and uns['metric_values']." + +test_resources: + - type: python_script + path: /src/testing/test_metrics.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_metric_proteomics_normalization_diagnostics.yaml b/src/api/comp_metric_proteomics_normalization_diagnostics.yaml new file mode 100644 index 0000000..4f714ca --- /dev/null +++ b/src/api/comp_metric_proteomics_normalization_diagnostics.yaml @@ -0,0 +1,52 @@ +namespace: metrics_proteomics +info: + type: metric + subtype: proteomics_normalization_diagnostics + type_info: + label: "Proteomics normalization diagnostics" + summary: "Compare raw and normalized intensity matrices" + description: | + Produces an AnnData object with normalization diagnostics stored in + uns['metric_ids'] and uns['metric_values']. The metrics compare raw counts + to the normalized representation and quantify total-intensity compression, + rank preservation, and variance changes. +arguments: + - name: "--input" + type: file + direction: input + required: true + description: "Input AnnData file with layers['counts'] and preferably layers['normalized']." + - name: "--normalized_layer" + type: string + direction: input + required: false + default: "normalized" + description: "Layer containing normalized values. If missing, X is used as fallback." + - name: "--raw_layer" + type: string + direction: input + required: false + default: "counts" + description: "Layer containing raw values. If missing, X is used as fallback." + - name: "--max_cells" + type: integer + direction: input + required: false + default: 5000 + description: "Maximum number of cells used for rank-preservation diagnostics. Use 0 to disable subsampling." + - name: "--random_state" + type: integer + direction: input + required: false + default: 0 + description: "Random seed used for subsampling." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData file with uns['metric_ids'] and uns['metric_values']." + +test_resources: + - type: python_script + path: /src/testing/test_metrics.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_metric_proteomics_segmentation_quality.yaml b/src/api/comp_metric_proteomics_segmentation_quality.yaml new file mode 100644 index 0000000..47eb1e7 --- /dev/null +++ b/src/api/comp_metric_proteomics_segmentation_quality.yaml @@ -0,0 +1,73 @@ +namespace: metrics_proteomics +info: + type: metric + subtype: proteomics_segmentation_quality + type_info: + label: "Proteomics segmentation quality metrics" + summary: "Score segmentation without reference masks, using known marker biology" + description: | + Segmentation is the central benchmarked step, but the dataset-provided + reference labels needed to score it directly are often unavailable. These + metrics exploit properties the data must satisfy if segmentation is + correct, so they carry a known good direction without any ground truth: + + - every segmented object should contain nuclear signal; + - nuclear stains repeated across imaging cycles are the same physical + nuclei, so within one correctly delineated cell they must agree; + - markers restricted to mutually exclusive lineages should rarely be + co-positive in the same cell; + - a marker positive in one cell should not appear in its neighbours. + + Reported over all cells and over the QC-passing subset. +arguments: + - name: "--input" + type: file + direction: input + required: true + description: "Input AnnData with cell-by-protein intensities and, ideally, QC annotations." + - name: "--layer" + type: string + direction: input + required: false + default: "counts" + description: "Layer to evaluate. Falls back to X when absent." + - name: "--nuclear_markers" + type: string + direction: input + required: false + default: null + description: "Comma-separated nuclear/DNA marker names. If null, marker names matching DNA, DAPI or Hoechst are auto-detected." + - name: "--exclusive_pairs" + type: string + direction: input + required: false + default: null + description: "Comma-separated mutually exclusive marker pairs, each as A:B (for example ECAD:CD45). Co-expression metrics are NaN when unset, rather than guessed for an unknown panel." + - name: "--n_neighbors" + type: integer + direction: input + required: false + default: 6 + description: "Neighbours per cell, by centroid distance, used for the spillover metric." + - name: "--max_cells" + type: integer + direction: input + required: false + default: 20000 + description: "Subsample cells above this count for the neighbour search. Use 0 to disable." + - name: "--random_state" + type: integer + direction: input + required: false + default: 0 + description: "Random seed used for subsampling." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData with uns['metric_ids'] and uns['metric_values']." + +test_resources: + - type: python_script + path: /src/testing/test_metrics.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_metric_proteomics_signal_background.yaml b/src/api/comp_metric_proteomics_signal_background.yaml new file mode 100644 index 0000000..8bc7d46 --- /dev/null +++ b/src/api/comp_metric_proteomics_signal_background.yaml @@ -0,0 +1,69 @@ +namespace: metrics_proteomics +info: + type: metric + subtype: proteomics_signal_background + type_info: + label: "Proteomics signal-versus-background metrics" + summary: "Score image correction and mask placement against a local background" + description: | + The only metrics in this benchmark computed from pixels rather than from + the aggregated matrix, and therefore the only ones that can see what the + cell table cannot: how much of each marker's signal actually falls inside + the masks, and how much of it is background that correction failed to + remove. + + Background is sampled from a ring immediately outside the masks rather + than from all non-cell pixels, so the comparison is local and is not + dominated by empty regions of the slide. + + Adapted from the scverse hackathon `sprot` prototype + (`metrics/protein_quality_metrics.py`). +arguments: + - name: "--input_image" + type: file + direction: input + required: true + description: "Corrected multichannel image as a NumPy array, channels by height by width." + - name: "--input_labels" + type: file + direction: input + required: true + description: "Segmentation label mask as a two-dimensional NumPy array." + - name: "--input_markers" + type: file + direction: input + required: false + description: "JSON file with marker names, either a list or under the key markers." + - name: "--input_qc" + type: file + direction: input + required: false + description: "Optional AnnData carrying obs['passed_QC'], used to report the QC-passing variants. Without it those are NaN." + - name: "--dilation_iterations" + type: integer + direction: input + required: false + default: 3 + description: "How far to grow the masks when sampling proximal background, in pixels." + - name: "--norm_percentile" + type: double + direction: input + required: false + default: 95.0 + description: "Percentile of per-cell intensity used to normalise the signal integral." + - name: "--random_state" + type: integer + direction: input + required: false + default: 0 + description: "Random seed for background sampling, which is otherwise irreproducible." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData with uns['metric_ids'] and uns['metric_values']." + +test_resources: + - type: python_script + path: /src/testing/test_signal_background.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/comp_metric_proteomics_technical_covariates.yaml b/src/api/comp_metric_proteomics_technical_covariates.yaml new file mode 100644 index 0000000..84d5a67 --- /dev/null +++ b/src/api/comp_metric_proteomics_technical_covariates.yaml @@ -0,0 +1,70 @@ +namespace: metrics_proteomics +info: + type: metric + subtype: proteomics_technical_covariates + type_info: + label: "Proteomics technical covariate metrics" + summary: "How much of the signal is still explained by technical factors" + description: | + Aggregation and normalization are meant to yield per-cell protein + profiles that reflect biology rather than how large a cell happened to be + or how bright its mask was. These metrics quantify how far that holds: + + - variance in expression attributable to cell area; + - correlation between the leading principal components and cell area; + - dependence of per-marker variance on per-marker mean, which is what a + variance-stabilising transform is supposed to remove; + - whether variance is spread across the panel or dominated by a few + bright markers. + + Adapted from the scverse hackathon `sprot` prototype + (`metrics/technical_correction.py`, `metrics/protein_contribution.py`), + with cell area replacing the single-cell covariates it assumed. + + Reported over all cells and over the QC-passing subset. +arguments: + - name: "--input" + type: file + direction: input + required: true + description: "Input AnnData with cell-by-protein intensities and obs['area']." + - name: "--layer" + type: string + direction: input + required: false + default: "normalized" + description: "Layer to evaluate. Falls back to counts, then X, when absent." + - name: "--covariate" + type: string + direction: input + required: false + default: "area" + description: "obs column treated as the technical covariate." + - name: "--n_pcs" + type: integer + direction: input + required: false + default: 10 + description: "Principal components correlated against the covariate. Clamped to the data's rank." + - name: "--max_cells" + type: integer + direction: input + required: false + default: 20000 + description: "Subsample cells above this count. Use 0 to disable." + - name: "--random_state" + type: integer + direction: input + required: false + default: 0 + description: "Random seed used for subsampling." + - name: "--output" + type: file + direction: output + required: true + description: "Output AnnData with uns['metric_ids'] and uns['metric_values']." + +test_resources: + - type: python_script + path: /src/testing/test_metrics.py + - path: /src/testing/isp_fixtures.py diff --git a/src/api/file_common_dataset.yaml b/src/api/file_common_dataset.yaml deleted file mode 100644 index e8a74a0..0000000 --- a/src/api/file_common_dataset.yaml +++ /dev/null @@ -1,72 +0,0 @@ -type: file -example: "resources_test/common/cxg_mouse_pancreas_atlas/dataset.h5ad" -label: "Common Dataset" -summary: A subset of the common dataset. -info: - format: - type: h5ad - layers: - - type: integer - name: counts - description: Raw counts - required: true - - type: double - name: normalized - description: Normalized expression values - required: true - obs: - - type: string - name: cell_type - description: Cell type information - required: true - - type: string - name: batch - description: Batch information - required: true - var: - - type: boolean - name: hvg - description: Whether or not the feature is considered to be a 'highly variable gene' - required: true - - type: double - name: hvg_score - description: A ranking of the features by hvg. - required: true - obsm: - - type: double - name: X_pca - description: The resulting PCA embedding. - required: true - uns: - - type: string - name: dataset_id - description: "A unique identifier for the dataset" - required: true - - name: dataset_name - type: string - description: Nicely formatted name. - required: true - - type: string - name: dataset_url - description: Link to the original source of the dataset. - required: false - - name: dataset_reference - type: string - description: Bibtex reference of the paper in which the dataset was published. - required: false - - name: dataset_summary - type: string - description: Short description of the dataset. - required: true - - name: dataset_description - type: string - description: Long description of the dataset. - required: true - - name: dataset_organism - type: string - description: The organism of the sample in the dataset. - required: false - - type: string - name: normalization_id - description: "Which normalization was used" - required: true diff --git a/src/api/file_prediction.yaml b/src/api/file_prediction.yaml deleted file mode 100644 index 26068ab..0000000 --- a/src/api/file_prediction.yaml +++ /dev/null @@ -1,26 +0,0 @@ -#TODO: Change to the required and/or optional fields of the anndata -type: file -example: "resources_test/task_template/cxg_mouse_pancreas_atlas/prediction.h5ad" -label: "Predicted data" -summary: A predicted dataset as output by a method. -info: - format: - type: h5ad - obs: - - type: string - name: label_pred - description: Predicted labels for the test cells. - required: true - uns: - - type: string - name: dataset_id - description: "A unique identifier for the dataset" - required: true - - type: string - name: normalization_id - description: "Which normalization was used" - required: true - - type: string - name: method_id - description: "A unique identifier for the method" - required: true \ No newline at end of file diff --git a/src/api/file_score.yaml b/src/api/file_score.yaml index 8bdad65..eb1e662 100644 --- a/src/api/file_score.yaml +++ b/src/api/file_score.yaml @@ -1,7 +1,7 @@ type: file -example: "resources_test/task_template/cxg_mouse_pancreas_atlas/score.h5ad" -label: Score -summary: "File indicating the score of a metric." +example: "resources_test/task_spatial_proteomics_preprocessing/exemplar_001/score.h5ad" +label: "Score" +summary: "Metric score file" info: format: type: h5ad @@ -16,7 +16,7 @@ info: required: true - type: string name: method_id - description: "A unique identifier for the method" + description: "A unique identifier for the method combination that produced this result" required: true - type: string name: metric_ids @@ -27,4 +27,4 @@ info: name: metric_values description: "The metric values obtained for the given prediction. Must be of same length as 'metric_ids'." multiple: true - required: true \ No newline at end of file + required: true diff --git a/src/api/file_solution.yaml b/src/api/file_solution.yaml deleted file mode 100644 index d2f6200..0000000 --- a/src/api/file_solution.yaml +++ /dev/null @@ -1,73 +0,0 @@ -#TODO: Change to the required and/or optional fields of the anndata -type: file -example: "resources_test/task_template/cxg_mouse_pancreas_atlas/solution.h5ad" -label: "Solution" -summary: "The solution for the test data" -info: - format: - type: h5ad - layers: - - type: integer - name: counts - description: Raw counts - required: true - - type: double - name: normalized - description: Normalized counts - required: true - obs: - - type: string - name: label - description: Ground truth cell type labels - required: true - - type: string - name: batch - description: Batch information - required: true - var: - - type: boolean - name: hvg - description: Whether or not the feature is considered to be a 'highly variable gene' - required: true - - type: double - name: hvg_score - description: A ranking of the features by hvg. - required: true - obsm: - - type: double - name: X_pca - description: The resulting PCA embedding. - required: true - uns: - - type: string - name: dataset_id - description: "A unique identifier for the dataset" - required: true - - name: dataset_name - type: string - description: Nicely formatted name. - required: true - - type: string - name: dataset_url - description: Link to the original source of the dataset. - required: false - - name: dataset_reference - type: string - description: Bibtex reference of the paper in which the dataset was published. - required: false - - name: dataset_summary - type: string - description: Short description of the dataset. - required: true - - name: dataset_description - type: string - description: Long description of the dataset. - required: true - - name: dataset_organism - type: string - description: The organism of the sample in the dataset. - required: false - - type: string - name: normalization_id - description: "Which normalization was used" - required: true diff --git a/src/api/file_test.yaml b/src/api/file_test.yaml deleted file mode 100644 index cb9d9a6..0000000 --- a/src/api/file_test.yaml +++ /dev/null @@ -1,45 +0,0 @@ -#TODO: Change to the required and/or optional fields of the anndata -type: file -example: "resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad" -label: "Test data" -summary: The subset of molecules used for the test dataset -info: - format: - type: h5ad - layers: - - type: integer - name: counts - description: Raw counts - required: true - - type: double - name: normalized - description: Normalized counts - required: true - obs: - - type: string - name: batch - description: Batch information - required: true - var: - - type: boolean - name: hvg - description: Whether or not the feature is considered to be a 'highly variable gene' - required: true - - type: double - name: hvg_score - description: A ranking of the features by hvg. - required: true - obsm: - - type: double - name: X_pca - description: The resulting PCA embedding. - required: true - uns: - - type: string - name: dataset_id - description: "A unique identifier for the dataset" - required: true - - type: string - name: normalization_id - description: "Which normalization was used" - required: true \ No newline at end of file diff --git a/src/api/file_train.yaml b/src/api/file_train.yaml deleted file mode 100644 index c01eda5..0000000 --- a/src/api/file_train.yaml +++ /dev/null @@ -1,49 +0,0 @@ -#TODO: Change to the required and/or optional fields of the anndata -type: file -example: "resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad" -label: "Training data" -summary: "The training data in h5ad format" -info: - format: - type: h5ad - layers: - - type: integer - name: counts - description: Raw counts - required: true - - type: double - name: normalized - description: Normalized counts - required: true - obs: - - type: string - name: label - description: Ground truth cell type labels - required: true - - type: string - name: batch - description: Batch information - required: true - var: - - type: boolean - name: hvg - description: Whether or not the feature is considered to be a 'highly variable gene' - required: true - - type: double - name: hvg_score - description: A ranking of the features by hvg. - required: true - obsm: - - type: double - name: X_pca - description: The resulting PCA embedding. - required: true - uns: - - type: string - name: dataset_id - description: "A unique identifier for the dataset" - required: true - - type: string - name: normalization_id - description: "Which normalization was used" - required: true \ No newline at end of file diff --git a/src/control_methods/random_labels/config.vsh.yaml b/src/control_methods/random_labels/config.vsh.yaml deleted file mode 100644 index 7218ec5..0000000 --- a/src/control_methods/random_labels/config.vsh.yaml +++ /dev/null @@ -1,45 +0,0 @@ -# The API specifies which type of component this is. -# It contains specifications for: -# - The input/output files -# - Common parameters -# - A unit test -__merge__: ../../api/comp_control_method.yaml - -# A unique identifier for your component (required). -# Can contain only lowercase letters or underscores. -name: random_labels - -# A relatively short label, used when rendering visualisations (required) -label: Random Labels -# A one sentence summary of how this method works (required). Used when -# rendering summary tables. -summary: "a negative control, where the labels are randomly predicted." -# A multi-line description of how this component works (required). Used -# when rendering reference documentation. -description: | - A negative control, where the labels are randomly predicted based on the - label distribution of the training data, without looking at the input data. - -# Metadata for your component -info: - # Which normalisation method this component prefers to use (required). - preferred_normalization: counts - -# Resources required to run the component -resources: - # The script of your component (required) - - type: python_script - path: script.py - -engines: - # Specifications for the Docker image for this component. - - type: docker - image: openproblems/base_python:1 - -runners: - # This platform allows running the component natively - - type: executable - # Allows turning the component into a Nextflow module / pipeline. - - type: nextflow - directives: - label: [midtime, lowmem, lowcpu] diff --git a/src/control_methods/random_labels/script.py b/src/control_methods/random_labels/script.py deleted file mode 100644 index 3e64303..0000000 --- a/src/control_methods/random_labels/script.py +++ /dev/null @@ -1,47 +0,0 @@ -import anndata as ad -import numpy as np - -## VIASH START -# Note: this section is auto-generated by viash at runtime. To edit it, make changes -# in config.vsh.yaml and then run `viash config inject config.vsh.yaml`. -par = { - 'input_train': 'resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad', - 'input_test': 'resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad', - 'input_solution': 'resources_test/task_template/cxg_mouse_pancreas_atlas/solution.h5ad', - 'output': 'output.h5ad' -} -meta = { - 'name': 'random_labels' -} -## VIASH END - -print('Reading input files', flush=True) -input_train = ad.read_h5ad(par['input_train']) -input_test = ad.read_h5ad(par['input_test']) - -print('Compute label distribution', flush=True) -label_distribution = input_train.obs["label"].value_counts() -label_distribution = label_distribution / label_distribution.sum() - -print('Generate predictions', flush=True) -obs_label_pred = np.random.choice( - label_distribution.index, - size=input_test.n_obs, - replace=True, - p=label_distribution -) - -print("Write output AnnData to file", flush=True) -output = ad.AnnData( - uns={ - 'dataset_id': input_train.uns['dataset_id'], - 'normalization_id': input_train.uns['normalization_id'], - 'method_id': meta['name'] - }, - obs={ - 'label_pred': obs_label_pred - } -) -output.obs_names = input_test.obs_names - -output.write_h5ad(par['output'], compression='gzip') diff --git a/src/control_methods/true_labels/config.vsh.yaml b/src/control_methods/true_labels/config.vsh.yaml deleted file mode 100644 index 0a71e50..0000000 --- a/src/control_methods/true_labels/config.vsh.yaml +++ /dev/null @@ -1,59 +0,0 @@ -# The API specifies which type of component this is. -# It contains specifications for: -# - The input/output files -# - Common parameters -# - A unit test -__merge__: ../../api/comp_control_method.yaml - -# A unique identifier for your component (required). -# Can contain only lowercase letters or underscores. -name: true_labels - -# A relatively short label, used when rendering visualisations (required) -label: True Labels -# A one sentence summary of how this method works (required). Used when -# rendering summary tables. -summary: "a positive control, solution labels are copied 1 to 1 to the predicted data." -# A multi-line description of how this component works (required). Used -# when rendering reference documentation. -description: | - A positive control, where the solution labels are copied 1 to 1 to the predicted data. - -# Metadata for your component -info: - # Which normalisation method this component prefers to use (required). - preferred_normalization: counts - -# Component-specific parameters (optional) -# arguments: -# - name: "--n_neighbors" -# type: "integer" -# default: 5 -# description: Number of neighbors to use. - -# Resources required to run the component -resources: - # The script of your component (required) - - type: python_script - path: script.py - # Additional resources your script needs (optional) - # - type: file - # path: weights.pt - -engines: - # Specifications for the Docker image for this component. - - type: docker - image: openproblems/base_python:1 - # Add custom dependencies here (optional). For more information, see - # https://viash.io/reference/config/engines/docker/#setup . - # setup: - # - type: python - # packages: scib==1.1.5 - -runners: - # This platform allows running the component natively - - type: executable - # Allows turning the component into a Nextflow module / pipeline. - - type: nextflow - directives: - label: [midtime, lowmem, lowcpu] diff --git a/src/control_methods/true_labels/script.py b/src/control_methods/true_labels/script.py deleted file mode 100644 index 935f3af..0000000 --- a/src/control_methods/true_labels/script.py +++ /dev/null @@ -1,45 +0,0 @@ -import anndata as ad - -## VIASH START -# Note: this section is auto-generated by viash at runtime. To edit it, make changes -# in config.vsh.yaml and then run `viash config inject config.vsh.yaml`. -par = { - 'input_train': 'resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad', - 'input_test': 'resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad', - 'input_solution': 'resources_test/task_template/cxg_mouse_pancreas_atlas/solution.h5ad', - 'output': 'output.h5ad' -} -meta = { - 'name': 'true_labels' -} -## VIASH END - -print('Reading input files', flush=True) -input_train = ad.read_h5ad(par['input_train']) -input_test = ad.read_h5ad(par['input_test']) -input_solution = ad.read_h5ad(par['input_solution']) - -print('Preprocess data', flush=True) -# ... preprocessing ... - -print('Train model', flush=True) -# ... train model ... - -print('Generate predictions', flush=True) -# ... generate predictions ... -obs_label_pred = input_solution.obs["label"] - -print("Write output AnnData to file", flush=True) -output = ad.AnnData( - uns={ - 'dataset_id': input_train.uns['dataset_id'], - 'normalization_id': input_train.uns['normalization_id'], - 'method_id': meta['name'] - }, - obs={ - 'label_pred': obs_label_pred - } -) -output.obs_names = input_test.obs_names - -output.write_h5ad(par['output'], compression='gzip') diff --git a/src/data_processors/process_dataset/config.vsh.yaml b/src/data_processors/process_dataset/config.vsh.yaml deleted file mode 100644 index 0047ae1..0000000 --- a/src/data_processors/process_dataset/config.vsh.yaml +++ /dev/null @@ -1,34 +0,0 @@ -__merge__: ../../api/comp_data_processor.yaml -name: process_dataset -arguments: - - name: "--method" - type: "string" - description: "The process method to assign train/test." - choices: ["batch", "random"] - default: "batch" - - name: "--obs_label" - type: "string" - description: "Which .obs slot to use as label." - default: "cell_type" - - name: "--obs_batch" - type: "string" - description: "Which .obs slot to use as batch covariate." - default: "batch" - - name: "--seed" - type: "integer" - description: "A seed for the subsampling." - example: 123 -resources: - - type: python_script - path: script.py - - path: /common/helper_functions/subset_h5ad_by_format.py - -engines: - - type: docker - image: openproblems/base_python:1 - -runners: - - type: executable - - type: nextflow - directives: - label: [highmem, midcpu, midtime] \ No newline at end of file diff --git a/src/data_processors/process_dataset/script.py b/src/data_processors/process_dataset/script.py deleted file mode 100644 index 0274c2d..0000000 --- a/src/data_processors/process_dataset/script.py +++ /dev/null @@ -1,87 +0,0 @@ -import sys -import random -import numpy as np -import anndata as ad -import openproblems as op - -## VIASH START -par = { - 'input': 'resources_test/common/cxg_mouse_pancreas_atlas/dataset.h5ad', - 'method': 'batch', - 'seed': None, - 'obs_batch': 'batch', - 'obs_label': 'cell_type', - 'output_train': 'train.h5ad', - 'output_test': 'test.h5ad', - 'output_solution': 'solution.h5ad' -} -meta = { - 'resources_dir': 'target/executable/data_processors/process_dataset', - 'config': 'target/executable/data_processors/process_dataset/.config.vsh.yaml' -} -## VIASH END - -# import helper functions -sys.path.append(meta['resources_dir']) -from subset_h5ad_by_format import subset_h5ad_by_format - -config = op.project.read_viash_config(meta["config"]) - -# set seed if need be -if par["seed"] is not None: - print(f">> Setting seed to {par['seed']}") - random.seed(par["seed"]) - np.random.seed(par["seed"]) - -print(">> Load data", flush=True) -adata = ad.read_h5ad(par["input"]) -print("input:", adata) - -print(f">> Process data using {par['method']} method") -if par["method"] == "batch": - batch_info = adata.obs[par["obs_batch"]] - batch_categories = batch_info.dtype.categories - test_batches = random.sample(list(batch_categories), 1) - is_test = [ x in test_batches for x in batch_info ] -elif par["method"] == "random": - train_ix = np.random.choice(adata.n_obs, round(adata.n_obs * 0.8), replace=False) - is_test = [ not x in train_ix for x in range(0, adata.n_obs) ] - -# subset the different adatas -print(">> Figuring which data needs to be copied to which output file", flush=True) -# use par arguments to look for label and batch value in different slots -slot_mapping = { - "obs": { - "label": par["obs_label"], - "batch": par["obs_batch"], - } -} - -print(">> Creating train data", flush=True) -output_train = subset_h5ad_by_format( - adata[[not x for x in is_test]], - config, - "output_train", - slot_mapping -) - -print(">> Creating test data", flush=True) -output_test = subset_h5ad_by_format( - adata[is_test], - config, - "output_test", - slot_mapping -) - -print(">> Creating solution data", flush=True) -output_solution = subset_h5ad_by_format( - adata[is_test], - config, - "output_solution", - slot_mapping -) - -print(">> Writing data", flush=True) -output_train.write_h5ad(par["output_train"]) -output_test.write_h5ad(par["output_test"]) -output_solution.write_h5ad(par["output_solution"]) diff --git a/src/methods/logistic_regression/config.vsh.yaml b/src/methods/logistic_regression/config.vsh.yaml deleted file mode 100644 index 3110b32..0000000 --- a/src/methods/logistic_regression/config.vsh.yaml +++ /dev/null @@ -1,95 +0,0 @@ -# The API specifies which type of component this is. -# It contains specifications for: -# - The input/output files -# - Common parameters -# - A unit test -__merge__: ../../api/comp_method.yaml - - -# A unique identifier for your component (required). -# Can contain only lowercase letters or underscores. -name: logistic_regression -# A relatively short label, used when rendering visualisations (required) -label: Logistic Regression -# A one sentence summary of how this method works (required). Used when -# rendering summary tables. -summary: "Logistic Regression with 100-dimensional PCA coordinates estimates parameters for multivariate classification by minimizing cross entropy loss over cell type classes." -# A multi-line description of how this component works (required). Used -# when rendering reference documentation. -description: | - Logistic Regression estimates parameters of a logistic function for - multivariate classification tasks. Here, we use 100-dimensional whitened PCA - coordinates as independent variables, and the model minimises the cross - entropy loss over all cell type classes. -# Metadata for your component -# A reference key from the bibtex library at src/common/library.bib (required). -references: - bibtex: - - | - @book{hosmer2013applied, - title = {Applied logistic regression}, - author = {Hosmer Jr, D.W. and Lemeshow, S. and Sturdivant, R.X.}, - year = {2013}, - publisher = {John Wiley \& Sons}, - volume = {398} - } - -links: - # URL to the code repository for this method (required). - repository: https://github.com/scikit-learn/scikit-learn - # URL to the documentation for this method (required). - documentation: "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html" - -info: - # Which normalisation method this component prefers to use (required). - preferred_normalization: log_cp10k - # Named parameter sets to evaluate in the benchmark (optional). The - # `run_benchmark` workflow runs the method once per variant, passing the - # listed values as arguments, and tags the scores with the variant name. - # Only the listed variants are run, so to also evaluate the default - # arguments, list them as a variant with their values spelled out (viash - # drops empty variants from the config). Without `variants`, the method is - # run once with its default arguments. - variants: - strong_regularization: - inverse_regularization_strength: 0.1 - default_regularization: - inverse_regularization_strength: 1.0 - weak_regularization: - inverse_regularization_strength: 10.0 - -# Component-specific parameters (optional) -arguments: - - name: "--inverse_regularization_strength" - type: double - default: 1.0 - description: | - Inverse of the regularization strength (`C` in scikit-learn). Smaller - values specify stronger regularization. - -# Resources required to run the component -resources: - # The script of your component (required) - - type: python_script - path: script.py - # Additional resources your script needs (optional) - # - type: file - # path: weights.pt - -engines: - # Specifications for the Docker image for this component. - - type: docker - image: openproblems/base_python:1 - # Add custom dependencies here (optional). For more information, see - # https://viash.io/reference/config/engines/docker/#setup . - setup: - - type: python - packages: scikit-learn - -runners: - # This platform allows running the component natively - - type: executable - # Allows turning the component into a Nextflow module / pipeline. - - type: nextflow - directives: - label: [midtime, midmem, lowcpu] diff --git a/src/methods/logistic_regression/script.py b/src/methods/logistic_regression/script.py deleted file mode 100644 index dd10667..0000000 --- a/src/methods/logistic_regression/script.py +++ /dev/null @@ -1,49 +0,0 @@ -import anndata as ad -import sklearn.linear_model - -## VIASH START -# Note: this section is auto-generated by viash at runtime. To edit it, make changes -# in config.vsh.yaml and then run `viash config inject config.vsh.yaml`. -par = { - 'input_train': 'resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad', - 'input_test': 'resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad', - 'output': 'output.h5ad', - 'inverse_regularization_strength': 1.0 -} -meta = { - 'name': 'logistic_regression' -} -## VIASH END - -print('Reading input files', flush=True) -input_train = ad.read_h5ad(par['input_train']) -input_test = ad.read_h5ad(par['input_test']) - -print('Preprocess data', flush=True) -# ... preprocessing ... - -print('Train model', flush=True) -# ... train model ... -classifier = sklearn.linear_model.LogisticRegression( - C=par['inverse_regularization_strength'] -) -classifier.fit(input_train.obsm["X_pca"], input_train.obs["label"].astype(str)) - -print('Generate predictions', flush=True) -# ... generate predictions ... -obs_label_pred = classifier.predict(input_test.obsm["X_pca"]) - -print("Write output AnnData to file", flush=True) -output = ad.AnnData( - uns={ - 'dataset_id': input_train.uns['dataset_id'], - 'normalization_id': input_train.uns['normalization_id'], - 'method_id': meta['name'] - }, - obs={ - 'label_pred': obs_label_pred - } -) -output.obs_names = input_test.obs_names - -output.write_h5ad(par['output'], compression='gzip') diff --git a/src/methods_image_correction/basic_image_correction/config.vsh.yaml b/src/methods_image_correction/basic_image_correction/config.vsh.yaml new file mode 100644 index 0000000..9f79391 --- /dev/null +++ b/src/methods_image_correction/basic_image_correction/config.vsh.yaml @@ -0,0 +1,34 @@ +__merge__: /src/api/comp_method_image_correction.yaml + +name: basic_image_correction +label: "Percentile background image correction" +summary: "Per-channel percentile background subtraction for multiplexed spatial proteomics images" +description: | + Simple image correction baseline: + - reads image from SpatialData zarr + - converts image to channel-y-x format + - subtracts per-channel percentile background + - clips negative values to zero + - optionally rescales each channel to max 1 + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "spatialdata==0.5.0" + # spatialdata pulls in xarray_schema, which imports pkg_resources + - "setuptools<81" + - "zarr<3.0.0" + - "numpy" + - "pandas" + - "xarray" + - type: native + +runners: + - type: executable diff --git a/src/methods_image_correction/basic_image_correction/script.py b/src/methods_image_correction/basic_image_correction/script.py new file mode 100644 index 0000000..92ae050 --- /dev/null +++ b/src/methods_image_correction/basic_image_correction/script.py @@ -0,0 +1,111 @@ +import json +import numpy as np +import spatialdata as sd + + +def _pick_first_key(mapping, label: str): + keys = list(mapping.keys()) + if not keys: + raise ValueError(f"No {label} found in SpatialData.") + return keys[0] + + +def _to_numpy(obj): + return np.asarray(obj) + + +def _ensure_cyx(img_np: np.ndarray) -> np.ndarray: + if img_np.ndim == 2: + return img_np[None, :, :] + + if img_np.ndim == 3: + s0, s1, s2 = img_np.shape + if s0 <= 256 and s1 > 16 and s2 > 16: + return img_np + if s2 <= 256 and s0 > 16 and s1 > 16: + return np.moveaxis(img_np, -1, 0) + + raise ValueError(f"Unsupported or ambiguous image shape: {img_np.shape}") + + +def _get_channel_names(image_obj, n_ch: int): + for coord_name in ["c", "channel", "channels", "marker", "markers"]: + coords = getattr(image_obj, "coords", None) + if coords is not None and coord_name in coords: + vals = coords[coord_name].values + if len(vals) == n_ch: + return [str(v) for v in vals] + return [f"marker_{i}" for i in range(n_ch)] + + +def _as_bool(x) -> bool: + if isinstance(x, bool): + return x + if x is None: + return False + return str(x).lower() in ["true", "1", "yes", "y"] + + +def run(par: dict): + input_zarr = par["input_zarr"] + image_key = par.get("image_key") + background_percentile = float(par.get("background_percentile", 1.0)) + rescale_max = _as_bool(par.get("rescale_max", False)) + output_image = par["output_image"] + output_markers = par["output_markers"] + + sdata = sd.read_zarr(input_zarr) + + if image_key in [None, "null", ""]: + image_key = _pick_first_key(sdata.images, "images") + + img_obj = sdata.images[image_key] + img_np = _ensure_cyx(_to_numpy(img_obj)).astype(np.float32) + markers = _get_channel_names(img_obj, img_np.shape[0]) + + corrected = np.empty_like(img_np, dtype=np.float32) + + for c in range(img_np.shape[0]): + x = np.nan_to_num(img_np[c].astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0) + finite = x[np.isfinite(x)] + bg = float(np.percentile(finite, background_percentile)) if finite.size else 0.0 + x = x - bg + x[x < 0] = 0 + + if rescale_max: + mx = float(np.max(x)) + if mx > 0: + x = x / mx + + corrected[c] = x + + np.save(output_image, corrected) + + with open(output_markers, "w") as f: + json.dump( + { + "method": "basic_image_correction", + "image_key": image_key, + "markers": markers, + "background_percentile": background_percentile, + "rescale_max": rescale_max, + }, + f, + indent=2, + ) + + +## VIASH START +par = { + "input_zarr": "input.zarr", + "image_key": None, + "background_percentile": 1.0, + "rescale_max": False, + "output_image": "corrected_image.npy", + "output_markers": "markers.json", +} +meta = {"name": "basic_image_correction"} +## VIASH END + +if __name__ == "__main__": + run(par) diff --git a/src/methods_image_correction/flatfield_image_correction/config.vsh.yaml b/src/methods_image_correction/flatfield_image_correction/config.vsh.yaml new file mode 100644 index 0000000..444cd7d --- /dev/null +++ b/src/methods_image_correction/flatfield_image_correction/config.vsh.yaml @@ -0,0 +1,47 @@ +__merge__: /src/api/comp_method_image_correction.yaml + +name: flatfield_image_correction +label: "Flat-field image correction" +summary: "Low-frequency illumination/background correction for multiplexed spatial proteomics images" +description: | + Classical image correction baseline for multiplexed immunofluorescence-like data. + For each channel, the method subtracts a robust percentile background and estimates + a smooth low-frequency illumination field with a Gaussian filter. The low-frequency + field is then either subtracted or used for multiplicative flat-field correction. + +arguments: + - name: --flatfield_sigma + type: double + direction: input + required: false + default: 50.0 + description: Gaussian sigma in pixels used to estimate the smooth illumination field. + - name: --flatfield_mode + type: string + direction: input + required: false + default: subtract + description: Correction mode. One of 'subtract' or 'divide'. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "spatialdata==0.5.0" + # spatialdata pulls in xarray_schema, which imports pkg_resources + - "setuptools<81" + - "zarr<3.0.0" + - "numpy" + - "pandas" + - "xarray" + - "scipy" + - type: native + +runners: + - type: executable diff --git a/src/methods_image_correction/flatfield_image_correction/script.py b/src/methods_image_correction/flatfield_image_correction/script.py new file mode 100644 index 0000000..1cdac98 --- /dev/null +++ b/src/methods_image_correction/flatfield_image_correction/script.py @@ -0,0 +1,163 @@ +import json +import numpy as np +import spatialdata as sd +from scipy.ndimage import gaussian_filter + + +def _pick_first_key(mapping, label: str): + keys = list(mapping.keys()) + if not keys: + raise ValueError(f"No {label} found in SpatialData.") + return keys[0] + + +def _to_numpy(obj): + return np.asarray(obj) + + +def _ensure_cyx(img_np: np.ndarray) -> np.ndarray: + if img_np.ndim == 2: + return img_np[None, :, :] + + if img_np.ndim == 3: + s0, s1, s2 = img_np.shape + if s0 <= 256 and s1 > 16 and s2 > 16: + return img_np + if s2 <= 256 and s0 > 16 and s1 > 16: + return np.moveaxis(img_np, -1, 0) + + raise ValueError(f"Unsupported or ambiguous image shape: {img_np.shape}") + + +def _get_channel_names(image_obj, n_ch: int): + for coord_name in ["c", "channel", "channels", "marker", "markers"]: + coords = getattr(image_obj, "coords", None) + if coords is not None and coord_name in coords: + vals = coords[coord_name].values + if len(vals) == n_ch: + return [str(v) for v in vals] + return [f"marker_{i}" for i in range(n_ch)] + + +def _as_bool(x) -> bool: + if isinstance(x, bool): + return x + if x is None: + return False + return str(x).lower() in ["true", "1", "yes", "y"] + + +def _finite_percentile(x: np.ndarray, q: float) -> float: + finite = x[np.isfinite(x)] + if finite.size == 0: + return 0.0 + return float(np.percentile(finite, q)) + + +def _flatfield_correct_channel( + x: np.ndarray, + background_percentile: float, + flatfield_sigma: float, + flatfield_mode: str, +) -> np.ndarray: + x = np.nan_to_num(x.astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0) + + # Robust background subtraction before estimating the illumination field. + bg = _finite_percentile(x, background_percentile) + x0 = x - bg + x0[x0 < 0] = 0 + + # Smooth low-frequency field. This is a classical baseline, not a learned model. + field = gaussian_filter(x0, sigma=float(flatfield_sigma)) + field = np.nan_to_num(field.astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0) + + mode = str(flatfield_mode).lower() + if mode == "subtract": + # Remove low-frequency spatial bias while preserving the global intensity scale. + baseline = float(np.median(field[field > 0])) if np.any(field > 0) else 0.0 + corrected = x0 - (field - baseline) + elif mode == "divide": + # Multiplicative flat-field correction. The median rescales values to the + # original approximate intensity scale. + positive = field[field > 0] + baseline = float(np.median(positive)) if positive.size else 1.0 + eps = max(baseline * 1e-6, 1e-6) + corrected = x0 / np.maximum(field, eps) * baseline + else: + raise ValueError("flatfield_mode must be either 'subtract' or 'divide'.") + + corrected = np.nan_to_num(corrected, nan=0.0, posinf=0.0, neginf=0.0) + corrected[corrected < 0] = 0 + return corrected.astype(np.float32) + + +def run(par: dict): + input_zarr = par["input_zarr"] + image_key = par.get("image_key") + background_percentile = float(par.get("background_percentile", 1.0)) + rescale_max = _as_bool(par.get("rescale_max", False)) + flatfield_sigma = float(par.get("flatfield_sigma", 50.0)) + flatfield_mode = str(par.get("flatfield_mode", "subtract")) + output_image = par["output_image"] + output_markers = par["output_markers"] + + sdata = sd.read_zarr(input_zarr) + + if image_key in [None, "null", ""]: + image_key = _pick_first_key(sdata.images, "images") + + img_obj = sdata.images[image_key] + img_np = _ensure_cyx(_to_numpy(img_obj)).astype(np.float32) + markers = _get_channel_names(img_obj, img_np.shape[0]) + + corrected = np.empty_like(img_np, dtype=np.float32) + + for c in range(img_np.shape[0]): + x = _flatfield_correct_channel( + img_np[c], + background_percentile=background_percentile, + flatfield_sigma=flatfield_sigma, + flatfield_mode=flatfield_mode, + ) + + if rescale_max: + mx = float(np.max(x)) + if mx > 0: + x = x / mx + + corrected[c] = x + + np.save(output_image, corrected) + + with open(output_markers, "w") as f: + json.dump( + { + "method": "flatfield_image_correction", + "image_key": image_key, + "markers": markers, + "background_percentile": background_percentile, + "flatfield_sigma": flatfield_sigma, + "flatfield_mode": flatfield_mode, + "rescale_max": rescale_max, + }, + f, + indent=2, + ) + + +## VIASH START +par = { + "input_zarr": "input.zarr", + "image_key": None, + "background_percentile": 1.0, + "flatfield_sigma": 50.0, + "flatfield_mode": "subtract", + "rescale_max": False, + "output_image": "corrected_image.npy", + "output_markers": "markers.json", +} +meta = {"name": "flatfield_image_correction"} +## VIASH END + +if __name__ == "__main__": + run(par) diff --git a/src/methods_image_correction/no_image_correction/config.vsh.yaml b/src/methods_image_correction/no_image_correction/config.vsh.yaml new file mode 100644 index 0000000..5752f48 --- /dev/null +++ b/src/methods_image_correction/no_image_correction/config.vsh.yaml @@ -0,0 +1,32 @@ +__merge__: /src/api/comp_method_image_correction.yaml + +name: no_image_correction +label: "No image correction" +summary: "Identity image correction baseline for multiplexed spatial proteomics images" +description: | + Control method for image correction benchmarking. The component reads a multichannel + image from a SpatialData zarr store, converts it to channel-y-x format, optionally + rescales each channel to max 1, and writes the image without background or + illumination correction. This method is useful as an uncorrected baseline. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "spatialdata==0.5.0" + # spatialdata pulls in xarray_schema, which imports pkg_resources + - "setuptools<81" + - "zarr<3.0.0" + - "numpy" + - "pandas" + - "xarray" + - type: native + +runners: + - type: executable diff --git a/src/methods_image_correction/no_image_correction/script.py b/src/methods_image_correction/no_image_correction/script.py new file mode 100644 index 0000000..34ba93a --- /dev/null +++ b/src/methods_image_correction/no_image_correction/script.py @@ -0,0 +1,102 @@ +import json +import numpy as np +import spatialdata as sd + + +def _pick_first_key(mapping, label: str): + keys = list(mapping.keys()) + if not keys: + raise ValueError(f"No {label} found in SpatialData.") + return keys[0] + + +def _to_numpy(obj): + return np.asarray(obj) + + +def _ensure_cyx(img_np: np.ndarray) -> np.ndarray: + if img_np.ndim == 2: + return img_np[None, :, :] + + if img_np.ndim == 3: + s0, s1, s2 = img_np.shape + if s0 <= 256 and s1 > 16 and s2 > 16: + return img_np + if s2 <= 256 and s0 > 16 and s1 > 16: + return np.moveaxis(img_np, -1, 0) + + raise ValueError(f"Unsupported or ambiguous image shape: {img_np.shape}") + + +def _get_channel_names(image_obj, n_ch: int): + for coord_name in ["c", "channel", "channels", "marker", "markers"]: + coords = getattr(image_obj, "coords", None) + if coords is not None and coord_name in coords: + vals = coords[coord_name].values + if len(vals) == n_ch: + return [str(v) for v in vals] + return [f"marker_{i}" for i in range(n_ch)] + + +def _as_bool(x) -> bool: + if isinstance(x, bool): + return x + if x is None: + return False + return str(x).lower() in ["true", "1", "yes", "y"] + + +def run(par: dict): + input_zarr = par["input_zarr"] + image_key = par.get("image_key") + rescale_max = _as_bool(par.get("rescale_max", False)) + output_image = par["output_image"] + output_markers = par["output_markers"] + + sdata = sd.read_zarr(input_zarr) + + if image_key in [None, "null", ""]: + image_key = _pick_first_key(sdata.images, "images") + + img_obj = sdata.images[image_key] + img_np = _ensure_cyx(_to_numpy(img_obj)).astype(np.float32) + markers = _get_channel_names(img_obj, img_np.shape[0]) + + corrected = np.nan_to_num(img_np, nan=0.0, posinf=0.0, neginf=0.0).astype(np.float32) + corrected[corrected < 0] = 0 + + if rescale_max: + for c in range(corrected.shape[0]): + mx = float(np.max(corrected[c])) + if mx > 0: + corrected[c] = corrected[c] / mx + + np.save(output_image, corrected) + + with open(output_markers, "w") as f: + json.dump( + { + "method": "no_image_correction", + "image_key": image_key, + "markers": markers, + "background_percentile": None, + "rescale_max": rescale_max, + }, + f, + indent=2, + ) + + +## VIASH START +par = { + "input_zarr": "input.zarr", + "image_key": None, + "rescale_max": False, + "output_image": "corrected_image.npy", + "output_markers": "markers.json", +} +meta = {"name": "no_image_correction"} +## VIASH END + +if __name__ == "__main__": + run(par) diff --git a/src/methods_protein_aggregation/mean_intensity_aggregation/config.vsh.yaml b/src/methods_protein_aggregation/mean_intensity_aggregation/config.vsh.yaml new file mode 100644 index 0000000..1289519 --- /dev/null +++ b/src/methods_protein_aggregation/mean_intensity_aggregation/config.vsh.yaml @@ -0,0 +1,31 @@ +__merge__: /src/api/comp_method_protein_aggregation_arrays.yaml + +name: mean_intensity_aggregation +namespace: methods_protein_aggregation + +info: + label: "mean_intensity_aggregation" + summary: "Aggregate per-cell marker intensities using the mean pixel intensity." + description: | + Benchmark method for converting corrected multichannel spatial proteomics images and segmentation masks into + cell-by-protein intensity matrices. This component implements the mean_intensity_aggregation strategy. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - numpy + - pandas + - anndata + - scikit-image + + - type: native + +runners: + - type: executable diff --git a/src/methods_protein_aggregation/mean_intensity_aggregation/script.py b/src/methods_protein_aggregation/mean_intensity_aggregation/script.py new file mode 100644 index 0000000..22add60 --- /dev/null +++ b/src/methods_protein_aggregation/mean_intensity_aggregation/script.py @@ -0,0 +1,97 @@ +import json +from pathlib import Path + +import anndata as ad +import numpy as np +import pandas as pd +from skimage.measure import regionprops_table + +## VIASH START +par = { + "input_image": "corrected_image.npy", + "input_labels": "segmentation_labels.npy", + "input_markers": "corrected_markers.json", + "output": "cell_by_protein.h5ad", +} +## VIASH END + + +def _load_markers(path, n_channels): + with open(path) as f: + data = json.load(f) + if isinstance(data, dict): + markers = data.get("markers") or data.get("channels") or data.get("marker_names") + else: + markers = data + if markers is None or len(markers) != n_channels: + markers = [f"marker_{i}" for i in range(n_channels)] + return [str(x) for x in markers] + + +def _ensure_cyx(image): + image = np.asarray(image) + if image.ndim != 3: + raise ValueError(f"Expected 3D image array, got shape {image.shape}.") + return image + + +def _obs_from_labels(labels): + props = regionprops_table( + labels.astype(np.int64), + properties=("label", "area", "centroid"), + ) + obs = pd.DataFrame(props) + obs = obs.rename(columns={"centroid-0": "centroid_y", "centroid-1": "centroid_x"}) + obs["cell_id"] = obs["label"].astype(str) + obs.index = obs["cell_id"].astype(str) + return obs + + +def _aggregate_mean(image, labels, label_ids): + flat_labels = labels.ravel().astype(np.int64) + valid = flat_labels > 0 + labs = flat_labels[valid] + areas = np.bincount(labs, minlength=int(flat_labels.max()) + 1).astype(float) + + values = [] + for c in range(image.shape[0]): + vals = image[c].ravel()[valid].astype(float) + sums = np.bincount(labs, weights=vals, minlength=len(areas)) + means = sums[label_ids] / np.maximum(areas[label_ids], 1.0) + values.append(means) + return np.vstack(values).T + + +def run(par): + image = _ensure_cyx(np.load(par["input_image"])) + labels = np.asarray(np.load(par["input_labels"])).astype(np.int64) + + if image.shape[1:] != labels.shape: + raise ValueError(f"Image spatial shape {image.shape[1:]} does not match labels shape {labels.shape}.") + + markers = _load_markers(par["input_markers"], image.shape[0]) + obs = _obs_from_labels(labels) + label_ids = obs["label"].astype(int).to_numpy() + X = _aggregate_mean(image, labels, label_ids) + + adata = ad.AnnData( + X=X.astype(np.float32), + obs=obs.drop(columns=["label"]), + var=pd.DataFrame(index=markers), + ) + adata.layers["counts"] = adata.X.copy() + adata.uns["aggregation"] = { + "method": "mean_intensity_aggregation", + "statistic": "mean", + "input_image": str(par["input_image"]), + "input_labels": str(par["input_labels"]), + "n_labels": int(len(obs)), + "n_markers": int(len(markers)), + } + + Path(par["output"]).parent.mkdir(parents=True, exist_ok=True) + adata.write_h5ad(par["output"]) + + +if __name__ == "__main__": + run(par) diff --git a/src/methods_protein_aggregation/median_intensity_aggregation/config.vsh.yaml b/src/methods_protein_aggregation/median_intensity_aggregation/config.vsh.yaml new file mode 100644 index 0000000..625b9f7 --- /dev/null +++ b/src/methods_protein_aggregation/median_intensity_aggregation/config.vsh.yaml @@ -0,0 +1,31 @@ +__merge__: /src/api/comp_method_protein_aggregation_arrays.yaml + +name: median_intensity_aggregation +namespace: methods_protein_aggregation + +info: + label: "median_intensity_aggregation" + summary: "Aggregate per-cell marker intensities using the median pixel intensity." + description: | + Benchmark method for converting corrected multichannel spatial proteomics images and segmentation masks into + cell-by-protein intensity matrices. This component implements the median_intensity_aggregation strategy. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - numpy + - pandas + - anndata + - scikit-image + + - type: native + +runners: + - type: executable diff --git a/src/methods_protein_aggregation/median_intensity_aggregation/script.py b/src/methods_protein_aggregation/median_intensity_aggregation/script.py new file mode 100644 index 0000000..ba3cf35 --- /dev/null +++ b/src/methods_protein_aggregation/median_intensity_aggregation/script.py @@ -0,0 +1,103 @@ +import json +from pathlib import Path + +import anndata as ad +import numpy as np +import pandas as pd +from skimage.measure import regionprops_table + +## VIASH START +par = { + "input_image": "corrected_image.npy", + "input_labels": "segmentation_labels.npy", + "input_markers": "corrected_markers.json", + "output": "cell_by_protein.h5ad", +} +## VIASH END + + +def _load_markers(path, n_channels): + with open(path) as f: + data = json.load(f) + if isinstance(data, dict): + markers = data.get("markers") or data.get("channels") or data.get("marker_names") + else: + markers = data + if markers is None or len(markers) != n_channels: + markers = [f"marker_{i}" for i in range(n_channels)] + return [str(x) for x in markers] + + +def _ensure_cyx(image): + image = np.asarray(image) + if image.ndim != 3: + raise ValueError(f"Expected 3D image array, got shape {image.shape}.") + return image + + +def _obs_from_labels(labels): + props = regionprops_table( + labels.astype(np.int64), + properties=("label", "area", "centroid"), + ) + obs = pd.DataFrame(props) + obs = obs.rename(columns={"centroid-0": "centroid_y", "centroid-1": "centroid_x"}) + obs["cell_id"] = obs["label"].astype(str) + obs.index = obs["cell_id"].astype(str) + return obs + + +def _aggregate_median(image, labels, label_ids): + flat_labels = labels.ravel().astype(np.int64) + valid = flat_labels > 0 + labs = flat_labels[valid] + + order = np.argsort(labs, kind="mergesort") + labs_sorted = labs[order] + unique_labels, starts, counts = np.unique(labs_sorted, return_index=True, return_counts=True) + + label_to_row = {int(label): i for i, label in enumerate(label_ids)} + X = np.zeros((len(label_ids), image.shape[0]), dtype=np.float32) + + for c in range(image.shape[0]): + vals_sorted = image[c].ravel()[valid][order].astype(float) + for label, start, count in zip(unique_labels, starts, counts): + row = label_to_row.get(int(label)) + if row is not None: + X[row, c] = np.median(vals_sorted[start:start + count]) + return X + + +def run(par): + image = _ensure_cyx(np.load(par["input_image"])) + labels = np.asarray(np.load(par["input_labels"])).astype(np.int64) + + if image.shape[1:] != labels.shape: + raise ValueError(f"Image spatial shape {image.shape[1:]} does not match labels shape {labels.shape}.") + + markers = _load_markers(par["input_markers"], image.shape[0]) + obs = _obs_from_labels(labels) + label_ids = obs["label"].astype(int).to_numpy() + X = _aggregate_median(image, labels, label_ids) + + adata = ad.AnnData( + X=X.astype(np.float32), + obs=obs.drop(columns=["label"]), + var=pd.DataFrame(index=markers), + ) + adata.layers["counts"] = adata.X.copy() + adata.uns["aggregation"] = { + "method": "median_intensity_aggregation", + "statistic": "median", + "input_image": str(par["input_image"]), + "input_labels": str(par["input_labels"]), + "n_labels": int(len(obs)), + "n_markers": int(len(markers)), + } + + Path(par["output"]).parent.mkdir(parents=True, exist_ok=True) + adata.write_h5ad(par["output"]) + + +if __name__ == "__main__": + run(par) diff --git a/src/methods_protein_aggregation/sum_intensity_aggregation/config.vsh.yaml b/src/methods_protein_aggregation/sum_intensity_aggregation/config.vsh.yaml new file mode 100644 index 0000000..2ccc624 --- /dev/null +++ b/src/methods_protein_aggregation/sum_intensity_aggregation/config.vsh.yaml @@ -0,0 +1,31 @@ +__merge__: /src/api/comp_method_protein_aggregation_arrays.yaml + +name: sum_intensity_aggregation +namespace: methods_protein_aggregation + +info: + label: "sum_intensity_aggregation" + summary: "Aggregate per-cell marker intensities using the summed pixel intensity." + description: | + Benchmark method for converting corrected multichannel spatial proteomics images and segmentation masks into + cell-by-protein intensity matrices. This component implements the sum_intensity_aggregation strategy. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - numpy + - pandas + - anndata + - scikit-image + + - type: native + +runners: + - type: executable diff --git a/src/methods_protein_aggregation/sum_intensity_aggregation/script.py b/src/methods_protein_aggregation/sum_intensity_aggregation/script.py new file mode 100644 index 0000000..16091fd --- /dev/null +++ b/src/methods_protein_aggregation/sum_intensity_aggregation/script.py @@ -0,0 +1,96 @@ +import json +from pathlib import Path + +import anndata as ad +import numpy as np +import pandas as pd +from skimage.measure import regionprops_table + +## VIASH START +par = { + "input_image": "corrected_image.npy", + "input_labels": "segmentation_labels.npy", + "input_markers": "corrected_markers.json", + "output": "cell_by_protein.h5ad", +} +## VIASH END + + +def _load_markers(path, n_channels): + with open(path) as f: + data = json.load(f) + if isinstance(data, dict): + markers = data.get("markers") or data.get("channels") or data.get("marker_names") + else: + markers = data + if markers is None or len(markers) != n_channels: + markers = [f"marker_{i}" for i in range(n_channels)] + return [str(x) for x in markers] + + +def _ensure_cyx(image): + image = np.asarray(image) + if image.ndim != 3: + raise ValueError(f"Expected 3D image array, got shape {image.shape}.") + return image + + +def _obs_from_labels(labels): + props = regionprops_table( + labels.astype(np.int64), + properties=("label", "area", "centroid"), + ) + obs = pd.DataFrame(props) + obs = obs.rename(columns={"centroid-0": "centroid_y", "centroid-1": "centroid_x"}) + obs["cell_id"] = obs["label"].astype(str) + obs.index = obs["cell_id"].astype(str) + return obs + + +def _aggregate_sum(image, labels, label_ids): + flat_labels = labels.ravel().astype(np.int64) + valid = flat_labels > 0 + labs = flat_labels[valid] + max_label = int(flat_labels.max()) + + values = [] + for c in range(image.shape[0]): + vals = image[c].ravel()[valid].astype(float) + sums = np.bincount(labs, weights=vals, minlength=max_label + 1) + values.append(sums[label_ids]) + return np.vstack(values).T + + +def run(par): + image = _ensure_cyx(np.load(par["input_image"])) + labels = np.asarray(np.load(par["input_labels"])).astype(np.int64) + + if image.shape[1:] != labels.shape: + raise ValueError(f"Image spatial shape {image.shape[1:]} does not match labels shape {labels.shape}.") + + markers = _load_markers(par["input_markers"], image.shape[0]) + obs = _obs_from_labels(labels) + label_ids = obs["label"].astype(int).to_numpy() + X = _aggregate_sum(image, labels, label_ids) + + adata = ad.AnnData( + X=X.astype(np.float32), + obs=obs.drop(columns=["label"]), + var=pd.DataFrame(index=markers), + ) + adata.layers["counts"] = adata.X.copy() + adata.uns["aggregation"] = { + "method": "sum_intensity_aggregation", + "statistic": "sum", + "input_image": str(par["input_image"]), + "input_labels": str(par["input_labels"]), + "n_labels": int(len(obs)), + "n_markers": int(len(markers)), + } + + Path(par["output"]).parent.mkdir(parents=True, exist_ok=True) + adata.write_h5ad(par["output"]) + + +if __name__ == "__main__": + run(par) diff --git a/src/methods_proteomics_normalization/arcsinh_normalization/config.vsh.yaml b/src/methods_proteomics_normalization/arcsinh_normalization/config.vsh.yaml new file mode 100644 index 0000000..6d9e5b6 --- /dev/null +++ b/src/methods_proteomics_normalization/arcsinh_normalization/config.vsh.yaml @@ -0,0 +1,28 @@ +__merge__: /src/api/comp_method_proteomics_normalization.yaml + +name: arcsinh_normalization +label: "Arcsinh normalization" +summary: "Apply cytometry-style arcsinh transformation to raw intensities" +description: | + Cytometry-style intensity transformation for spatial proteomics. Raw marker + intensities are transformed using arcsinh(x / cofactor). This stabilizes + high-intensity values without enforcing equal total intensity per cell. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "anndata>=0.10.0" + - "numpy" + - "scipy" + - "zarr<3.0.0" + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_normalization/arcsinh_normalization/script.py b/src/methods_proteomics_normalization/arcsinh_normalization/script.py new file mode 100644 index 0000000..69482e3 --- /dev/null +++ b/src/methods_proteomics_normalization/arcsinh_normalization/script.py @@ -0,0 +1,73 @@ +import numpy as np +import anndata as ad + +try: + import scipy.sparse as sp +except Exception: # pragma: no cover + sp = None + + +def _dense(x) -> np.ndarray: + if sp is not None and sp.issparse(x): + return x.toarray() + return np.asarray(x) + + +def _get_counts_matrix(adata: ad.AnnData) -> np.ndarray: + if "counts" in adata.layers: + return _dense(adata.layers["counts"]) + return _dense(adata.X) + + +def run(par: dict): + in_path = par["input"] + out_path = par["output"] + cofactor = float(par.get("cofactor", 5.0)) + + if cofactor <= 0: + raise ValueError("--cofactor must be > 0") + + adata = ad.read_h5ad(in_path) + X = _get_counts_matrix(adata).astype(np.float32) + + Xn = np.arcsinh(X / cofactor).astype(np.float32) + + adata.layers["normalized"] = Xn + adata.uns.setdefault("normalization", {}) + adata.uns["normalization"].update( + { + "method": "arcsinh_normalization", + "description": "Cytometry-style arcsinh(x / cofactor) transformation.", + "scale": None, + "cofactor": cofactor, + } + ) + + adata.write_h5ad(out_path, compression="gzip") + + +## VIASH START +par = { + "input": "input.h5ad", + "scale": 10000.0, + "cofactor": 5.0, + "transform": "log1p", + "output": "output.h5ad", +} +meta = {"name": "arcsinh_normalization"} +## VIASH END + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + import argparse + + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--scale", type=float, default=10000.0) + ap.add_argument("--cofactor", type=float, default=5.0) + ap.add_argument("--transform", default="log1p") + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) diff --git a/src/methods_proteomics_normalization/no_normalization/config.vsh.yaml b/src/methods_proteomics_normalization/no_normalization/config.vsh.yaml new file mode 100644 index 0000000..40f6b46 --- /dev/null +++ b/src/methods_proteomics_normalization/no_normalization/config.vsh.yaml @@ -0,0 +1,27 @@ +__merge__: /src/api/comp_method_proteomics_normalization.yaml + +name: no_normalization +label: "No normalization" +summary: "Control method that copies raw intensities into layers['normalized']" +description: | + Baseline normalization method for benchmarking. It leaves the cell-by-protein + intensity matrix unchanged and stores the raw intensities as layers['normalized']. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "anndata>=0.10.0" + - "numpy" + - "scipy" + - "zarr<3.0.0" + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_normalization/no_normalization/script.py b/src/methods_proteomics_normalization/no_normalization/script.py new file mode 100644 index 0000000..f97149b --- /dev/null +++ b/src/methods_proteomics_normalization/no_normalization/script.py @@ -0,0 +1,67 @@ +import numpy as np +import anndata as ad + +try: + import scipy.sparse as sp +except Exception: # pragma: no cover + sp = None + + +def _dense(x) -> np.ndarray: + if sp is not None and sp.issparse(x): + return x.toarray() + return np.asarray(x) + + +def _get_counts_matrix(adata: ad.AnnData) -> np.ndarray: + if "counts" in adata.layers: + return _dense(adata.layers["counts"]) + return _dense(adata.X) + + +def run(par: dict): + in_path = par["input"] + out_path = par["output"] + + adata = ad.read_h5ad(in_path) + X = _get_counts_matrix(adata).astype(np.float32) + + adata.layers["normalized"] = X.copy() + adata.uns.setdefault("normalization", {}) + adata.uns["normalization"].update( + { + "method": "no_normalization", + "description": "Raw intensities copied to layers['normalized'] without transformation.", + "scale": None, + "cofactor": None, + } + ) + + adata.write_h5ad(out_path, compression="gzip") + + +## VIASH START +par = { + "input": "input.h5ad", + "scale": 10000.0, + "cofactor": 5.0, + "transform": "log1p", + "output": "output.h5ad", +} +meta = {"name": "no_normalization"} +## VIASH END + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + import argparse + + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--scale", type=float, default=10000.0) + ap.add_argument("--cofactor", type=float, default=5.0) + ap.add_argument("--transform", default="log1p") + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) diff --git a/src/methods_proteomics_normalization/total_log1p_normalization/config.vsh.yaml b/src/methods_proteomics_normalization/total_log1p_normalization/config.vsh.yaml new file mode 100644 index 0000000..b7e818f --- /dev/null +++ b/src/methods_proteomics_normalization/total_log1p_normalization/config.vsh.yaml @@ -0,0 +1,28 @@ +__merge__: /src/api/comp_method_proteomics_normalization.yaml + +name: total_log1p_normalization +label: "Total-intensity log1p normalization" +summary: "Normalize each cell to a fixed total intensity and apply log1p" +description: | + Total-intensity normalization for spatial proteomics. For each cell, raw + marker intensities are divided by the total cell intensity, multiplied by a + target scale factor, and transformed using log1p. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "anndata>=0.10.0" + - "numpy" + - "scipy" + - "zarr<3.0.0" + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_normalization/total_log1p_normalization/script.py b/src/methods_proteomics_normalization/total_log1p_normalization/script.py new file mode 100644 index 0000000..a572041 --- /dev/null +++ b/src/methods_proteomics_normalization/total_log1p_normalization/script.py @@ -0,0 +1,77 @@ +import numpy as np +import anndata as ad + +try: + import scipy.sparse as sp +except Exception: # pragma: no cover + sp = None + + +def _dense(x) -> np.ndarray: + if sp is not None and sp.issparse(x): + return x.toarray() + return np.asarray(x) + + +def _get_counts_matrix(adata: ad.AnnData) -> np.ndarray: + if "counts" in adata.layers: + return _dense(adata.layers["counts"]) + return _dense(adata.X) + + +def run(par: dict): + in_path = par["input"] + out_path = par["output"] + scale = float(par.get("scale", 10000.0)) + + adata = ad.read_h5ad(in_path) + X = _get_counts_matrix(adata).astype(np.float32) + + totals = X.sum(axis=1, keepdims=True).astype(np.float32) + zero_total = totals.squeeze() <= 0 + totals[totals <= 0] = 1.0 + + Xn = (X / totals) * scale + Xn = np.log1p(Xn).astype(np.float32) + + adata.layers["normalized"] = Xn + adata.obs["normalization_total_intensity"] = X.sum(axis=1).astype(np.float32) + adata.obs["normalization_zero_total"] = zero_total.astype(bool) + adata.uns.setdefault("normalization", {}) + adata.uns["normalization"].update( + { + "method": "total_log1p_normalization", + "description": "Per-cell total-intensity scaling followed by log1p.", + "scale": scale, + "cofactor": None, + } + ) + + adata.write_h5ad(out_path, compression="gzip") + + +## VIASH START +par = { + "input": "input.h5ad", + "scale": 10000.0, + "cofactor": 5.0, + "transform": "log1p", + "output": "output.h5ad", +} +meta = {"name": "total_log1p_normalization"} +## VIASH END + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + import argparse + + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--scale", type=float, default=10000.0) + ap.add_argument("--cofactor", type=float, default=5.0) + ap.add_argument("--transform", default="log1p") + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) diff --git a/src/methods_proteomics_qc/adaptive_quantile_qc_filter/config.vsh.yaml b/src/methods_proteomics_qc/adaptive_quantile_qc_filter/config.vsh.yaml new file mode 100644 index 0000000..b8fcb37 --- /dev/null +++ b/src/methods_proteomics_qc/adaptive_quantile_qc_filter/config.vsh.yaml @@ -0,0 +1,31 @@ +__merge__: /src/api/comp_method_proteomics_qc.yaml + +name: adaptive_quantile_qc_filter +namespace: methods_proteomics_qc + +info: + label: "adaptive_quantile_qc_filter" + summary: "Data-adaptive QC based on lower quantiles of area and total intensity." + description: | + Benchmark QC method that estimates dataset-specific lower thresholds from the cell area and total protein + intensity distributions. This provides a data-adaptive alternative to fixed absolute thresholds. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "anndata>=0.10.0" + - numpy + - pandas + - scipy + + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_qc/adaptive_quantile_qc_filter/script.py b/src/methods_proteomics_qc/adaptive_quantile_qc_filter/script.py new file mode 100644 index 0000000..678b164 --- /dev/null +++ b/src/methods_proteomics_qc/adaptive_quantile_qc_filter/script.py @@ -0,0 +1,107 @@ +import anndata as ad +import numpy as np +import pandas as pd + + +def _get_counts_matrix(adata: ad.AnnData): + return adata.layers["counts"] if "counts" in adata.layers else adata.X + + +def _get_area(adata: ad.AnnData): + for col in ["area", "cell_area", "volume"]: + if col in adata.obs.columns: + return pd.to_numeric(adata.obs[col], errors="coerce").astype(float), col + return pd.Series(np.nan, index=adata.obs_names, dtype=float), None + + +def _total_intensity(adata: ad.AnnData): + X = _get_counts_matrix(adata) + return pd.Series(np.asarray(X.sum(axis=1)).ravel().astype(float), index=adata.obs_names) + + +def _safe_quantile(values, q): + arr = np.asarray(values, dtype=float) + arr = arr[np.isfinite(arr)] + if arr.size == 0: + return float("nan") + return float(np.quantile(arr, q)) + + +def _failure_reason(area_ok: pd.Series, total_ok: pd.Series): + reasons = [] + for a_ok, t_ok in zip(area_ok.values, total_ok.values): + if bool(a_ok) and bool(t_ok): + reasons.append("pass") + elif (not bool(a_ok)) and (not bool(t_ok)): + reasons.append("low_area_quantile_and_low_total_quantile") + elif not bool(a_ok): + reasons.append("low_area_quantile") + else: + reasons.append("low_total_quantile") + return reasons + + +def run(par: dict): + adata = ad.read_h5ad(par["input"]) + out_path = par["output"] + area_quantile = float(par.get("area_quantile", 0.05)) + total_quantile = float(par.get("total_quantile", 0.05)) + + if not (0 <= area_quantile < 1): + raise ValueError("area_quantile must be in [0, 1).") + if not (0 <= total_quantile < 1): + raise ValueError("total_quantile must be in [0, 1).") + + area, area_col = _get_area(adata) + total = _total_intensity(adata) + + if area_col is None: + area_threshold = float("nan") + area_ok = pd.Series(True, index=adata.obs_names) + else: + area_threshold = _safe_quantile(area.values, area_quantile) + area_ok = (area >= area_threshold).fillna(False) + + total_threshold = _safe_quantile(total.values, total_quantile) + total_ok = (total >= total_threshold).fillna(False) + passed = (area_ok & total_ok).astype(bool) + + adata.obs["passed_QC"] = passed.values + adata.obs["total_intensity"] = total.values + if area_col is None: + adata.obs["area"] = area.values + adata.obs["qc_area_ok"] = area_ok.values.astype(bool) + adata.obs["qc_total_ok"] = total_ok.values.astype(bool) + adata.obs["qc_filter_reason"] = _failure_reason(area_ok, total_ok) + + adata.uns["qc"] = { + "method": "adaptive_quantile_qc_filter", + "description": "Dataset-specific lower quantile thresholds on area and total protein intensity.", + "area_quantile": area_quantile, + "total_quantile": total_quantile, + "area_threshold": area_threshold, + "total_threshold": total_threshold, + "area_column_used": area_col if area_col is not None else "none", + "n_cells": int(adata.n_obs), + "n_passed": int(passed.sum()), + "pass_rate": float(passed.mean()) if adata.n_obs > 0 else float("nan"), + "median_total_intensity": float(np.nanmedian(total.values)) if adata.n_obs > 0 else float("nan"), + "median_area": float(np.nanmedian(area.values)) if adata.n_obs > 0 else float("nan"), + } + + adata.write_h5ad(out_path, compression="gzip") + + +if __name__ == "__main__": + ## VIASH START + par = { + "input": None, + "min_area": 50, + "min_total": 0.0, + "area_quantile": 0.05, + "total_quantile": 0.05, + "output": None, + } + meta = {} + ## VIASH END + run(par) diff --git a/src/methods_proteomics_qc/basic_qc_filter/config.vsh.yaml b/src/methods_proteomics_qc/basic_qc_filter/config.vsh.yaml new file mode 100644 index 0000000..4618ea9 --- /dev/null +++ b/src/methods_proteomics_qc/basic_qc_filter/config.vsh.yaml @@ -0,0 +1,31 @@ +__merge__: /src/api/comp_method_proteomics_qc.yaml + +name: basic_qc_filter +namespace: methods_proteomics_qc + +info: + label: "basic_qc_filter" + summary: "Fixed-threshold QC based on cell area and total protein intensity." + description: | + Benchmark QC method that marks cells as passing when they satisfy fixed thresholds for segmented cell area + and total protein intensity. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "anndata>=0.10.0" + - numpy + - pandas + - scipy + + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_qc/basic_qc_filter/script.py b/src/methods_proteomics_qc/basic_qc_filter/script.py new file mode 100644 index 0000000..bc7cb84 --- /dev/null +++ b/src/methods_proteomics_qc/basic_qc_filter/script.py @@ -0,0 +1,88 @@ +import anndata as ad +import numpy as np +import pandas as pd + + +def _get_counts_matrix(adata: ad.AnnData): + return adata.layers["counts"] if "counts" in adata.layers else adata.X + + +def _get_area(adata: ad.AnnData): + for col in ["area", "cell_area", "volume"]: + if col in adata.obs.columns: + return pd.to_numeric(adata.obs[col], errors="coerce").astype(float), col + return pd.Series(np.nan, index=adata.obs_names, dtype=float), None + + +def _total_intensity(adata: ad.AnnData): + X = _get_counts_matrix(adata) + return pd.Series(np.asarray(X.sum(axis=1)).ravel().astype(float), index=adata.obs_names) + + +def _failure_reason(area_ok: pd.Series, total_ok: pd.Series): + reasons = [] + for a_ok, t_ok in zip(area_ok.values, total_ok.values): + if bool(a_ok) and bool(t_ok): + reasons.append("pass") + elif (not bool(a_ok)) and (not bool(t_ok)): + reasons.append("low_area_and_low_total") + elif not bool(a_ok): + reasons.append("low_area") + else: + reasons.append("low_total") + return reasons + + +def run(par: dict): + adata = ad.read_h5ad(par["input"]) + out_path = par["output"] + min_area = float(par.get("min_area", 50)) + min_total = float(par.get("min_total", 0.0)) + + area, area_col = _get_area(adata) + total = _total_intensity(adata) + + if area_col is None: + area_ok = pd.Series(True, index=adata.obs_names) + else: + area_ok = (area >= min_area).fillna(False) + total_ok = (total >= min_total).fillna(False) + passed = (area_ok & total_ok).astype(bool) + + adata.obs["passed_QC"] = passed.values + adata.obs["total_intensity"] = total.values + if area_col is None: + adata.obs["area"] = area.values + adata.obs["qc_area_ok"] = area_ok.values.astype(bool) + adata.obs["qc_total_ok"] = total_ok.values.astype(bool) + adata.obs["qc_filter_reason"] = _failure_reason(area_ok, total_ok) + + adata.uns["qc"] = { + "method": "basic_qc_filter", + "description": "Fixed thresholds on cell area and total protein intensity.", + "min_area": min_area, + "min_total": min_total, + "area_column_used": area_col if area_col is not None else "none", + "n_cells": int(adata.n_obs), + "n_passed": int(passed.sum()), + "pass_rate": float(passed.mean()) if adata.n_obs > 0 else float("nan"), + "median_total_intensity": float(np.nanmedian(total.values)) if adata.n_obs > 0 else float("nan"), + "median_area": float(np.nanmedian(area.values)) if adata.n_obs > 0 else float("nan"), + } + + adata.write_h5ad(out_path, compression="gzip") + + +if __name__ == "__main__": + ## VIASH START + par = { + "input": None, + "min_area": 50, + "min_total": 0.0, + "area_quantile": 0.05, + "total_quantile": 0.05, + "output": None, + } + meta = {} + ## VIASH END + run(par) diff --git a/src/methods_proteomics_qc/no_qc_filter/config.vsh.yaml b/src/methods_proteomics_qc/no_qc_filter/config.vsh.yaml new file mode 100644 index 0000000..91ef1c1 --- /dev/null +++ b/src/methods_proteomics_qc/no_qc_filter/config.vsh.yaml @@ -0,0 +1,31 @@ +__merge__: /src/api/comp_method_proteomics_qc.yaml + +name: no_qc_filter +namespace: methods_proteomics_qc + +info: + label: "no_qc_filter" + summary: "Control QC method that marks all cells as passing QC." + description: | + Baseline/control method for QC benchmarking. It computes total intensity and preserves morphology metadata, + but sets obs['passed_QC'] to True for every segmented object. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "anndata>=0.10.0" + - numpy + - pandas + - scipy + + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_qc/no_qc_filter/script.py b/src/methods_proteomics_qc/no_qc_filter/script.py new file mode 100644 index 0000000..6de4a53 --- /dev/null +++ b/src/methods_proteomics_qc/no_qc_filter/script.py @@ -0,0 +1,64 @@ +import anndata as ad +import numpy as np +import pandas as pd + + +def _get_counts_matrix(adata: ad.AnnData): + return adata.layers["counts"] if "counts" in adata.layers else adata.X + + +def _get_area(adata: ad.AnnData): + for col in ["area", "cell_area", "volume"]: + if col in adata.obs.columns: + return pd.to_numeric(adata.obs[col], errors="coerce").astype(float), col + return pd.Series(np.nan, index=adata.obs_names, dtype=float), None + + +def _total_intensity(adata: ad.AnnData): + X = _get_counts_matrix(adata) + return pd.Series(np.asarray(X.sum(axis=1)).ravel().astype(float), index=adata.obs_names) + + +def run(par: dict): + adata = ad.read_h5ad(par["input"]) + out_path = par["output"] + + area, area_col = _get_area(adata) + total = _total_intensity(adata) + passed = pd.Series(True, index=adata.obs_names) + + adata.obs["passed_QC"] = passed.values.astype(bool) + adata.obs["total_intensity"] = total.values + if area_col is None: + adata.obs["area"] = area.values + adata.obs["qc_area_ok"] = True + adata.obs["qc_total_ok"] = True + adata.obs["qc_filter_reason"] = "pass_no_qc" + + adata.uns["qc"] = { + "method": "no_qc_filter", + "description": "All cells are marked as passing QC.", + "area_column_used": area_col if area_col is not None else "none", + "n_cells": int(adata.n_obs), + "n_passed": int(passed.sum()), + "pass_rate": float(passed.mean()) if adata.n_obs > 0 else float("nan"), + "median_total_intensity": float(np.nanmedian(total.values)) if adata.n_obs > 0 else float("nan"), + "median_area": float(np.nanmedian(area.values)) if adata.n_obs > 0 else float("nan"), + } + + adata.write_h5ad(out_path, compression="gzip") + + +if __name__ == "__main__": + ## VIASH START + par = { + "input": None, + "min_area": 50, + "min_total": 0.0, + "area_quantile": 0.05, + "total_quantile": 0.05, + "output": None, + } + meta = {} + ## VIASH END + run(par) diff --git a/src/methods_proteomics_segmentation/basic_threshold_segmentation/config.vsh.yaml b/src/methods_proteomics_segmentation/basic_threshold_segmentation/config.vsh.yaml new file mode 100644 index 0000000..21b2156 --- /dev/null +++ b/src/methods_proteomics_segmentation/basic_threshold_segmentation/config.vsh.yaml @@ -0,0 +1,27 @@ +__merge__: /src/api/comp_method_proteomics_segmentation.yaml + +name: basic_threshold_segmentation +label: "Basic threshold segmentation" +summary: "Simple threshold-based segmentation from a nuclear channel" +description: | + Baseline image-derived segmentation method. + It selects a nuclear marker channel, thresholds it using a configurable intensity quantile, + removes small objects, and converts connected components into cell labels. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "numpy" + - "scikit-image" + - "zarr<3.0.0" + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_segmentation/basic_threshold_segmentation/script.py b/src/methods_proteomics_segmentation/basic_threshold_segmentation/script.py new file mode 100644 index 0000000..27ff6c9 --- /dev/null +++ b/src/methods_proteomics_segmentation/basic_threshold_segmentation/script.py @@ -0,0 +1,92 @@ +import json + +import numpy as np +from skimage.filters import threshold_otsu +from skimage.measure import label +from skimage.morphology import remove_small_objects, binary_closing, disk + + +def _nuclear_threshold(values: np.ndarray, method: str, quantile: float) -> float: + """Threshold the nuclear channel using the requested strategy. + + Unknown strategies raise instead of silently falling back, so that a + mistyped --threshold_method is not reported as a successful run. + """ + method = str(method).lower() + if method == "quantile": + return float(np.quantile(values, quantile)) + if method == "otsu": + return float(threshold_otsu(values)) + raise ValueError(f"threshold_method must be either 'quantile' or 'otsu', got '{method}'.") + + +def run(par: dict): + input_image = par["input_image"] + nuclei_channel = int(par.get("nuclei_channel", 0)) + threshold_method = par.get("threshold_method") or "quantile" + threshold_quantile = float(par.get("threshold_quantile", 0.90)) + min_area = int(par.get("min_area", 30)) + output_labels = par["output_labels"] + output_metadata = par.get("output_metadata") + + img = np.load(input_image) + + if img.ndim != 3: + raise ValueError(f"Expected image in channel-y-x format, got shape {img.shape}") + + if nuclei_channel < 0 or nuclei_channel >= img.shape[0]: + raise ValueError( + f"nuclei_channel={nuclei_channel} out of range for image with {img.shape[0]} channels" + ) + + nuc = img[nuclei_channel].astype(np.float32) + finite = nuc[np.isfinite(nuc)] + + if finite.size == 0: + raise ValueError("Nuclei channel contains no finite values.") + + thr = _nuclear_threshold(finite, threshold_method, threshold_quantile) + + mask = nuc > thr + mask = binary_closing(mask, disk(1)) + mask = remove_small_objects(mask, min_size=min_area) + + # label() on a boolean mask always numbers objects 1..n without gaps. + labels = label(mask).astype(np.int32) + + np.save(output_labels, labels) + + if output_metadata: + n_objects = int(labels.max()) + with open(output_metadata, "w") as f: + json.dump( + { + "method": "basic_threshold_segmentation", + "nuclei_channel": nuclei_channel, + "threshold_method": str(threshold_method).lower(), + "threshold_quantile": threshold_quantile, + "threshold_value": thr, + "min_area": min_area, + "n_objects": n_objects, + "labels_shape": list(labels.shape), + }, + f, + indent=2, + ) + + +## VIASH START +par = { + "input_image": "corrected_image.npy", + "nuclei_channel": 0, + "threshold_method": "quantile", + "threshold_quantile": 0.90, + "min_area": 30, + "output_labels": "segmentation_labels.npy", + "output_metadata": None, +} +meta = {"name": "basic_threshold_segmentation"} +## VIASH END + +if __name__ == "__main__": + run(par) diff --git a/src/methods_proteomics_segmentation/reference_segmentation/config.vsh.yaml b/src/methods_proteomics_segmentation/reference_segmentation/config.vsh.yaml new file mode 100644 index 0000000..2512c3a --- /dev/null +++ b/src/methods_proteomics_segmentation/reference_segmentation/config.vsh.yaml @@ -0,0 +1,28 @@ +__merge__: /src/api/comp_method_proteomics_segmentation.yaml + +name: reference_segmentation +label: "Reference label segmentation" +summary: "Use dataset-provided segmentation labels from a SpatialData Zarr store" +description: | + Control segmentation method that extracts existing segmentation labels from SpatialData. + This is useful as an oracle/reference branch when evaluating image-derived segmentation methods. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "numpy" + - "spatialdata==0.5.0" + # spatialdata pulls in xarray_schema, which imports pkg_resources + - "setuptools<81" + - "zarr<3.0.0" + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_segmentation/reference_segmentation/script.py b/src/methods_proteomics_segmentation/reference_segmentation/script.py new file mode 100644 index 0000000..377e76d --- /dev/null +++ b/src/methods_proteomics_segmentation/reference_segmentation/script.py @@ -0,0 +1,101 @@ +import json + +import numpy as np +import spatialdata as sd + + +def _to_numpy(x): + if hasattr(x, "data"): + x = x.data + if hasattr(x, "compute"): + x = x.compute() + return np.asarray(x) + + +def _as_2d_labels(label_obj) -> np.ndarray: + arr = _to_numpy(label_obj) + dims = list(getattr(label_obj, "dims", [])) + + if dims and len(dims) == arr.ndim and "y" in dims and "x" in dims: + y_axis = dims.index("y") + x_axis = dims.index("x") + index = [0] * arr.ndim + index[y_axis] = slice(None) + index[x_axis] = slice(None) + arr = arr[tuple(index)] + else: + arr = np.squeeze(arr) + if arr.ndim == 3 and arr.shape[0] == 1: + arr = arr[0] + elif arr.ndim == 3 and arr.shape[-1] == 1: + arr = arr[..., 0] + + arr = np.squeeze(arr) + if arr.ndim != 2: + raise ValueError(f"Expected a 2D label image after squeezing/indexing, got shape {arr.shape}.") + + return arr.astype(np.int32, copy=False) + + +def run(par: dict): + input_zarr = par.get("input_zarr") + labels_key = par.get("labels_key", None) + output_labels = par["output_labels"] + output_metadata = par.get("output_metadata") + + if input_zarr is None or str(input_zarr).strip() == "": + raise ValueError("reference_segmentation requires --input_zarr.") + + sdata = sd.read_zarr(input_zarr) + + if not hasattr(sdata, "labels") or len(sdata.labels) == 0: + raise ValueError("No labels elements found in the SpatialData object.") + + if labels_key is None or str(labels_key).strip() in {"", "null", "None"}: + labels_key = list(sdata.labels.keys())[0] + + if labels_key not in sdata.labels: + available = list(sdata.labels.keys()) + raise KeyError(f"labels_key='{labels_key}' not found. Available labels keys: {available}") + + labels = _as_2d_labels(sdata.labels[labels_key]) + + if labels.min() < 0: + raise ValueError("Reference labels contain negative label values, which are not supported.") + + labels = labels.astype(np.int32, copy=False) + np.save(output_labels, labels) + + if output_metadata: + # Reference label ids are deliberately preserved as-is (not renumbered), + # so that predictions stay comparable to the dataset's own cell ids. + unique_labels = np.unique(labels) + n_objects = int((unique_labels > 0).sum()) + with open(output_metadata, "w") as f: + json.dump( + { + "method": "reference_segmentation", + "input_zarr": str(input_zarr), + "labels_key": str(labels_key), + "n_objects": n_objects, + "max_label": int(labels.max()), + "relabeled": False, + "labels_shape": list(labels.shape), + }, + f, + indent=2, + ) + + +## VIASH START +par = { + "input_zarr": "data.zarr", + "labels_key": None, + "output_labels": "segmentation_labels.npy", + "output_metadata": None, +} +meta = {"name": "reference_segmentation"} +## VIASH END + +if __name__ == "__main__": + run(par) diff --git a/src/methods_proteomics_segmentation/watershed_expand_segmentation/config.vsh.yaml b/src/methods_proteomics_segmentation/watershed_expand_segmentation/config.vsh.yaml new file mode 100644 index 0000000..83b1013 --- /dev/null +++ b/src/methods_proteomics_segmentation/watershed_expand_segmentation/config.vsh.yaml @@ -0,0 +1,28 @@ +__merge__: /src/api/comp_method_proteomics_segmentation.yaml + +name: watershed_expand_segmentation +label: "Watershed and expansion segmentation" +summary: "Classical nuclear watershed segmentation followed by label expansion" +description: | + Image-derived segmentation method using nuclear foreground detection, distance-transform watershed, + and optional expansion of nuclear labels toward approximate cell regions. This provides a stronger + classical baseline than simple connected-component thresholding. + +resources: + - type: python_script + path: script.py + +engines: + - type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - "numpy" + - "scipy" + - "scikit-image" + - "zarr<3.0.0" + - type: native + +runners: + - type: executable diff --git a/src/methods_proteomics_segmentation/watershed_expand_segmentation/script.py b/src/methods_proteomics_segmentation/watershed_expand_segmentation/script.py new file mode 100644 index 0000000..18b9507 --- /dev/null +++ b/src/methods_proteomics_segmentation/watershed_expand_segmentation/script.py @@ -0,0 +1,144 @@ +import json + +import numpy as np +from scipy import ndimage as ndi +from skimage.filters import gaussian, threshold_otsu +from skimage.feature import peak_local_max +from skimage.measure import label +from skimage.morphology import remove_small_objects, remove_small_holes, binary_closing, disk +from skimage.segmentation import watershed, expand_labels, relabel_sequential + + +def _nuclear_threshold(values: np.ndarray, method: str, quantile: float) -> float: + """Threshold the nuclear channel using the requested strategy. + + Unknown strategies raise instead of silently falling back, so that a + mistyped --threshold_method is not reported as a successful run. + """ + method = str(method).lower() + if method == "quantile": + return float(np.quantile(values, quantile)) + if method == "otsu": + return float(threshold_otsu(values)) + raise ValueError(f"threshold_method must be either 'quantile' or 'otsu', got '{method}'.") + + +def _make_markers(distance: np.ndarray, mask: np.ndarray, min_distance: int) -> np.ndarray: + """Create watershed seed markers from distance-transform local maxima.""" + if mask.sum() == 0: + return np.zeros(mask.shape, dtype=np.int32) + + coords = peak_local_max( + distance, + labels=mask, + min_distance=max(1, int(min_distance)), + exclude_border=False, + ) + + marker_img = np.zeros(mask.shape, dtype=bool) + if coords.size > 0: + marker_img[tuple(coords.T)] = True + markers = label(marker_img).astype(np.int32) + else: + # Fallback: connected components if no robust local maxima are detected. + markers = label(mask).astype(np.int32) + + return markers + + +def run(par: dict): + input_image = par["input_image"] + nuclei_channel = int(par.get("nuclei_channel", 0)) + threshold_method = par.get("threshold_method") or "quantile" + threshold_quantile = float(par.get("threshold_quantile", 0.90)) + min_area = int(par.get("min_area", 30)) + smoothing_sigma = float(par.get("smoothing_sigma", 1.0)) + hole_area = int(par.get("hole_area", 64)) + seed_min_distance = int(par.get("seed_min_distance", 5)) + expansion_distance = int(par.get("expansion_distance", 5)) + output_labels = par["output_labels"] + output_metadata = par.get("output_metadata") + + img = np.load(input_image) + if img.ndim != 3: + raise ValueError(f"Expected image in channel-y-x format, got shape {img.shape}") + if nuclei_channel < 0 or nuclei_channel >= img.shape[0]: + raise ValueError( + f"nuclei_channel={nuclei_channel} out of range for image with {img.shape[0]} channels" + ) + + nuc = img[nuclei_channel].astype(np.float32) + finite = nuc[np.isfinite(nuc)] + if finite.size == 0: + raise ValueError("Nuclei channel contains no finite values.") + + if smoothing_sigma > 0: + nuc_smooth = gaussian(nuc, sigma=smoothing_sigma, preserve_range=True) + else: + nuc_smooth = nuc + + thr = _nuclear_threshold(finite, threshold_method, threshold_quantile) + mask = nuc_smooth > thr + mask = binary_closing(mask, disk(1)) + mask = remove_small_holes(mask, area_threshold=max(1, hole_area)) + mask = remove_small_objects(mask, min_size=min_area) + + distance = ndi.distance_transform_edt(mask) + markers = _make_markers(distance, mask, seed_min_distance) + labels = watershed(-distance, markers=markers, mask=mask).astype(np.int32) + labels = remove_small_objects(labels, min_size=min_area).astype(np.int32) + + if expansion_distance > 0 and labels.max() > 0: + labels = expand_labels(labels, distance=expansion_distance).astype(np.int32) + + # Dropping small objects from a *label* image leaves holes in the numbering + # (e.g. max label 7202 for only 6848 surviving objects). Downstream + # aggregation counts unique labels, so renumber to a gapless 1..n range and + # keep max(labels) == n_objects a usable invariant. + labels, _, _ = relabel_sequential(labels) + labels = labels.astype(np.int32) + + np.save(output_labels, labels) + + if output_metadata: + n_objects = int(labels.max()) + with open(output_metadata, "w") as f: + json.dump( + { + "method": "watershed_expand_segmentation", + "nuclei_channel": nuclei_channel, + "threshold_method": str(threshold_method).lower(), + "threshold_quantile": threshold_quantile, + "threshold_value": thr, + "smoothing_sigma": smoothing_sigma, + "min_area": min_area, + "hole_area": hole_area, + "seed_min_distance": seed_min_distance, + "expansion_distance": expansion_distance, + "n_objects": n_objects, + "labels_shape": list(labels.shape), + }, + f, + indent=2, + ) + + +## VIASH START +par = { + "input_image": "corrected_image.npy", + "nuclei_channel": 0, + "threshold_method": "quantile", + "threshold_quantile": 0.90, + "min_area": 30, + "smoothing_sigma": 1.0, + "hole_area": 64, + "seed_min_distance": 5, + "expansion_distance": 5, + "output_labels": "segmentation_labels.npy", + "output_metadata": None, +} +meta = {"name": "watershed_expand_segmentation"} +## VIASH END + +if __name__ == "__main__": + run(par) diff --git a/src/metrics/accuracy/config.vsh.yaml b/src/metrics/accuracy/config.vsh.yaml deleted file mode 100644 index ac197bd..0000000 --- a/src/metrics/accuracy/config.vsh.yaml +++ /dev/null @@ -1,70 +0,0 @@ -# The API specifies which type of component this is. -# It contains specifications for: -# - The input/output files -# - Common parameters -# - A unit test -__merge__: ../../api/comp_metric.yaml - - -# A unique identifier for your component (required). -# Can contain only lowercase letters or underscores. -name: accuracy - -# Metadata for your component -info: - metrics: - # A unique identifier for your metric (required). - # Can contain only lowercase letters or underscores. - - name: accuracy - # A relatively short label, used when rendering visualisarions (required) - label: Accuracy - # A one sentence summary of how this metric works (required). Used when - # rendering summary tables. - summary: "The percentage of correctly predicted labels." - # A multi-line description of how this component works (required). Used - # when rendering reference documentation. - description: | - The percentage of correctly predicted labels. - # A reference key from the bibtex library at src/common/library.bib (required). - references: - doi: 10.48550/arXiv.2008.05756 - # The minimum possible value for this metric (required) - min: 0 - # The maximum possible value for this metric (required) - max: 1 - # Whether a higher value represents a 'better' solution (required) - maximize: true - -# Component-specific parameters (optional) -# arguments: -# - name: "--n_neighbors" -# type: "integer" -# default: 5 -# description: Number of neighbors to use. - -# Resources required to run the component -resources: - # The script of your component (required) - - type: python_script - path: script.py - # Additional resources your script needs (optional) - # - type: file - # path: weights.pt - -engines: - # Specifications for the Docker image for this component. - - type: docker - image: openproblems/base_python:1 - # Add custom dependencies here (optional). For more information, see - # https://viash.io/reference/config/engines/docker/#setup . - setup: - - type: python - packages: scikit-learn - -runners: - # This platform allows running the component natively - - type: executable - # Allows turning the component into a Nextflow module / pipeline. - - type: nextflow - directives: - label: [midtime, midmem, midcpu] diff --git a/src/metrics/accuracy/script.py b/src/metrics/accuracy/script.py deleted file mode 100644 index 6260ae7..0000000 --- a/src/metrics/accuracy/script.py +++ /dev/null @@ -1,47 +0,0 @@ -import anndata as ad -import numpy as np -import sklearn.preprocessing - -## VIASH START -# Note: this section is auto-generated by viash at runtime. To edit it, make changes -# in config.vsh.yaml and then run `viash config inject config.vsh.yaml`. -par = { - 'input_solution': 'resources_test/task_template/cxg_mouse_pancreas_atlas/solution.h5ad', - 'input_prediction': 'resources_test/task_template/cxg_mouse_pancreas_atlas/prediction.h5ad', - 'output': 'output.h5ad' -} -meta = { - 'name': 'accuracy' -} -## VIASH END - -print('Reading input files', flush=True) -input_solution = ad.read_h5ad(par['input_solution']) -input_prediction = ad.read_h5ad(par['input_prediction']) - -assert (input_prediction.obs_names == input_solution.obs_names).all(), "obs_names not the same in prediction and solution inputs" - -print("Encode labels", flush=True) -cats = list(input_solution.obs["label"].dtype.categories) + list(input_prediction.obs["label_pred"].dtype.categories) -encoder = sklearn.preprocessing.LabelEncoder().fit(cats) -input_solution.obs["label"] = encoder.transform(input_solution.obs["label"]) -input_prediction.obs["label_pred"] = encoder.transform(input_prediction.obs["label_pred"]) - - -print('Compute metrics', flush=True) -# metric_ids and metric_values can have length > 1 -# but should be of equal length -uns_metric_ids = [ 'accuracy' ] -uns_metric_values = [ np.mean(input_solution.obs["label"] == input_prediction.obs["label_pred"]) ] - -print("Write output AnnData to file", flush=True) -output = ad.AnnData( - uns={ - 'dataset_id': input_prediction.uns['dataset_id'], - 'normalization_id': input_prediction.uns['normalization_id'], - 'method_id': input_prediction.uns['method_id'], - 'metric_ids': uns_metric_ids, - 'metric_values': uns_metric_values - } -) -output.write_h5ad(par['output'], compression='gzip') diff --git a/src/metrics_proteomics/basic/__pycache__/script.cpython-312.pyc b/src/metrics_proteomics/basic/__pycache__/script.cpython-312.pyc new file mode 100644 index 0000000..36da591 Binary files /dev/null and b/src/metrics_proteomics/basic/__pycache__/script.cpython-312.pyc differ diff --git a/src/metrics_proteomics/basic/__pycache__/script.cpython-313.pyc b/src/metrics_proteomics/basic/__pycache__/script.cpython-313.pyc new file mode 100644 index 0000000..542b741 Binary files /dev/null and b/src/metrics_proteomics/basic/__pycache__/script.cpython-313.pyc differ diff --git a/src/metrics_proteomics/basic/config.vsh.yaml b/src/metrics_proteomics/basic/config.vsh.yaml new file mode 100644 index 0000000..6d98ecb --- /dev/null +++ b/src/metrics_proteomics/basic/config.vsh.yaml @@ -0,0 +1,72 @@ +__merge__: /src/api/comp_metric_proteomics_basic.yaml +name: basic +label: Basic preprocessing metrics +summary: Compute core QC, morphology, intensity, and marker coverage metrics +description: 'Computes a compact summary of the processed cell-by-protein matrix, including + + number of cells, number of markers, QC pass rate, median cell area, median + + total raw intensity, and marker coverage. + + ' +resources: +- type: python_script + path: script.py +engines: +- type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - sprotmetrics==0.1.0 + - anndata>=0.10.0 + - zarr<3.0.0 +- type: native +runners: +- type: executable +info: + metrics: + - name: median_marker_coverage + label: Per-marker detection rate + summary: Fraction of cells whose intensity exceeds `threshold`, per marker. + description: Fraction of cells whose intensity exceeds `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: median_marker_coverage_qc_pass + label: Per-marker detection rate + summary: Fraction of cells whose intensity exceeds `threshold`, per marker. + description: Fraction of cells whose intensity exceeds `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: min_marker_coverage + label: Per-marker detection rate + summary: Fraction of cells whose intensity exceeds `threshold`, per marker. + description: Fraction of cells whose intensity exceeds `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: min_marker_coverage_qc_pass + label: Per-marker detection rate + summary: Fraction of cells whose intensity exceeds `threshold`, per marker. + description: Fraction of cells whose intensity exceeds `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true diff --git a/src/metrics_proteomics/basic/script.py b/src/metrics_proteomics/basic/script.py new file mode 100644 index 0000000..dcbd46f --- /dev/null +++ b/src/metrics_proteomics/basic/script.py @@ -0,0 +1,68 @@ +"""Dataset and object statistics. + +The metric itself lives in `sprotmetrics`, which this component now depends on; +what remains here is the Viash contract: read the inputs, call the library, +write `uns['metric_ids']` and `uns['metric_values']` in the established order. +""" + +import argparse + +import anndata as ad +import numpy as np +import sprotmetrics as spm + +QC_COLUMN = "passed_QC" + + +def run(par: dict): + cov_thr = par.get("cov_thr", par.get("coverage_threshold", 0.1)) + cov_thr = 0.1 if cov_thr is None else float(cov_thr) + + adata = ad.read_h5ad(par["input"]) + scores = spm.basic_statistics( + adata, + layer="counts", + area="area", + threshold=cov_thr, + qc_variants=True, + qc_column=QC_COLUMN, + ) + + out = ad.AnnData( + uns={ + "metric_ids": np.array(list(scores), dtype=object), + "metric_values": np.array(list(scores.values()), dtype=float), + "metric_group": "basic", + "coverage_threshold": cov_thr, + "qc_column": QC_COLUMN, + } + ) + out.write_h5ad(par["output"], compression="gzip") + + +## VIASH START +par = { + "input": "cell_by_protein_norm.h5ad", + "cov_thr": 0.1, + "output": "metrics_basic.h5ad", +} +meta = {"name": "basic"} +## VIASH END + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--cov_thr", type=float, default=None) + ap.add_argument("--coverage_threshold", type=float, default=None) + ap.add_argument("--output", required=True) + args = ap.parse_args() + cov_thr = args.cov_thr if args.cov_thr is not None else args.coverage_threshold + run({"input": args.input, "cov_thr": cov_thr, "output": args.output}) + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + main() diff --git a/src/metrics_proteomics/marker_distribution/__pycache__/script.cpython-312.pyc b/src/metrics_proteomics/marker_distribution/__pycache__/script.cpython-312.pyc new file mode 100644 index 0000000..48e659a Binary files /dev/null and b/src/metrics_proteomics/marker_distribution/__pycache__/script.cpython-312.pyc differ diff --git a/src/metrics_proteomics/marker_distribution/__pycache__/script.cpython-313.pyc b/src/metrics_proteomics/marker_distribution/__pycache__/script.cpython-313.pyc new file mode 100644 index 0000000..c59f269 Binary files /dev/null and b/src/metrics_proteomics/marker_distribution/__pycache__/script.cpython-313.pyc differ diff --git a/src/metrics_proteomics/marker_distribution/config.vsh.yaml b/src/metrics_proteomics/marker_distribution/config.vsh.yaml new file mode 100644 index 0000000..9329bf4 --- /dev/null +++ b/src/metrics_proteomics/marker_distribution/config.vsh.yaml @@ -0,0 +1,164 @@ +__merge__: /src/api/comp_metric_proteomics_marker_distribution.yaml +name: marker_distribution +label: Marker distribution metrics +summary: Compute marker-level intensity, sparsity, coverage, and dynamic-range metrics +description: 'Computes marker-level distribution diagnostics that are useful for comparing + + image correction, segmentation, aggregation, QC, and normalization choices. + + ' +resources: +- type: python_script + path: script.py +engines: +- type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - sprotmetrics==0.1.0 + - anndata>=0.10.0 + - zarr<3.0.0 +- type: native +runners: +- type: executable +info: + metrics: + - name: median_marker_zero_fraction + label: Per-marker fraction at or below zero + summary: Fraction of cells at or below `threshold`, per marker. + description: Fraction of cells at or below `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: median_marker_zero_fraction_qc_pass + label: Per-marker fraction at or below zero + summary: Fraction of cells at or below `threshold`, per marker. + description: Fraction of cells at or below `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_marker_zero_fraction + label: Per-marker fraction at or below zero + summary: Fraction of cells at or below `threshold`, per marker. + description: Fraction of cells at or below `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_marker_zero_fraction_qc_pass + label: Per-marker fraction at or below zero + summary: Fraction of cells at or below `threshold`, per marker. + description: Fraction of cells at or below `threshold`, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: median_marker_dynamic_range + label: Per-marker dynamic range + summary: High minus low quantile of intensity, per marker. + description: High minus low quantile of intensity, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: median_marker_dynamic_range_qc_pass + label: Per-marker dynamic range + summary: High minus low quantile of intensity, per marker. + description: High minus low quantile of intensity, per marker. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: median_marker_bimodality + label: Marker positive-population separability + summary: Separability of the two populations an Otsu split produces, per marker. + description: "Separability of the two populations an Otsu split produces, per marker.\n\n ``otsu_variance_ratio``\ + \ is between-class over within-class variance.\n ``bimodality_index`` is Wang et al. (2009),\ + \ ``sqrt(p(1-p)) * |m1-m2| / sigma``,\n with both components taken from the Otsu split rather\ + \ than an EM fit; it is\n the version with a conventional cutoff (roughly BI > 1.1).\n\n This\ + \ is what image correction and normalization are *for*, and nothing in the\n benchmark measured\ + \ it." + references: + doi: + - 10.4137/CIN.S2846 + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: median_marker_bimodality_qc_pass + label: Marker positive-population separability + summary: Separability of the two populations an Otsu split produces, per marker. + description: "Separability of the two populations an Otsu split produces, per marker.\n\n ``otsu_variance_ratio``\ + \ is between-class over within-class variance.\n ``bimodality_index`` is Wang et al. (2009),\ + \ ``sqrt(p(1-p)) * |m1-m2| / sigma``,\n with both components taken from the Otsu split rather\ + \ than an EM fit; it is\n the version with a conventional cutoff (roughly BI > 1.1).\n\n This\ + \ is what image correction and normalization are *for*, and nothing in the\n benchmark measured\ + \ it." + references: + doi: + - 10.4137/CIN.S2846 + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: min_marker_bimodality + label: Marker positive-population separability + summary: Separability of the two populations an Otsu split produces, per marker. + description: "Separability of the two populations an Otsu split produces, per marker.\n\n ``otsu_variance_ratio``\ + \ is between-class over within-class variance.\n ``bimodality_index`` is Wang et al. (2009),\ + \ ``sqrt(p(1-p)) * |m1-m2| / sigma``,\n with both components taken from the Otsu split rather\ + \ than an EM fit; it is\n the version with a conventional cutoff (roughly BI > 1.1).\n\n This\ + \ is what image correction and normalization are *for*, and nothing in the\n benchmark measured\ + \ it." + references: + doi: + - 10.4137/CIN.S2846 + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: min_marker_bimodality_qc_pass + label: Marker positive-population separability + summary: Separability of the two populations an Otsu split produces, per marker. + description: "Separability of the two populations an Otsu split produces, per marker.\n\n ``otsu_variance_ratio``\ + \ is between-class over within-class variance.\n ``bimodality_index`` is Wang et al. (2009),\ + \ ``sqrt(p(1-p)) * |m1-m2| / sigma``,\n with both components taken from the Otsu split rather\ + \ than an EM fit; it is\n the version with a conventional cutoff (roughly BI > 1.1).\n\n This\ + \ is what image correction and normalization are *for*, and nothing in the\n benchmark measured\ + \ it." + references: + doi: + - 10.4137/CIN.S2846 + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true diff --git a/src/metrics_proteomics/marker_distribution/script.py b/src/metrics_proteomics/marker_distribution/script.py new file mode 100644 index 0000000..9025897 --- /dev/null +++ b/src/metrics_proteomics/marker_distribution/script.py @@ -0,0 +1,83 @@ +"""Per-marker distribution summaries. + +Thin wrapper over `sprotmetrics.marker_distribution`; the per-marker table it +also publishes to `uns` comes from `sprotmetrics.marker_table`, which is the +same computation the summaries reduce. +""" + +import argparse + +import anndata as ad +import numpy as np +import sprotmetrics as spm + +QC_COLUMN = "passed_QC" + + +def run(par: dict): + layer = par.get("layer", "counts") + zero_threshold = float(par.get("zero_threshold", 0.0)) + coverage_threshold = float(par.get("coverage_threshold", 0.1)) + low_q = float(par.get("low_quantile", 0.01)) + high_q = float(par.get("high_quantile", 0.99)) + + adata = ad.read_h5ad(par["input"]) + shared = { + "layer": layer, + "zero_threshold": zero_threshold, + "coverage_threshold": coverage_threshold, + "quantiles": (low_q, high_q), + } + scores = spm.marker_distribution(adata, qc_variants=True, qc_column=QC_COLUMN, **shared) + marker_table = spm.marker_table(adata, **shared) + + uns = { + "metric_ids": np.array(list(scores), dtype=object), + "metric_values": np.array(list(scores.values()), dtype=float), + "metric_group": "marker_distribution", + "evaluated_layer": layer, + "zero_threshold": zero_threshold, + "coverage_threshold": coverage_threshold, + "low_quantile": low_q, + "high_quantile": high_q, + "qc_column": QC_COLUMN, + "marker_table": marker_table.reset_index(drop=True).to_dict(orient="list"), + } + if QC_COLUMN in adata.obs.columns and np.asarray(adata.obs[QC_COLUMN]).any(): + qc_table = spm.marker_table(adata, subset=QC_COLUMN, **shared) + uns["marker_table_qc_pass"] = qc_table.reset_index(drop=True).to_dict(orient="list") + + ad.AnnData(uns=uns).write_h5ad(par["output"], compression="gzip") + + +## VIASH START +par = { + "input": "cell_by_protein_norm.h5ad", + "layer": "counts", + "zero_threshold": 0.0, + "coverage_threshold": 0.1, + "low_quantile": 0.01, + "high_quantile": 0.99, + "output": "metrics_marker_distribution.h5ad", +} +meta = {"name": "marker_distribution"} +## VIASH END + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--layer", default="counts") + ap.add_argument("--zero_threshold", type=float, default=0.0) + ap.add_argument("--coverage_threshold", type=float, default=0.1) + ap.add_argument("--low_quantile", type=float, default=0.01) + ap.add_argument("--high_quantile", type=float, default=0.99) + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + main() diff --git a/src/metrics_proteomics/normalization_diagnostics/__pycache__/script.cpython-312.pyc b/src/metrics_proteomics/normalization_diagnostics/__pycache__/script.cpython-312.pyc new file mode 100644 index 0000000..1cc1c2e Binary files /dev/null and b/src/metrics_proteomics/normalization_diagnostics/__pycache__/script.cpython-312.pyc differ diff --git a/src/metrics_proteomics/normalization_diagnostics/__pycache__/script.cpython-313.pyc b/src/metrics_proteomics/normalization_diagnostics/__pycache__/script.cpython-313.pyc new file mode 100644 index 0000000..3215e12 Binary files /dev/null and b/src/metrics_proteomics/normalization_diagnostics/__pycache__/script.cpython-313.pyc differ diff --git a/src/metrics_proteomics/normalization_diagnostics/config.vsh.yaml b/src/metrics_proteomics/normalization_diagnostics/config.vsh.yaml new file mode 100644 index 0000000..3b5fe96 --- /dev/null +++ b/src/metrics_proteomics/normalization_diagnostics/config.vsh.yaml @@ -0,0 +1,98 @@ +__merge__: /src/api/comp_metric_proteomics_normalization_diagnostics.yaml +name: normalization_diagnostics +label: Normalization diagnostics +summary: Compute diagnostics comparing raw and normalized protein intensity matrices +description: 'Computes metrics for assessing how normalization changes the data, including + + total-intensity coefficient of variation before and after normalization, + + per-cell rank preservation, and marker variance changes. + + ' +resources: +- type: python_script + path: script.py +engines: +- type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - sprotmetrics==0.1.0 + - anndata>=0.10.0 + - zarr<3.0.0 +- type: native +runners: +- type: executable +info: + metrics: + - name: total_cv_raw + label: Spread of per-cell total intensity + summary: Coefficient of variation of per-cell summed intensity. + description: Coefficient of variation of per-cell summed intensity. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: total_cv_raw_qc_pass + label: Spread of per-cell total intensity + summary: Coefficient of variation of per-cell summed intensity. + description: Coefficient of variation of per-cell summed intensity. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: total_cv_normalized + label: Spread of per-cell total intensity + summary: Coefficient of variation of per-cell summed intensity. + description: Coefficient of variation of per-cell summed intensity. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: total_cv_normalized_qc_pass + label: Spread of per-cell total intensity + summary: Coefficient of variation of per-cell summed intensity. + description: Coefficient of variation of per-cell summed intensity. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: total_cv_ratio_norm_over_raw + label: Compression of total-intensity spread + summary: Normalized over raw coefficient of variation of per-cell totals. + description: "Normalized over raw coefficient of variation of per-cell totals.\n\n Below 1 means\ + \ the normalization compressed the spread in per-cell total\n intensity, which is what it is\ + \ for." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: total_cv_ratio_norm_over_raw_qc_pass + label: Compression of total-intensity spread + summary: Normalized over raw coefficient of variation of per-cell totals. + description: "Normalized over raw coefficient of variation of per-cell totals.\n\n Below 1 means\ + \ the normalization compressed the spread in per-cell total\n intensity, which is what it is\ + \ for." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false diff --git a/src/metrics_proteomics/normalization_diagnostics/script.py b/src/metrics_proteomics/normalization_diagnostics/script.py new file mode 100644 index 0000000..21168d4 --- /dev/null +++ b/src/metrics_proteomics/normalization_diagnostics/script.py @@ -0,0 +1,72 @@ +"""Raw-versus-normalized diagnostics. Thin wrapper over `sprotmetrics`.""" + +import argparse + +import anndata as ad +import numpy as np +import sprotmetrics as spm + +QC_COLUMN = "passed_QC" + + +def run(par: dict): + raw_layer = par.get("raw_layer", "counts") + normalized_layer = par.get("normalized_layer", "normalized") + max_cells = int(par.get("max_cells", 5000)) + random_state = int(par.get("random_state", 0)) + + adata = ad.read_h5ad(par["input"]) + scores = spm.normalization_diagnostics( + adata, + raw_layer=raw_layer, + normalized_layer=normalized_layer, + max_cells=max_cells, + random_state=random_state, + qc_variants=True, + qc_column=QC_COLUMN, + ) + + out = ad.AnnData( + uns={ + "metric_ids": np.array(list(scores), dtype=object), + "metric_values": np.array(list(scores.values()), dtype=float), + "metric_group": "normalization_diagnostics", + "raw_layer": raw_layer, + "normalized_layer": normalized_layer, + "max_cells": max_cells, + "random_state": random_state, + "qc_column": QC_COLUMN, + } + ) + out.write_h5ad(par["output"], compression="gzip") + + +## VIASH START +par = { + "input": "cell_by_protein_norm.h5ad", + "raw_layer": "counts", + "normalized_layer": "normalized", + "max_cells": 5000, + "random_state": 0, + "output": "metrics_normalization_diagnostics.h5ad", +} +meta = {"name": "normalization_diagnostics"} +## VIASH END + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--raw_layer", default="counts") + ap.add_argument("--normalized_layer", default="normalized") + ap.add_argument("--max_cells", type=int, default=5000) + ap.add_argument("--random_state", type=int, default=0) + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + main() diff --git a/src/metrics_proteomics/segmentation_quality/__pycache__/script.cpython-312.pyc b/src/metrics_proteomics/segmentation_quality/__pycache__/script.cpython-312.pyc new file mode 100644 index 0000000..bc00a9e Binary files /dev/null and b/src/metrics_proteomics/segmentation_quality/__pycache__/script.cpython-312.pyc differ diff --git a/src/metrics_proteomics/segmentation_quality/__pycache__/script.cpython-313.pyc b/src/metrics_proteomics/segmentation_quality/__pycache__/script.cpython-313.pyc new file mode 100644 index 0000000..b72c551 Binary files /dev/null and b/src/metrics_proteomics/segmentation_quality/__pycache__/script.cpython-313.pyc differ diff --git a/src/metrics_proteomics/segmentation_quality/config.vsh.yaml b/src/metrics_proteomics/segmentation_quality/config.vsh.yaml new file mode 100644 index 0000000..59ea3c5 --- /dev/null +++ b/src/metrics_proteomics/segmentation_quality/config.vsh.yaml @@ -0,0 +1,219 @@ +__merge__: /src/api/comp_metric_proteomics_segmentation_quality.yaml +name: segmentation_quality +label: Segmentation quality metrics +summary: Reference-free segmentation scores derived from known marker biology +description: "Scores segmentation without reference masks, using four properties the data\nmust satisfy\ + \ if cells were delineated correctly:\n\n- `dapi_positive_ratio` - fraction of objects containing nuclear\ + \ signal.\n Every segmented cell should have a nucleus, so a low value means objects are\n being produced\ + \ that are not cells. Higher is better.\n- `replicate_correlation` - agreement between nuclear stains\ + \ repeated across\n imaging cycles. They image the same physical nuclei, so disagreement is\n direct\ + \ evidence of mis-segmentation or mis-registration. Higher is better.\n- `median_replicate_cv` - the\ + \ same signal seen per cell: spread across the\n repeated stains within each cell. Lower is better.\n\ + - `mutually_exclusive_coexpression_ratio` - how often cells are co-positive\n for markers restricted\ + \ to incompatible lineages, as a fraction of cells\n positive for either. Merged or bled masks raise\ + \ it. Lower is better.\n- `spillover_ratio` - intensity in negative cells adjacent to a positive cell\n\ + \ against negative cells that are not adjacent. Over-expanded masks raise it\n above 1. Lower is better.\n\ + \nPositivity is set per marker by Otsu on log1p intensity rather than a fixed\ncutoff, so the metrics\ + \ survive a change of normalization or intensity scale.\nCo-expression is NaN unless `--exclusive_pairs`\ + \ is given, since the correct\npairs depend on the panel.\n" +resources: +- type: python_script + path: script.py +engines: +- type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - sprotmetrics==0.1.0 + - anndata>=0.10.0 + - zarr<3.0.0 +- type: native +runners: +- type: executable +info: + metrics: + - name: dapi_positive_ratio + label: Fraction of objects containing nuclear signal + summary: Fraction of segmented objects positive for the averaged nuclear channels. + description: "Fraction of segmented objects positive for the averaged nuclear channels.\n\n Every\ + \ segmented cell should contain a nucleus; objects that do not are debris\n or mask fragments." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: dapi_positive_ratio_qc_pass + label: Fraction of objects containing nuclear signal + summary: Fraction of segmented objects positive for the averaged nuclear channels. + description: "Fraction of segmented objects positive for the averaged nuclear channels.\n\n Every\ + \ segmented cell should contain a nucleus; objects that do not are debris\n or mask fragments." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: replicate_correlation + label: Agreement between repeated nuclear stains + summary: Mean pairwise correlation between nuclear stains repeated across imaging cycles. + description: "Mean pairwise correlation between nuclear stains repeated across imaging cycles.\n\n\ + \ The same nuclei imaged twice should agree; disagreement means masks are\n picking up different\ + \ amounts of each restain." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: -1 + max: 1 + maximize: true + - name: replicate_correlation_qc_pass + label: Agreement between repeated nuclear stains + summary: Mean pairwise correlation between nuclear stains repeated across imaging cycles. + description: "Mean pairwise correlation between nuclear stains repeated across imaging cycles.\n\n\ + \ The same nuclei imaged twice should agree; disagreement means masks are\n picking up different\ + \ amounts of each restain." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: -1 + max: 1 + maximize: true + - name: median_replicate_cv + label: Per-cell spread across repeated nuclear stains + summary: Coefficient of variation across the repeated nuclear stains, per cell. + description: "Coefficient of variation across the repeated nuclear stains, per cell.\n\n The per-cell\ + \ counterpart of :func:`replicate_correlation`: a mask that strays\n off its nucleus picks up\ + \ different amounts of each restain." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: median_replicate_cv_qc_pass + label: Per-cell spread across repeated nuclear stains + summary: Coefficient of variation across the repeated nuclear stains, per cell. + description: "Coefficient of variation across the repeated nuclear stains, per cell.\n\n The per-cell\ + \ counterpart of :func:`replicate_correlation`: a mask that strays\n off its nucleus picks up\ + \ different amounts of each restain." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: mutually_exclusive_coexpression_ratio + label: Co-positivity of incompatible lineage markers + summary: Cells co-positive for incompatible markers, over cells positive for either. + description: "Cells co-positive for incompatible markers, over cells positive for either.\n\n Normalising\ + \ by the union rather than by all cells stops the value shrinking\n simply because few cells\ + \ express either marker. A merged mask spanning an\n epithelial and an immune cell shows up here." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mutually_exclusive_coexpression_ratio_qc_pass + label: Co-positivity of incompatible lineage markers + summary: Cells co-positive for incompatible markers, over cells positive for either. + description: "Cells co-positive for incompatible markers, over cells positive for either.\n\n Normalising\ + \ by the union rather than by all cells stops the value shrinking\n simply because few cells\ + \ express either marker. A merged mask spanning an\n epithelial and an immune cell shows up here." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: spillover_ratio + label: Signal bleed into neighbouring cells + summary: Intensity of negative cells touching a positive cell, over those that do not. + description: "Intensity of negative cells touching a positive cell, over those that do not.\n\n \ + \ A marker positive in one cell should be absent from its neighbours. Masks\n that over-expand\ + \ across a boundary raise this above 1." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: spillover_ratio_qc_pass + label: Signal bleed into neighbouring cells + summary: Intensity of negative cells touching a positive cell, over those that do not. + description: "Intensity of negative cells touching a positive cell, over those that do not.\n\n \ + \ A marker positive in one cell should be absent from its neighbours. Masks\n that over-expand\ + \ across a boundary raise this above 1." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: median_tile_cv + label: Variation of marker intensity across the slide + summary: Coefficient of variation of a marker's mean across a grid of tiles, per marker. + description: "Coefficient of variation of a marker's mean across a grid of tiles, per marker.\n\n\ + \ Illumination shading and stitching seams make one region of the slide\n systematically brighter.\ + \ Real tissue is spatially structured too, so a high\n value is suggestive rather than conclusive\ + \ -- but a *fall* after flat-field\n correction is evidence the correction worked." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: median_tile_cv_qc_pass + label: Variation of marker intensity across the slide + summary: Coefficient of variation of a marker's mean across a grid of tiles, per marker. + description: "Coefficient of variation of a marker's mean across a grid of tiles, per marker.\n\n\ + \ Illumination shading and stitching seams make one region of the slide\n systematically brighter.\ + \ Real tissue is spatially structured too, so a high\n value is suggestive rather than conclusive\ + \ -- but a *fall* after flat-field\n correction is evidence the correction worked." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: max_tile_cv + label: Variation of marker intensity across the slide + summary: Coefficient of variation of a marker's mean across a grid of tiles, per marker. + description: "Coefficient of variation of a marker's mean across a grid of tiles, per marker.\n\n\ + \ Illumination shading and stitching seams make one region of the slide\n systematically brighter.\ + \ Real tissue is spatially structured too, so a high\n value is suggestive rather than conclusive\ + \ -- but a *fall* after flat-field\n correction is evidence the correction worked." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false + - name: max_tile_cv_qc_pass + label: Variation of marker intensity across the slide + summary: Coefficient of variation of a marker's mean across a grid of tiles, per marker. + description: "Coefficient of variation of a marker's mean across a grid of tiles, per marker.\n\n\ + \ Illumination shading and stitching seams make one region of the slide\n systematically brighter.\ + \ Real tissue is spatially structured too, so a high\n value is suggestive rather than conclusive\ + \ -- but a *fall* after flat-field\n correction is evidence the correction worked." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: false diff --git a/src/metrics_proteomics/segmentation_quality/script.py b/src/metrics_proteomics/segmentation_quality/script.py new file mode 100644 index 0000000..2c9bf14 --- /dev/null +++ b/src/metrics_proteomics/segmentation_quality/script.py @@ -0,0 +1,101 @@ +"""Reference-free segmentation quality. Thin wrapper over `sprotmetrics`. + +The five metrics come from two places in the library: panel-knowledge checks, +and `spillover_ratio`, which needs centroids instead. The split is an internal +detail of the package; the ids and their order here are unchanged. +""" + +import argparse + +import anndata as ad +import numpy as np +import sprotmetrics as spm + +QC_COLUMN = "passed_QC" + + +def _clean(value): + """Treat Viash's 'null' placeholder as absent.""" + if value is None: + return None + text = str(value).strip() + return None if text.lower() in {"", "null", "none"} else text + + +def run(par: dict): + layer = par.get("layer", "counts") + n_neighbors = int(par.get("n_neighbors", 6)) + max_cells = int(par.get("max_cells", 5000)) + random_state = int(par.get("random_state", 0)) + nuclear_markers = _clean(par.get("nuclear_markers")) + exclusive_pairs = _clean(par.get("exclusive_pairs")) + + adata = ad.read_h5ad(par["input"]) + scores = spm.segmentation_quality( + adata, + layer=layer, + nuclear_markers=nuclear_markers, + exclusive_pairs=exclusive_pairs, + n_neighbors=n_neighbors, + max_cells=max_cells, + random_state=random_state, + qc_variants=True, + qc_column=QC_COLUMN, + ) + + markers = [str(m) for m in adata.var_names] + resolved_nuclear = [markers[i] for i in spm.resolve_nuclear_markers(markers, nuclear_markers)] + # Kept as "A:B" strings, which is the shape downstream readers already expect. + resolved_pairs = [ + f"{markers[i]}:{markers[j]}" + for i, j in spm.resolve_exclusive_pairs(markers, exclusive_pairs) + ] + + out = ad.AnnData( + uns={ + "metric_ids": np.array(list(scores), dtype=object), + "metric_values": np.array(list(scores.values()), dtype=float), + "metric_group": "segmentation_quality", + "evaluated_layer": layer, + "nuclear_markers": np.array(resolved_nuclear, dtype=object), + "exclusive_pairs": np.array(resolved_pairs, dtype=object), + "n_neighbors": n_neighbors, + "qc_column": QC_COLUMN, + } + ) + out.write_h5ad(par["output"], compression="gzip") + + +## VIASH START +par = { + "input": "cell_by_protein_norm.h5ad", + "layer": "counts", + "nuclear_markers": None, + "exclusive_pairs": None, + "n_neighbors": 6, + "max_cells": 5000, + "random_state": 0, + "output": "metrics_segmentation_quality.h5ad", +} +meta = {"name": "segmentation_quality"} +## VIASH END + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--layer", default="counts") + ap.add_argument("--nuclear_markers", default=None) + ap.add_argument("--exclusive_pairs", default=None) + ap.add_argument("--n_neighbors", type=int, default=6) + ap.add_argument("--max_cells", type=int, default=5000) + ap.add_argument("--random_state", type=int, default=0) + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + main() diff --git a/src/metrics_proteomics/signal_background/__pycache__/script.cpython-312.pyc b/src/metrics_proteomics/signal_background/__pycache__/script.cpython-312.pyc new file mode 100644 index 0000000..b838479 Binary files /dev/null and b/src/metrics_proteomics/signal_background/__pycache__/script.cpython-312.pyc differ diff --git a/src/metrics_proteomics/signal_background/__pycache__/script.cpython-313.pyc b/src/metrics_proteomics/signal_background/__pycache__/script.cpython-313.pyc new file mode 100644 index 0000000..a56a8d6 Binary files /dev/null and b/src/metrics_proteomics/signal_background/__pycache__/script.cpython-313.pyc differ diff --git a/src/metrics_proteomics/signal_background/config.vsh.yaml b/src/metrics_proteomics/signal_background/config.vsh.yaml new file mode 100644 index 0000000..cdc1ec7 --- /dev/null +++ b/src/metrics_proteomics/signal_background/config.vsh.yaml @@ -0,0 +1,261 @@ +__merge__: /src/api/comp_metric_proteomics_signal_background.yaml +name: signal_background +label: Signal versus background metrics +summary: Pixel-level scores for image correction and mask placement +description: "The only metrics computed from pixels rather than the aggregated matrix, and\nso the only\ + \ ones that can see what the cell table cannot: signal that never\nmade it inside a mask, and background\ + \ that correction failed to remove.\n\n- `median_rel_sni` / `min_rel_sni` - area between the sorted\ + \ per-cell\n intensity curve and a rank-matched background curve, normalised by a high\n percentile\ + \ of cell intensity. The headline signal-to-background measure.\n Higher is better; `min` exposes the\ + \ worst marker in the panel.\n- `median_intracell_coverage` - among the brightest 5% of cells, the fraction\n\ + \ of their own pixels above threshold. Separates real staining, which is\n bright *and* extensive,\ + \ from punctate artefact, which is bright and sparse.\n Higher is better.\n- `median_signal_to_background_ratio`\ + \ - the simple mean-inside over\n mean-in-ring version. Higher is better.\n- `median_cellular_specificity`\ + \ - positive pixels inside the masks as a share\n of positive pixels inside plus in the ring around\ + \ them. Higher is better.\n- `median_marker_gini` - concentration of per-cell intensity across cells.\n\ + \ Descriptive rather than directional: high means a marker is restricted to\n few cells, which is\ + \ right for a lineage marker and wrong for a housekeeper.\n\nBackground is sampled from a ring immediately\ + \ outside the masks, not from all\nnon-cell pixels, so the comparison stays local instead of being diluted\ + \ by\nempty slide.\n\nAdapted from the scverse hackathon `sprot` prototype\n(`metrics/protein_quality_metrics.py`).\ + \ Three changes were needed: per-cell\nstatistics are computed by `bincount` instead of masking the\ + \ whole image once\nper cell, positivity uses an Otsu threshold instead of the `thresh=0` default\n\ + that made coverage identically 1.0, and background sampling is seeded so the\nresult is reproducible.\n" +resources: +- type: python_script + path: script.py +engines: +- type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - sprotmetrics==0.1.0 + - anndata>=0.10.0 + - zarr<3.0.0 +- type: native +runners: +- type: executable +info: + metrics: + - name: median_rel_sni + label: Signal integral against a rank-matched background + summary: Area between the sorted per-cell intensity curve and a rank-matched background curve. + description: "Area between the sorted per-cell intensity curve and a rank-matched background curve.\n\ + \n Normalised by a high percentile of cell intensity so markers of different\n brightness\ + \ are comparable. The headline signal-to-background measure." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: '-.inf' + max: '+.inf' + maximize: true + - name: median_rel_sni_qc_pass + label: Signal integral against a rank-matched background + summary: Area between the sorted per-cell intensity curve and a rank-matched background curve. + description: "Area between the sorted per-cell intensity curve and a rank-matched background curve.\n\ + \n Normalised by a high percentile of cell intensity so markers of different\n brightness\ + \ are comparable. The headline signal-to-background measure." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: '-.inf' + max: '+.inf' + maximize: true + - name: min_rel_sni + label: Signal integral against a rank-matched background + summary: Area between the sorted per-cell intensity curve and a rank-matched background curve. + description: "Area between the sorted per-cell intensity curve and a rank-matched background curve.\n\ + \n Normalised by a high percentile of cell intensity so markers of different\n brightness\ + \ are comparable. The headline signal-to-background measure." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: '-.inf' + max: '+.inf' + maximize: true + - name: min_rel_sni_qc_pass + label: Signal integral against a rank-matched background + summary: Area between the sorted per-cell intensity curve and a rank-matched background curve. + description: "Area between the sorted per-cell intensity curve and a rank-matched background curve.\n\ + \n Normalised by a high percentile of cell intensity so markers of different\n brightness\ + \ are comparable. The headline signal-to-background measure." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: '-.inf' + max: '+.inf' + maximize: true + - name: median_intracell_coverage + label: Extent of staining within the brightest cells + summary: Fraction of its own pixels above threshold, in the most-covered cells, per marker. + description: "Fraction of its own pixels above threshold, in the most-covered cells, per marker.\n\ + \n Separates real staining, which is bright *and* extensive, from punctate\n artefact, which\ + \ is bright and sparse." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: median_intracell_coverage_qc_pass + label: Extent of staining within the brightest cells + summary: Fraction of its own pixels above threshold, in the most-covered cells, per marker. + description: "Fraction of its own pixels above threshold, in the most-covered cells, per marker.\n\ + \n Separates real staining, which is bright *and* extensive, from punctate\n artefact, which\ + \ is bright and sparse." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: median_signal_to_background_ratio + label: Mean inside masks over mean in the ring + summary: Mean intensity inside the masks over mean intensity in the surrounding ring. + description: "Mean intensity inside the masks over mean intensity in the surrounding ring.\n\n \ + \ Needs no pseudo-cell sampling, so it survives the cases where the\n rank-matched curve of :func:`rel_sni`\ + \ cannot be built.\n\n The division is carried out in the image's own dtype. Multiplexed images\ + \ are\n stored float32, and under NumPy's weak scalar promotion the benchmark's\n ``np.mean(image[c][labels\ + \ > 0]) / ring_mean`` stays float32; doing it in\n float64 is marginally more accurate but shifts\ + \ the result in the eighth\n significant figure, which would silently move published numbers." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: median_signal_to_background_ratio_qc_pass + label: Mean inside masks over mean in the ring + summary: Mean intensity inside the masks over mean intensity in the surrounding ring. + description: "Mean intensity inside the masks over mean intensity in the surrounding ring.\n\n \ + \ Needs no pseudo-cell sampling, so it survives the cases where the\n rank-matched curve of :func:`rel_sni`\ + \ cannot be built.\n\n The division is carried out in the image's own dtype. Multiplexed images\ + \ are\n stored float32, and under NumPy's weak scalar promotion the benchmark's\n ``np.mean(image[c][labels\ + \ > 0]) / ring_mean`` stays float32; doing it in\n float64 is marginally more accurate but shifts\ + \ the result in the eighth\n significant figure, which would silently move published numbers." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: '+.inf' + maximize: true + - name: median_cellular_specificity + label: Share of positive pixels falling inside masks + summary: Positive pixels inside the masks, over positive pixels inside plus in the ring. + description: Positive pixels inside the masks, over positive pixels inside plus in the ring. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: median_cellular_specificity_qc_pass + label: Share of positive pixels falling inside masks + summary: Positive pixels inside the masks, over positive pixels inside plus in the ring. + description: Positive pixels inside the masks, over positive pixels inside plus in the ring. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: nuclear_signal_containment + label: Nuclear signal falling inside any mask + summary: Fraction of total nuclear-channel signal that falls inside a mask. + description: "Fraction of total nuclear-channel signal that falls inside a mask.\n\n The only way\ + \ to see cells the segmentation missed entirely: every metric\n computed from the cell table\ + \ is blind to them, because a missed cell is\n simply absent from the table." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: nuclear_signal_containment_qc_pass + label: Nuclear signal falling inside any mask + summary: Fraction of total nuclear-channel signal that falls inside a mask. + description: "Fraction of total nuclear-channel signal that falls inside a mask.\n\n The only way\ + \ to see cells the segmentation missed entirely: every metric\n computed from the cell table\ + \ is blind to them, because a missed cell is\n simply absent from the table." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: median_signal_drop_on_aggregation + label: Positive cells aggregated away to zero + summary: Cells holding positive pixels whose aggregated value is nonetheless zero, per marker. + description: "Cells holding positive pixels whose aggregated value is nonetheless zero, per marker.\n\ + \n Aggregation is the axis the benchmark covers worst, and this asks the\n question directly:\ + \ did summarising the pixels throw the signal away?\n\n Rows of `table` are aligned to label\ + \ ids through ``obs[cell_id_key]`` when\n given, else through integer-castable ``obs_names``,\ + \ else positionally -- and\n a positional fallback whose length disagrees raises rather than\ + \ scoring a\n silently mismatched join." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: median_signal_drop_on_aggregation_qc_pass + label: Positive cells aggregated away to zero + summary: Cells holding positive pixels whose aggregated value is nonetheless zero, per marker. + description: "Cells holding positive pixels whose aggregated value is nonetheless zero, per marker.\n\ + \n Aggregation is the axis the benchmark covers worst, and this asks the\n question directly:\ + \ did summarising the pixels throw the signal away?\n\n Rows of `table` are aligned to label\ + \ ids through ``obs[cell_id_key]`` when\n given, else through integer-castable ``obs_names``,\ + \ else positionally -- and\n a positional fallback whose length disagrees raises rather than\ + \ scoring a\n silently mismatched join." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: max_signal_drop_on_aggregation + label: Positive cells aggregated away to zero + summary: Cells holding positive pixels whose aggregated value is nonetheless zero, per marker. + description: "Cells holding positive pixels whose aggregated value is nonetheless zero, per marker.\n\ + \n Aggregation is the axis the benchmark covers worst, and this asks the\n question directly:\ + \ did summarising the pixels throw the signal away?\n\n Rows of `table` are aligned to label\ + \ ids through ``obs[cell_id_key]`` when\n given, else through integer-castable ``obs_names``,\ + \ else positionally -- and\n a positional fallback whose length disagrees raises rather than\ + \ scoring a\n silently mismatched join." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: max_signal_drop_on_aggregation_qc_pass + label: Positive cells aggregated away to zero + summary: Cells holding positive pixels whose aggregated value is nonetheless zero, per marker. + description: "Cells holding positive pixels whose aggregated value is nonetheless zero, per marker.\n\ + \n Aggregation is the axis the benchmark covers worst, and this asks the\n question directly:\ + \ did summarising the pixels throw the signal away?\n\n Rows of `table` are aligned to label\ + \ ids through ``obs[cell_id_key]`` when\n given, else through integer-castable ``obs_names``,\ + \ else positionally -- and\n a positional fallback whose length disagrees raises rather than\ + \ scoring a\n silently mismatched join." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false diff --git a/src/metrics_proteomics/signal_background/script.py b/src/metrics_proteomics/signal_background/script.py new file mode 100644 index 0000000..3e3a493 --- /dev/null +++ b/src/metrics_proteomics/signal_background/script.py @@ -0,0 +1,114 @@ +"""Pixel-level signal-versus-background metrics. Thin wrapper over `sprotmetrics`. + +The only metric component that reads pixels rather than the cell table. The +expensive pass over the image is `sprotmetrics.cell_pixel_stats`, done once +here so the reported thresholds and object counts describe the same pass the +metrics were computed from. +""" + +import argparse +import json + +import anndata as ad +import numpy as np +import sprotmetrics as spm +from sprotmetrics._signal import qc_mask_from_table, signal_background_from_stats + +QC_COLUMN = "passed_QC" + + +def _load_markers(path, n_channels): + if not path: + return [f"marker_{i}" for i in range(n_channels)] + with open(path) as handle: + payload = json.load(handle) + markers = payload["markers"] if isinstance(payload, dict) else payload + markers = [str(m) for m in markers] + if len(markers) != n_channels: + raise ValueError(f"got {len(markers)} marker names for {n_channels} channels") + return markers + + +def run(par: dict): + image = np.load(par["input_image"]) + if image.ndim == 2: + image = image[None, :, :] + labels = np.load(par["input_labels"]) + markers = _load_markers(par.get("input_markers"), image.shape[0]) + + dilation_iterations = int(par.get("dilation_iterations", 3)) + norm_percentile = float(par.get("norm_percentile", 95.0)) + random_state = int(par.get("random_state", 0)) + + stats = spm.cell_pixel_stats( + image, + labels, + markers, + dilation_iterations=dilation_iterations, + random_state=random_state, + ) + + # QC lives in the cell table, not in the pixels, so it has to be carried + # across by label id. + qc_mask = None + if par.get("input_qc"): + qc_mask = qc_mask_from_table( + ad.read_h5ad(par["input_qc"]), stats.means.shape[0], QC_COLUMN + ) + if qc_mask is None: + qc_mask = np.zeros(stats.means.shape[0], dtype=bool) + + scores = signal_background_from_stats( + stats, norm_percentile=norm_percentile, qc_mask=qc_mask + ) + + out = ad.AnnData( + uns={ + "metric_ids": np.array(list(scores), dtype=object), + "metric_values": np.array(list(scores.values()), dtype=float), + "metric_group": "signal_background", + "markers": np.array(markers, dtype=object), + "otsu_thresholds": stats.thresholds.astype(float), + "dilation_iterations": dilation_iterations, + "norm_percentile": norm_percentile, + "n_objects": int(stats.present.sum()), + "n_background_samples": int(stats.n_background_pseudocells), + "qc_column": QC_COLUMN, + } + ) + out.write_h5ad(par["output"], compression="gzip") + + +## VIASH START +par = { + "input_image": "corrected_image.npy", + "input_labels": "segmentation_labels.npy", + "input_markers": "corrected_markers.json", + "input_qc": None, + "dilation_iterations": 3, + "norm_percentile": 95.0, + "random_state": 0, + "output": "metrics_signal_background.h5ad", +} +meta = {"name": "signal_background"} +## VIASH END + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--input_image", required=True) + ap.add_argument("--input_labels", required=True) + ap.add_argument("--input_markers", default=None) + ap.add_argument("--input_qc", default=None) + ap.add_argument("--dilation_iterations", type=int, default=3) + ap.add_argument("--norm_percentile", type=float, default=95.0) + ap.add_argument("--random_state", type=int, default=0) + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + main() diff --git a/src/metrics_proteomics/technical_covariates/__pycache__/script.cpython-312.pyc b/src/metrics_proteomics/technical_covariates/__pycache__/script.cpython-312.pyc new file mode 100644 index 0000000..350e9f6 Binary files /dev/null and b/src/metrics_proteomics/technical_covariates/__pycache__/script.cpython-312.pyc differ diff --git a/src/metrics_proteomics/technical_covariates/__pycache__/script.cpython-313.pyc b/src/metrics_proteomics/technical_covariates/__pycache__/script.cpython-313.pyc new file mode 100644 index 0000000..1257168 Binary files /dev/null and b/src/metrics_proteomics/technical_covariates/__pycache__/script.cpython-313.pyc differ diff --git a/src/metrics_proteomics/technical_covariates/config.vsh.yaml b/src/metrics_proteomics/technical_covariates/config.vsh.yaml new file mode 100644 index 0000000..0e27b5a --- /dev/null +++ b/src/metrics_proteomics/technical_covariates/config.vsh.yaml @@ -0,0 +1,223 @@ +__merge__: /src/api/comp_metric_proteomics_technical_covariates.yaml +name: technical_covariates +label: Technical covariate metrics +summary: How much signal is still explained by cell area and by marker brightness +description: "Aggregation and normalization should leave per-cell profiles reflecting\nbiology rather\ + \ than how large a cell was or how bright its channel is.\n\nThe area metrics are reported on **both**\ + \ layers, suffixed `_counts` and\n`_normalized`. The raw layer asks whether aggregation introduced area\n\ + dependence; the normalized layer asks whether normalization removed it. Only\nthe raw layer can see\ + \ the aggregation axis: per-cell total normalization\ndivides out exactly the area scaling that `sum`\ + \ aggregation creates, so\nevaluating the normalized layer alone reports `mean` and `sum` as identical.\n\ + \n- `variance_explained_by_covariate_{counts,normalized}` - mean R² of\n expression regressed on cell\ + \ area, linear and logged. Lower is better.\n- `max_marker_variance_explained_by_covariate_{counts,normalized}`\ + \ - the worst\n marker, so a single area-driven channel is not hidden by the panel average.\n Lower\ + \ is better.\n- `covariate_pc_correlation_{counts,normalized}` - mean absolute correlation\n between\ + \ the leading principal components and area. Lower is better.\n- `mean_variance_correlation` - dependence\ + \ of per-marker variance on\n per-marker mean, which a variance-stabilising transform should remove.\n\ + \ Lower is better.\n- `marker_variance_evenness` - Shannon entropy of the per-marker variance\n shares,\ + \ normalised to 0-1. High means variance is spread across the panel;\n low means a few bright markers\ + \ dominate. Higher is better.\n\nAdapted from the scverse hackathon `sprot` prototype\n(`metrics/technical_correction.py`\ + \ and `metrics/protein_contribution.py`).\nCell area replaces the single-cell covariates that prototype\ + \ assumed\n(`total_counts`, `pct_mito`, `batch`), none of which exist here; PCA is done\nby SVD rather\ + \ than scikit-learn to avoid the extra dependency.\n" +resources: +- type: python_script + path: script.py +engines: +- type: docker + image: openproblems/base_python:1 + setup: + - type: python + pypi: + - sprotmetrics==0.1.0 + - anndata>=0.10.0 + - zarr<3.0.0 +- type: native +runners: +- type: executable +info: + metrics: + - name: mean_variance_explained_by_covariate_counts + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_variance_explained_by_covariate_counts_qc_pass + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_variance_explained_by_covariate_normalized + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_variance_explained_by_covariate_normalized_qc_pass + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: max_variance_explained_by_covariate_counts + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: max_variance_explained_by_covariate_counts_qc_pass + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: max_variance_explained_by_covariate_normalized + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: max_variance_explained_by_covariate_normalized_qc_pass + label: Expression variance explained by cell area + summary: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + description: R^2 of each marker on cell area, fitted on ``[1, area, log1p(area)]``. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_covariate_pc_correlation_counts + label: Correlation of leading PCs with cell area + summary: Absolute correlation of each leading principal component with cell area. + description: Absolute correlation of each leading principal component with cell area. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_covariate_pc_correlation_counts_qc_pass + label: Correlation of leading PCs with cell area + summary: Absolute correlation of each leading principal component with cell area. + description: Absolute correlation of each leading principal component with cell area. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_covariate_pc_correlation_normalized + label: Correlation of leading PCs with cell area + summary: Absolute correlation of each leading principal component with cell area. + description: Absolute correlation of each leading principal component with cell area. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_covariate_pc_correlation_normalized_qc_pass + label: Correlation of leading PCs with cell area + summary: Absolute correlation of each leading principal component with cell area. + description: Absolute correlation of each leading principal component with cell area. + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: false + - name: mean_variance_correlation + label: Dependence of marker variance on marker mean + summary: Correlation between per-marker mean and per-marker variance. + description: "Correlation between per-marker mean and per-marker variance.\n\n Tests whether the\ + \ transform stabilised variance: a strong positive\n correlation means bright markers are still\ + \ the variable ones." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: -1 + max: 1 + maximize: false + - name: mean_variance_correlation_qc_pass + label: Dependence of marker variance on marker mean + summary: Correlation between per-marker mean and per-marker variance. + description: "Correlation between per-marker mean and per-marker variance.\n\n Tests whether the\ + \ transform stabilised variance: a strong positive\n correlation means bright markers are still\ + \ the variable ones." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: -1 + max: 1 + maximize: false + - name: marker_variance_evenness + label: Evenness of variance across the panel + summary: Shannon entropy of each marker's share of total variance, normalised to [0, 1]. + description: "Shannon entropy of each marker's share of total variance, normalised to [0, 1].\n\n\ + \ Low means a few bright markers dominate the variance and the rest contribute\n nothing to\ + \ any downstream embedding." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true + - name: marker_variance_evenness_qc_pass + label: Evenness of variance across the panel + summary: Shannon entropy of each marker's share of total variance, normalised to [0, 1]. + description: "Shannon entropy of each marker's share of total variance, normalised to [0, 1].\n\n\ + \ Low means a few bright markers dominate the variance and the rest contribute\n nothing to\ + \ any downstream embedding." + references: {} + links: + repository: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing + docs: https://github.com/openproblems-bio/task_spatial_proteomics_preprocessing#readme + min: 0 + max: 1 + maximize: true diff --git a/src/metrics_proteomics/technical_covariates/script.py b/src/metrics_proteomics/technical_covariates/script.py new file mode 100644 index 0000000..8361ee1 --- /dev/null +++ b/src/metrics_proteomics/technical_covariates/script.py @@ -0,0 +1,78 @@ +"""Residual dependence on cell area. Thin wrapper over `sprotmetrics`.""" + +import argparse + +import anndata as ad +import numpy as np +import sprotmetrics as spm + +QC_COLUMN = "passed_QC" + + +def run(par: dict): + layer = par.get("layer", "normalized") + covariate = par.get("covariate", "area") + n_pcs = int(par.get("n_pcs", 10)) + max_cells = int(par.get("max_cells", 5000)) + random_state = int(par.get("random_state", 0)) + + adata = ad.read_h5ad(par["input"]) + scores = spm.covariate_dependence( + adata, + covariate, + layer=layer, + counts_layer="counts", + n_pcs=n_pcs, + max_cells=max_cells, + random_state=random_state, + qc_variants=True, + qc_column=QC_COLUMN, + ) + + out = ad.AnnData( + uns={ + "metric_ids": np.array(list(scores), dtype=object), + "metric_values": np.array(list(scores.values()), dtype=float), + "metric_group": "technical_covariates", + "evaluated_layer": layer, + "covariate": covariate, + "covariate_present": covariate in adata.obs.columns, + "counts_layer_present": "counts" in adata.layers, + "n_pcs": n_pcs, + "qc_column": QC_COLUMN, + } + ) + out.write_h5ad(par["output"], compression="gzip") + + +## VIASH START +par = { + "input": "cell_by_protein_norm.h5ad", + "layer": "normalized", + "covariate": "area", + "n_pcs": 10, + "max_cells": 5000, + "random_state": 0, + "output": "metrics_technical_covariates.h5ad", +} +meta = {"name": "technical_covariates"} +## VIASH END + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--input", required=True) + ap.add_argument("--layer", default="normalized") + ap.add_argument("--covariate", default="area") + ap.add_argument("--n_pcs", type=int, default=10) + ap.add_argument("--max_cells", type=int, default=5000) + ap.add_argument("--random_state", type=int, default=0) + ap.add_argument("--output", required=True) + run(vars(ap.parse_args())) + + +if __name__ == "__main__": + if "par" in globals(): + run(par) + else: + main() diff --git a/src/testing/isp_fixtures.py b/src/testing/isp_fixtures.py new file mode 100644 index 0000000..9334f26 --- /dev/null +++ b/src/testing/isp_fixtures.py @@ -0,0 +1,184 @@ +"""Tiny synthetic fixtures for ISP component smoke tests. + +The ISP benchmark was developed against a single large MCMICRO dataset that is +not redistributable and takes minutes to process. These helpers build +deliberately small, deterministic stand-ins (64x64 pixels, 3 channels) so that +every component can be exercised in CI without any external resource. + +Imports are kept local to each helper: component images only install the +dependencies that component needs, so a segmentation test must not pay for +anndata and a QC test must not pay for spatialdata. +""" + +IMAGE_SHAPE = (4, 64, 64) +# Two nuclear replicates (the same nuclei imaged in two cycles, as multiplexed +# panels do) plus a mutually exclusive immune/epithelial pair. This shape is +# what lets the segmentation-quality metrics be tested at all. +MARKERS = ["DNA_1", "DNA_2", "CD45", "ECAD"] +NUCLEAR_MARKERS = ["DNA_1", "DNA_2"] +EXCLUSIVE_PAIR = ("CD45", "ECAD") + +# Blob centres used by both the image and the label fixtures, with a deliberate +# range of radii so that min_area filtering has something to remove. +_BLOBS = [(14, 14, 5), (14, 46, 5), (46, 14, 5), (46, 46, 4), (30, 30, 2)] + + +def build_image(seed: int = 0): + """Return a (3, 64, 64) float32 channel-y-x image with bright nuclear blobs.""" + import numpy as np + + rng = np.random.default_rng(seed) + img = (rng.random(IMAGE_SHAPE) * 10.0).astype(np.float32) + + yy, xx = np.ogrid[: IMAGE_SHAPE[1], : IMAGE_SHAPE[2]] + for cy, cx, radius in _BLOBS: + disk = (yy - cy) ** 2 + (xx - cx) ** 2 <= radius**2 + # Both nuclear channels light up the same blobs, as replicate stains of + # the same nuclei do, with a little independent noise so their + # correlation is high but not exactly 1. + img[0][disk] += 500.0 + img[1][disk] += 500.0 + rng.normal(0.0, 5.0, size=disk.sum()).astype(np.float32) + img[2][disk] += 120.0 # CD45, tracks the nuclei + # The last channel gets a smooth left-to-right illumination gradient so that + # flat-field correction has a low-frequency field to remove. + img[3] += np.linspace(0.0, 200.0, IMAGE_SHAPE[2], dtype=np.float32)[None, :] + + return img + + +def build_labels(): + """Return a (64, 64) int32 label mask with gapless labels 1..n.""" + import numpy as np + + labels = np.zeros(IMAGE_SHAPE[1:], dtype=np.int32) + yy, xx = np.ogrid[: IMAGE_SHAPE[1], : IMAGE_SHAPE[2]] + for i, (cy, cx, radius) in enumerate(_BLOBS, start=1): + labels[(yy - cy) ** 2 + (xx - cx) ** 2 <= radius**2] = i + return labels + + +def make_image(path, seed: int = 0): + """Write the fixture image to a .npy file.""" + import numpy as np + + img = build_image(seed=seed) + np.save(path, img) + return img + + +def make_labels(path): + """Write the fixture label mask to a .npy file.""" + import numpy as np + + labels = build_labels() + np.save(path, labels) + return labels + + +def make_markers_json(path, markers=None): + """Write the marker sidecar produced by the image-correction components.""" + import json + + markers = list(markers) if markers is not None else list(MARKERS) + with open(path, "w") as f: + json.dump({"method": "fixture", "image_key": "fixture_image", "markers": markers}, f) + return markers + + +def make_cell_by_protein( + path, + n_cells: int = 40, + seed: int = 0, + with_normalized: bool = False, + with_qc: bool = False, + with_celltypes: bool = False, +): + """Write a cell-by-protein AnnData matching the aggregation output contract.""" + import anndata as ad + import numpy as np + import pandas as pd + + rng = np.random.default_rng(seed) + X = (rng.lognormal(mean=3.0, sigma=1.0, size=(n_cells, len(MARKERS)))).astype(np.float32) + + # DNA_2 is the same nuclei restained, so it tracks DNA_1 closely. Leaving + # them independent would make replicate concordance read ~0 on data that is + # by construction fine. + i0, i1 = MARKERS.index("DNA_1"), MARKERS.index("DNA_2") + X[:, i1] = X[:, i0] * rng.normal(1.0, 0.05, size=n_cells).astype(np.float32) + X[:, i1] = np.abs(X[:, i1]) + + # Applied before anything is derived from X, so counts, the normalized + # layer and the QC totals all describe the same matrix. + cell_types = None + if with_celltypes: + # Two groups that differ in marker *composition*, not just in overall + # brightness. A uniform intensity shift would be largely undone by + # per-cell total normalization, which is the layer the separability + # metric scores, so the groups would not actually be separable there. + group = rng.integers(0, 2, size=n_cells) + X[group == 1, MARKERS.index("CD45")] *= 20.0 + X[group == 0, MARKERS.index("ECAD")] *= 20.0 + cell_types = np.where(group == 1, "type_a", "type_b") + + # A quarter of the cells are debris: small *and* dim, which is what QC + # exists to catch. Making this a fixed fraction rather than a lucky draw + # keeps QC-sensitive assertions deterministic - with areas drawn uniformly + # the number falling below a min_area of 50 varied with the RNG stream, and + # at one point only 2 of 40 cells were rejected. + n_debris = max(1, n_cells // 4) + is_debris = np.zeros(n_cells, dtype=bool) + is_debris[:n_debris] = True + X[is_debris] *= 0.05 + + area = np.empty(n_cells, dtype=float) + area[is_debris] = rng.integers(5, 45, size=n_debris) + area[~is_debris] = rng.integers(60, 400, size=n_cells - n_debris) + + obs = pd.DataFrame( + { + "cell_id": [str(i + 1) for i in range(n_cells)], + "area": area, + "is_debris": is_debris, + "centroid_y": rng.random(n_cells) * 64.0, + "centroid_x": rng.random(n_cells) * 64.0, + }, + index=[str(i + 1) for i in range(n_cells)], + ) + + adata = ad.AnnData(X=X, obs=obs, var=pd.DataFrame(index=list(MARKERS))) + adata.layers["counts"] = X.copy() + if cell_types is not None: + adata.obs["cell_type"] = cell_types + if with_qc: + # Stand in for the QC step: annotate, do not filter. + adata.obs["passed_QC"] = (adata.obs["area"] >= 50).to_numpy() + adata.obs["total_intensity"] = X.sum(axis=1).astype(float) + if with_normalized: + totals = X.sum(axis=1, keepdims=True) + totals[totals <= 0] = 1.0 + adata.layers["normalized"] = np.log1p(X / totals * 1e4).astype(np.float32) + adata.uns["aggregation"] = {"method": "fixture", "statistic": "mean"} + + adata.write_h5ad(path) + return adata + + +def make_spatialdata_zarr(path, seed: int = 0): + """Write a minimal SpatialData store with one image and one labels element.""" + import spatialdata as sd + from spatialdata.models import Image2DModel, Labels2DModel + + img = build_image(seed=seed) + labels = build_labels() + + sdata = sd.SpatialData( + images={ + "fixture_image": Image2DModel.parse( + img, dims=("c", "y", "x"), c_coords=list(MARKERS) + ) + }, + labels={"fixture_cell": Labels2DModel.parse(labels, dims=("y", "x"))}, + ) + sdata.write(path) + return sdata diff --git a/src/testing/test_aggregation.py b/src/testing/test_aggregation.py new file mode 100644 index 0000000..f0907cf --- /dev/null +++ b/src/testing/test_aggregation.py @@ -0,0 +1,99 @@ +"""Smoke test shared by every methods_protein_aggregation component.""" + +import subprocess +import sys +from pathlib import Path + +import anndata as ad +import numpy as np + +## VIASH START +meta = { + "name": "mean_intensity_aggregation", + "executable": "target/executable/methods_protein_aggregation/mean_intensity_aggregation/mean_intensity_aggregation", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +image_path = work / "corrected_image.npy" +labels_path = work / "segmentation_labels.npy" +markers_path = work / "corrected_markers.json" +output_path = work / "cell_by_protein.h5ad" + +image = isp_fixtures.make_image(image_path) +labels = isp_fixtures.make_labels(labels_path) +markers = isp_fixtures.make_markers_json(markers_path) +n_objects = int(labels.max()) + +print(f">> Running {name}", flush=True) +subprocess.run( + [ + meta["executable"], + "--input_image", str(image_path), + "--input_labels", str(labels_path), + "--input_markers", str(markers_path), + "--output", str(output_path), + ], + check=True, +) + +print(">> Checking output file exists", flush=True) +assert output_path.exists(), "cell_by_protein.h5ad was not written" + +print(">> Checking the cell-by-protein contract", flush=True) +adata = ad.read_h5ad(output_path) +assert adata.shape == (n_objects, len(markers)), ( + f"expected {n_objects} cells x {len(markers)} markers, got {adata.shape}" +) +assert list(adata.var_names) == markers, f"marker names were not preserved: {list(adata.var_names)}" +assert "counts" in adata.layers, "layers['counts'] is required by the downstream QC and metric components" +assert np.allclose(adata.layers["counts"], adata.X), "layers['counts'] should mirror X on aggregation output" +assert np.isfinite(adata.X).all(), "aggregated intensities contain NaN or infinite values" +assert (adata.X >= 0).all(), "aggregated intensities should not be negative" + +for col in ["cell_id", "area", "centroid_y", "centroid_x"]: + assert col in adata.obs.columns, f"obs['{col}'] is required by downstream components" + +print(">> Checking areas against the fixture", flush=True) +expected_areas = np.bincount(labels.ravel())[1:] +assert np.array_equal(adata.obs["area"].to_numpy().astype(int), expected_areas), ( + "reported cell areas do not match the fixture label mask" +) + +print(">> Checking the aggregation statistic", flush=True) +statistic = adata.uns["aggregation"]["statistic"] +assert adata.uns["aggregation"]["method"] == name + +# Recompute the expected value for the first object and first channel directly +# from the fixture, so a mislabelled statistic cannot pass. +pixels = image[0][labels == 1] +expected = {"mean": np.mean, "sum": np.sum, "median": np.median}[statistic](pixels) +assert np.isclose(adata.X[0, 0], expected, rtol=1e-4), ( + f"{name} reports statistic '{statistic}' but the value for cell 1, marker 0 " + f"is {adata.X[0, 0]}, expected {expected}" +) + +print(">> Rejecting an image/labels shape mismatch", flush=True) +bad_labels_path = work / "bad_labels.npy" +np.save(bad_labels_path, labels[:32, :32]) +bad = subprocess.run( + [ + meta["executable"], + "--input_image", str(image_path), + "--input_labels", str(bad_labels_path), + "--input_markers", str(markers_path), + "--output", str(work / "bad.h5ad"), + ], + capture_output=True, + text=True, +) +assert bad.returncode != 0, "aggregating an image and label mask of different shapes must fail" + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_aggregation_zarr.py b/src/testing/test_aggregation_zarr.py new file mode 100644 index 0000000..0d7b0a5 --- /dev/null +++ b/src/testing/test_aggregation_zarr.py @@ -0,0 +1,82 @@ +"""Smoke test for the SpatialData-based protein aggregation component. + +This is the older aggregation interface, which reads the image and the label +mask straight from a SpatialData store instead of from .npy arrays. The +benchmark workflow uses the array-based components (see test_aggregation.py); +this one is covered so that both interfaces stay runnable. +""" + +import subprocess +import sys +from pathlib import Path + +import anndata as ad +import numpy as np + +## VIASH START +meta = { + "name": "basic_protein_aggregation", + "executable": "target/executable/methods_protein_aggregation/basic_protein_aggregation/basic_protein_aggregation", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +zarr_path = work / "fixture.zarr" +output_path = work / "cell_by_protein.h5ad" + +print(">> Building a SpatialData fixture", flush=True) +isp_fixtures.make_spatialdata_zarr(zarr_path) +labels = isp_fixtures.build_labels() +n_objects = int(labels.max()) + +print(f">> Running {name}", flush=True) +subprocess.run( + [ + meta["executable"], + "--input_zarr", str(zarr_path), + "--image_key", "fixture_image", + "--labels_key", "fixture_cell", + "--stat", "mean", + "--output", str(output_path), + ], + check=True, +) + +print(">> Checking output file exists", flush=True) +assert output_path.exists(), "cell_by_protein.h5ad was not written" + +adata = ad.read_h5ad(output_path) + +print(">> Checking the cell-by-protein contract", flush=True) +assert adata.shape == (n_objects, len(isp_fixtures.MARKERS)), ( + f"expected {n_objects} cells x {len(isp_fixtures.MARKERS)} markers, got {adata.shape}" +) +assert list(adata.var_names) == isp_fixtures.MARKERS, ( + f"marker names were not preserved: {list(adata.var_names)}" +) +assert "counts" in adata.layers, "layers['counts'] is required by the downstream components" +assert np.isfinite(adata.X).all(), "aggregated intensities contain NaN or infinite values" +assert (adata.X >= 0).all(), "aggregated intensities should not be negative" + +print(">> Checking uns['aggregation']", flush=True) +aggregation = adata.uns["aggregation"] +assert aggregation["method"] == name, ( + f"uns['aggregation']['method'] is {aggregation['method']}, expected {name}" +) +assert aggregation["statistic"] == "mean" + +print(">> Checking the aggregated values against the fixture", flush=True) +image = isp_fixtures.build_image() +expected = image[0][labels == 1].mean() +assert np.isclose(adata.X[0, 0], expected, rtol=1e-4), ( + f"mean intensity for cell 1, marker 0 is {adata.X[0, 0]}, expected {expected}" +) + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_image_correction.py b/src/testing/test_image_correction.py new file mode 100644 index 0000000..be52522 --- /dev/null +++ b/src/testing/test_image_correction.py @@ -0,0 +1,87 @@ +"""Smoke test shared by every methods_image_correction component.""" + +import json +import subprocess +import sys +from pathlib import Path + +import numpy as np + +## VIASH START +meta = { + "name": "basic_image_correction", + "executable": "target/executable/methods_image_correction/basic_image_correction/basic_image_correction", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +zarr_path = work / "fixture.zarr" +image_path = work / "corrected_image.npy" +markers_path = work / "corrected_markers.json" + +print(">> Building a SpatialData fixture", flush=True) +isp_fixtures.make_spatialdata_zarr(zarr_path) + +print(f">> Running {name}", flush=True) +subprocess.run( + [ + meta["executable"], + "--input_zarr", str(zarr_path), + "--image_key", "fixture_image", + "--output_image", str(image_path), + "--output_markers", str(markers_path), + ], + check=True, +) + +print(">> Checking output files exist", flush=True) +assert image_path.exists(), "corrected_image.npy was not written" +assert markers_path.exists(), "corrected_markers.json was not written" + +print(">> Checking corrected image", flush=True) +img = np.load(image_path) +assert img.shape == isp_fixtures.IMAGE_SHAPE, ( + f"expected channel-y-x shape {isp_fixtures.IMAGE_SHAPE}, got {img.shape}" +) +assert img.dtype == np.float32, f"expected float32, got {img.dtype}" +assert np.isfinite(img).all(), "corrected image contains NaN or infinite values" +assert img.min() >= 0.0, "corrected image contains negative intensities" +assert img.max() > 0.0, "corrected image is empty" + +print(">> Checking marker metadata", flush=True) +with markers_path.open() as f: + markers = json.load(f) +assert isinstance(markers, dict), "corrected_markers.json should be a JSON object" +assert markers["markers"] == isp_fixtures.MARKERS, ( + f"marker names were not preserved: {markers['markers']}" +) +assert markers["image_key"] == "fixture_image" +assert markers["method"] == name, f"metadata reports method {markers['method']}, expected {name}" + +print(">> Checking --rescale_max", flush=True) +rescaled_path = work / "rescaled.npy" +subprocess.run( + [ + meta["executable"], + "--input_zarr", str(zarr_path), + "--image_key", "fixture_image", + "--rescale_max", "true", + "--output_image", str(rescaled_path), + "--output_markers", str(work / "rescaled_markers.json"), + ], + check=True, +) +rescaled = np.load(rescaled_path) +per_channel_max = rescaled.reshape(rescaled.shape[0], -1).max(axis=1) +assert np.allclose(per_channel_max, 1.0, atol=1e-5), ( + f"--rescale_max should leave every channel at max 1, got {per_channel_max}" +) + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_metrics.py b/src/testing/test_metrics.py new file mode 100644 index 0000000..99aa92d --- /dev/null +++ b/src/testing/test_metrics.py @@ -0,0 +1,293 @@ +"""Smoke test shared by every metrics_proteomics component.""" + +import subprocess +import sys +from pathlib import Path + +import anndata as ad +import numpy as np + +## VIASH START +meta = { + "name": "basic", + "executable": "target/executable/metrics_proteomics/basic/basic", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +input_path = work / "cell_by_protein_norm.h5ad" +output_path = work / f"metrics_{name}.h5ad" + +# Metrics run at the end of the workflow, so the fixture carries everything the +# preceding steps would have added: counts, a normalized layer and QC columns. +# Both fixtures below must differ *only* in QC, or the comparison at the end of +# this script is meaningless. Keep this shared. +WITH_CELLTYPES = name in {"cluster_separability", "segmentation_quality"} +FIXTURE_CELLS = 200 + +original = isp_fixtures.make_cell_by_protein( + input_path, + # Large enough that the neighbour-based spillover metric has both adjacent + # and non-adjacent negative cells to compare; at 40 cells every cell has a + # positive neighbour and the metric is undefined. + n_cells=FIXTURE_CELLS, + with_normalized=True, + with_qc=True, + # cluster_separability scores obs["cell_type"]; segmentation_quality needs + # the mutually exclusive lineages that the cell types create. + with_celltypes=WITH_CELLTYPES, +) + +# Metric components take different arguments; only --input and --output are shared. +extra_args = { + "basic": ["--cov_thr", "0.1"], + "marker_distribution": [ + "--layer", "counts", + "--zero_threshold", "0.0", + "--coverage_threshold", "0.1", + "--low_quantile", "0.01", + "--high_quantile", "0.99", + ], + "normalization_diagnostics": [ + "--raw_layer", "counts", + "--normalized_layer", "normalized", + "--max_cells", "5000", + "--random_state", "0", + ], + "cluster_separability": [ + "--layer", "normalized", + "--max_cells", "5000", + "--random_state", "0", + ], + "segmentation_quality": [ + "--layer", "counts", + "--nuclear_markers", ",".join(isp_fixtures.NUCLEAR_MARKERS), + "--exclusive_pairs", ":".join(isp_fixtures.EXCLUSIVE_PAIR), + "--n_neighbors", "4", + "--random_state", "0", + ], + "technical_covariates": [ + "--layer", "normalized", + "--covariate", "area", + "--n_pcs", "3", + "--random_state", "0", + ], +} +assert name in extra_args, f"no argument set defined for metric component '{name}'" + +print(f">> Running {name}", flush=True) +subprocess.run( + [meta["executable"], "--input", str(input_path), "--output", str(output_path)] + extra_args[name], + check=True, +) + +print(">> Checking output file exists", flush=True) +assert output_path.exists(), f"metrics_{name}.h5ad was not written" + +out = ad.read_h5ad(output_path) + +print(">> Checking the metric output contract", flush=True) +assert "metric_ids" in out.uns, "uns['metric_ids'] is required by the metric API" +assert "metric_values" in out.uns, "uns['metric_values'] is required by the metric API" + +ids = [str(x) for x in out.uns["metric_ids"]] +values = np.asarray(out.uns["metric_values"], dtype=float) + +assert len(ids) == len(values), ( + f"uns['metric_ids'] has {len(ids)} entries but uns['metric_values'] has {len(values)}" +) +assert len(ids) > 0, "no metrics were reported" +assert len(set(ids)) == len(ids), f"duplicate metric ids reported: {ids}" +assert out.uns["metric_group"] == name, ( + f"uns['metric_group'] is {out.uns['metric_group']}, expected {name}" +) + +print(">> Checking metric values are usable", flush=True) +# The collection script writes these straight into the summary table, so a NaN +# here silently becomes an empty cell in the published results. +nan_metrics = [i for i, v in zip(ids, values) if not np.isfinite(v)] +assert not nan_metrics, f"metrics are not finite on the fixture: {nan_metrics}" + +print(">> Checking the family reports its headline metrics", flush=True) +expected_ids = { + "basic": ["n_cells", "n_markers", "qc_pass_rate", "median_area", "median_total_intensity"], + "marker_distribution": [ + "median_marker_cv", + "median_marker_zero_fraction", + "median_marker_coverage", + ], + "normalization_diagnostics": ["raw_total_cv", "normalized_total_cv", "total_cv_ratio_norm_over_raw"], + "cluster_separability": ["silhouette_cell_type"], + "segmentation_quality": [ + "dapi_positive_ratio", + "replicate_correlation", + "mutually_exclusive_coexpression_ratio", + "spillover_ratio", + ], + "technical_covariates": [ + "variance_explained_by_covariate_normalized", + "variance_explained_by_covariate_counts", + "covariate_pc_correlation_normalized", + "covariate_pc_correlation_counts", + "mean_variance_correlation", + "marker_variance_evenness", + ], +} +missing = [m for m in expected_ids[name] if m not in ids] +assert not missing, f"{name} did not report expected metrics: {missing}" + +reported = dict(zip(ids, values)) + +if name == "basic": + assert reported["n_cells"] == original.n_obs, ( + f"n_cells is {reported['n_cells']}, expected {original.n_obs}" + ) + assert reported["n_markers"] == original.n_vars + expected_pass_rate = float(original.obs["passed_QC"].mean()) + assert np.isclose(reported["qc_pass_rate"], expected_pass_rate), ( + f"qc_pass_rate is {reported['qc_pass_rate']}, expected {expected_pass_rate}" + ) + assert 0.0 <= reported["qc_pass_rate"] <= 1.0 + +if name == "marker_distribution": + for bounded in ["median_marker_zero_fraction", "median_marker_coverage"]: + assert 0.0 <= reported[bounded] <= 1.0, f"{bounded} is out of [0, 1]: {reported[bounded]}" + +if name == "normalization_diagnostics": + assert reported["raw_total_cv"] > 0, "the fixture has variable totals, so raw CV must be positive" + # The fixture's normalized layer is total-scaled, so normalization must + # compress the spread of per-cell totals rather than inflate it. + assert reported["total_cv_ratio_norm_over_raw"] < 1.0, ( + "total-intensity CV should shrink after total-count normalization, got ratio " + f"{reported['total_cv_ratio_norm_over_raw']}" + ) + +if name == "cluster_separability": + score = reported["silhouette_cell_type"] + assert -1.0 <= score <= 1.0, f"silhouette must lie in [-1, 1], got {score}" + # The fixture's two cell types differ in CD45/ECAD composition. The panel + # also carries two correlated nuclear replicates, shared between the types, + # which caps how separable they can look - so require clearly positive + # rather than near-perfect. + assert score > 0.3, f"expected clearly separated fixture clusters, got silhouette {score}" + +if name == "segmentation_quality": + # The fixture's two nuclear channels are the same signal restained, so + # concordance must be high and per-cell disagreement small. + assert reported["replicate_correlation"] > 0.8, ( + f"replicate_correlation should be high on replicate stains, got " + f"{reported['replicate_correlation']}" + ) + assert 0.0 <= reported["median_replicate_cv"] < 0.5 + assert 0.0 <= reported["dapi_positive_ratio"] <= 1.0 + # CD45-high and ECAD-high cells are disjoint by construction, so few cells + # should be co-positive. + assert 0.0 <= reported["mutually_exclusive_coexpression_ratio"] <= 1.0 + assert reported["mutually_exclusive_coexpression_ratio"] < 0.5, ( + "the fixture's lineages are disjoint, so co-expression should be low, got " + f"{reported['mutually_exclusive_coexpression_ratio']}" + ) + assert reported["spillover_ratio"] > 0 + +if name == "technical_covariates": + bounded = ["marker_variance_evenness"] + for suffix in ("normalized", "counts"): + bounded += [ + f"variance_explained_by_covariate_{suffix}", + f"max_marker_variance_explained_by_covariate_{suffix}", + f"covariate_pc_correlation_{suffix}", + ] + for b in bounded: + assert 0.0 <= reported[b] <= 1.0, f"{b} out of [0, 1]: {reported[b]}" + assert -1.0 <= reported["mean_variance_correlation"] <= 1.0 + for suffix in ("normalized", "counts"): + assert reported[f"max_marker_variance_explained_by_covariate_{suffix}"] >= reported[ + f"variance_explained_by_covariate_{suffix}" + ], "the worst marker cannot explain less variance than the panel mean" + +print(">> Checking the QC-passing variants", flush=True) +# QC components annotate rather than filter, so without a second pass over the +# QC-passing cells every metric here would be identical whichever QC method ran. +paired = [i for i in ids if i.endswith("_qc_pass")] +assert paired, f"{name} reported no _qc_pass variants" + +# Metrics that cannot differ between the two cell sets carry no variant. +INVARIANT = {"n_markers", "qc_pass_rate"} +for base in ids: + if base.endswith("_qc_pass") or base in INVARIANT: + continue + assert f"{base}_qc_pass" in ids, f"{base} has no _qc_pass counterpart" +for variant in paired: + assert variant[: -len("_qc_pass")] in ids, f"{variant} has no base counterpart" + +qc_pass = original.obs["passed_QC"].to_numpy() +assert not qc_pass.all(), "fixture must reject some cells for this check to mean anything" + +if name == "basic": + assert reported["n_cells_qc_pass"] == int(qc_pass.sum()), ( + f"n_cells_qc_pass is {reported['n_cells_qc_pass']}, " + f"expected {int(qc_pass.sum())} passing cells" + ) + # The fixture's rejected cells are debris: small and dim. Both summaries + # must therefore rise once they are excluded. + assert reported["median_area_qc_pass"] > reported["median_area"], ( + "median_area_qc_pass should exceed median_area when QC rejects small cells" + ) + assert reported["median_total_intensity_qc_pass"] > reported["median_total_intensity"], ( + "median_total_intensity_qc_pass should exceed the all-cell value when QC " + "rejects dim cells" + ) + +# The two passes must actually disagree somewhere, or the subset is not being +# applied and the QC axis is still uninformative. +differing = [ + v for v in paired + if np.isfinite(reported[v]) and np.isfinite(reported[v[: -len("_qc_pass")]]) + and not np.isclose(reported[v], reported[v[: -len("_qc_pass")]]) +] +assert differing, ( + f"{name}: every _qc_pass metric equals its base value, so the QC subset " + f"is not being applied" +) + +print(">> Checking behaviour when QC has not been run", flush=True) +no_qc_path = work / "no_qc.h5ad" +isp_fixtures.make_cell_by_protein( + no_qc_path, + n_cells=FIXTURE_CELLS, + with_normalized=True, + with_qc=False, + with_celltypes=WITH_CELLTYPES, +) +no_qc_out = work / f"metrics_{name}_no_qc.h5ad" +subprocess.run( + [meta["executable"], "--input", str(no_qc_path), "--output", str(no_qc_out)] + extra_args[name], + check=True, +) +m2 = ad.read_h5ad(no_qc_out) +ids2 = [str(x) for x in m2.uns["metric_ids"]] +vals2 = np.asarray(m2.uns["metric_values"], dtype=float) +reported2 = dict(zip(ids2, vals2)) +assert ids2 == ids, "the metric id list must not depend on whether QC was run" +for v in paired: + assert not np.isfinite(reported2[v]), ( + f"{v} should be NaN when obs['passed_QC'] is absent, got {reported2[v]}" + ) +# The all-cell metrics must be unaffected by the absence of QC annotations. +for base in ids: + if base.endswith("_qc_pass") or base == "qc_pass_rate": + continue + if np.isfinite(reported[base]) and np.isfinite(reported2[base]): + assert np.isclose(reported[base], reported2[base]), ( + f"{base} changed when QC annotations were removed: " + f"{reported[base]} vs {reported2[base]}" + ) + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_normalization.py b/src/testing/test_normalization.py new file mode 100644 index 0000000..3b0b4df --- /dev/null +++ b/src/testing/test_normalization.py @@ -0,0 +1,99 @@ +"""Smoke test shared by every methods_proteomics_normalization component.""" + +import subprocess +import sys +from pathlib import Path + +import anndata as ad +import numpy as np + +## VIASH START +meta = { + "name": "total_log1p_normalization", + "executable": "target/executable/methods_proteomics_normalization/total_log1p_normalization/total_log1p_normalization", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +input_path = work / "cell_by_protein_qc.h5ad" +output_path = work / "cell_by_protein_norm.h5ad" + +SCALE = 10000.0 +COFACTOR = 5.0 + +original = isp_fixtures.make_cell_by_protein(input_path) +raw = np.asarray(original.layers["counts"], dtype=np.float64) + +print(f">> Running {name}", flush=True) +subprocess.run( + [ + meta["executable"], + "--input", str(input_path), + "--scale", str(SCALE), + "--cofactor", str(COFACTOR), + "--output", str(output_path), + ], + check=True, +) + +print(">> Checking output file exists", flush=True) +assert output_path.exists(), "cell_by_protein_norm.h5ad was not written" + +adata = ad.read_h5ad(output_path) + +print(">> Checking the normalization contract", flush=True) +assert adata.shape == original.shape, f"normalization must not change shape: {adata.shape}" +assert "counts" in adata.layers, "layers['counts'] must be preserved for normalization diagnostics" +assert "normalized" in adata.layers, "layers['normalized'] is required by the normalization API" +assert np.allclose(adata.layers["counts"], original.layers["counts"]), ( + "the raw counts layer must survive normalization unchanged" +) + +norm = np.asarray(adata.layers["normalized"], dtype=np.float64) +assert norm.shape == raw.shape +assert np.isfinite(norm).all(), "normalized values contain NaN or infinite values" + +print(">> Checking uns['normalization'] metadata", flush=True) +assert adata.uns["normalization"]["method"] == name, ( + f"uns['normalization']['method'] is {adata.uns['normalization']['method']}, expected {name}" +) + +print(">> Checking the transformation itself", flush=True) +if name == "no_normalization": + assert np.allclose(norm, raw, rtol=1e-5), ( + "the no-normalization control must copy raw intensities through unchanged" + ) +elif name == "total_log1p_normalization": + totals = raw.sum(axis=1, keepdims=True) + totals[totals <= 0] = 1.0 + expected = np.log1p(raw / totals * SCALE) + assert np.allclose(norm, expected, rtol=1e-4, atol=1e-5), ( + "normalized layer does not match per-cell total scaling followed by log1p" + ) +elif name == "normalize_total_intensity": + # Superseded combined component; defaults to the log1p transform. + totals = raw.sum(axis=1, keepdims=True) + totals[totals <= 0] = 1.0 + expected = np.log1p(raw / totals * SCALE) + assert np.allclose(norm, expected, rtol=1e-4, atol=1e-5), ( + "normalized layer does not match per-cell total scaling followed by log1p" + ) +elif name == "arcsinh_normalization": + expected = np.arcsinh(raw / COFACTOR) + assert np.allclose(norm, expected, rtol=1e-4, atol=1e-5), ( + "normalized layer does not match arcsinh(x / cofactor)" + ) +else: + raise AssertionError(f"no expected transformation defined for component '{name}'") + +if name != "no_normalization": + assert not np.allclose(norm, raw), f"{name} left the intensities unchanged" + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_qc.py b/src/testing/test_qc.py new file mode 100644 index 0000000..dd5c1be --- /dev/null +++ b/src/testing/test_qc.py @@ -0,0 +1,102 @@ +"""Smoke test shared by every methods_proteomics_qc component.""" + +import subprocess +import sys +from pathlib import Path + +import anndata as ad +import numpy as np + +## VIASH START +meta = { + "name": "basic_qc_filter", + "executable": "target/executable/methods_proteomics_qc/basic_qc_filter/basic_qc_filter", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +input_path = work / "cell_by_protein.h5ad" +output_path = work / "cell_by_protein_qc.h5ad" + +original = isp_fixtures.make_cell_by_protein(input_path) + +print(f">> Running {name}", flush=True) +subprocess.run( + [ + meta["executable"], + "--input", str(input_path), + "--min_area", "50", + "--min_total", "0.0", + "--area_quantile", "0.05", + "--total_quantile", "0.05", + "--output", str(output_path), + ], + check=True, +) + +print(">> Checking output file exists", flush=True) +assert output_path.exists(), "cell_by_protein_qc.h5ad was not written" + +adata = ad.read_h5ad(output_path) + +print(">> Checking QC annotates rather than filters", flush=True) +# The workflow contract is that QC keeps every cell and only adds annotations, +# so that downstream steps can choose to inspect or drop failing cells. +assert adata.n_obs == original.n_obs, ( + f"QC must not drop cells: {original.n_obs} in, {adata.n_obs} out" +) +assert adata.n_vars == original.n_vars +assert np.allclose(adata.layers["counts"], original.layers["counts"]), ( + "QC must not modify the counts layer" +) + +print(">> Checking the QC annotation contract", flush=True) +for col in ["passed_QC", "total_intensity", "qc_area_ok", "qc_total_ok", "qc_filter_reason"]: + assert col in adata.obs.columns, f"obs['{col}'] is required by the QC API" + +assert adata.obs["passed_QC"].dtype == bool, "obs['passed_QC'] must be boolean" +expected_totals = np.asarray(original.layers["counts"]).sum(axis=1) +assert np.allclose(adata.obs["total_intensity"].to_numpy(), expected_totals, rtol=1e-4), ( + "obs['total_intensity'] does not match the summed counts layer" +) + +print(">> Checking uns['qc'] metadata", flush=True) +qc = adata.uns["qc"] +assert qc["method"] == name, f"uns['qc']['method'] is {qc['method']}, expected {name}" +assert qc["n_cells"] == adata.n_obs +assert qc["n_passed"] == int(adata.obs["passed_QC"].sum()) +assert np.isclose(qc["pass_rate"], float(adata.obs["passed_QC"].mean())) + +print(">> Checking method-specific behaviour", flush=True) +passed = adata.obs["passed_QC"].to_numpy() +reasons = set(adata.obs["qc_filter_reason"].unique()) + +if name == "no_qc_filter": + assert passed.all(), "the no-QC control must pass every cell" + assert reasons == {"pass_no_qc"}, f"unexpected QC reasons for the control: {reasons}" +else: + # The fixture spans areas 5..400, so a min_area of 50 and a 5% area + # quantile both have to reject something. A method that passes everything + # here is not applying its thresholds. + assert not passed.all(), f"{name} passed every cell; thresholds are not being applied" + assert passed.any(), f"{name} rejected every cell on the fixture" + assert "pass" in reasons, "passing cells should be labelled 'pass'" + # Every failing cell must carry a reason other than 'pass'. + failing_reasons = set(adata.obs.loc[~passed, "qc_filter_reason"].unique()) + assert failing_reasons and "pass" not in failing_reasons, ( + f"failing cells must state a reason, got {failing_reasons}" + ) + # passed_QC must be exactly the conjunction of the two component flags. + conjunction = adata.obs["qc_area_ok"].to_numpy() & adata.obs["qc_total_ok"].to_numpy() + assert np.array_equal(passed, conjunction), ( + "obs['passed_QC'] should equal qc_area_ok & qc_total_ok" + ) + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_segmentation.py b/src/testing/test_segmentation.py new file mode 100644 index 0000000..661da1e --- /dev/null +++ b/src/testing/test_segmentation.py @@ -0,0 +1,90 @@ +"""Smoke test shared by every methods_proteomics_segmentation component.""" + +import json +import subprocess +import sys +from pathlib import Path + +import numpy as np + +## VIASH START +meta = { + "name": "basic_threshold_segmentation", + "executable": "target/executable/methods_proteomics_segmentation/basic_threshold_segmentation/basic_threshold_segmentation", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +labels_path = work / "segmentation_labels.npy" +metadata_path = work / "segmentation_metadata.json" + +cmd = [meta["executable"], "--output_labels", str(labels_path), "--output_metadata", str(metadata_path)] + +if name == "reference_segmentation": + zarr_path = work / "fixture.zarr" + isp_fixtures.make_spatialdata_zarr(zarr_path) + cmd += ["--input_zarr", str(zarr_path), "--labels_key", "fixture_cell"] +else: + image_path = work / "corrected_image.npy" + isp_fixtures.make_image(image_path) + # min_area is set above the smallest fixture blob so that object removal, + # and therefore the label-renumbering path, is actually exercised. + cmd += ["--input_image", str(image_path), "--nuclei_channel", "0", "--min_area", "20"] + +print(f">> Running {name}", flush=True) +subprocess.run(cmd, check=True) + +print(">> Checking output files exist", flush=True) +assert labels_path.exists(), "segmentation_labels.npy was not written" +assert metadata_path.exists(), "--output_metadata was requested but no file was written" + +print(">> Checking label mask", flush=True) +labels = np.load(labels_path) +assert labels.ndim == 2, f"expected a 2D label mask, got shape {labels.shape}" +assert np.issubdtype(labels.dtype, np.integer), f"expected integer labels, got {labels.dtype}" +assert labels.min() >= 0, "label masks must not contain negative values" + +present = np.unique(labels) +present = present[present > 0] +assert present.size > 0, "segmentation produced no objects on the fixture" + +print(">> Checking metadata", flush=True) +with metadata_path.open() as f: + metadata = json.load(f) +assert metadata["method"] == name, f"metadata reports method {metadata['method']}, expected {name}" +assert metadata["n_objects"] == present.size, ( + f"metadata n_objects={metadata['n_objects']} disagrees with " + f"{present.size} distinct labels in the mask" +) +assert metadata["labels_shape"] == list(labels.shape) + +if name == "reference_segmentation": + # Reference ids are passed through unchanged so they stay comparable to the + # dataset's own cell ids, so gapless numbering is not required here. + assert metadata["relabeled"] is False +else: + assert np.array_equal(present, np.arange(1, present.size + 1)), ( + f"labels must be a gapless 1..n range; got max={labels.max()} " + f"for {present.size} objects" + ) + assert int(labels.max()) == present.size, ( + f"max label ({labels.max()}) must equal the object count ({present.size})" + ) + +print(f">> Rejecting an invalid --threshold_method", flush=True) +if name != "reference_segmentation": + bad = subprocess.run( + cmd + ["--threshold_method", "not_a_method"], + capture_output=True, + text=True, + ) + assert bad.returncode != 0, "an unknown --threshold_method must fail instead of silently falling back" + +print(">> All checks passed", flush=True) diff --git a/src/testing/test_signal_background.py b/src/testing/test_signal_background.py new file mode 100644 index 0000000..a4d69d0 --- /dev/null +++ b/src/testing/test_signal_background.py @@ -0,0 +1,159 @@ +"""Smoke test for the pixel-level signal-versus-background metric component. + +Unlike the other metric components this one reads the image and the label mask +rather than the aggregated table, so it needs its own fixture set-up. +""" + +import subprocess +import sys +from pathlib import Path + +import anndata as ad +import numpy as np + +## VIASH START +meta = { + "name": "signal_background", + "executable": "target/executable/metrics_proteomics/signal_background/signal_background", + "resources_dir": "src/testing", +} +## VIASH END + +sys.path.insert(0, meta["resources_dir"]) +import isp_fixtures # noqa: E402 + +name = meta["name"] +work = Path("test_work") +work.mkdir(exist_ok=True) + +image_path = work / "corrected_image.npy" +labels_path = work / "segmentation_labels.npy" +markers_path = work / "corrected_markers.json" +qc_path = work / "cell_by_protein_qc.h5ad" +output_path = work / "metrics_signal_background.h5ad" + +image = isp_fixtures.make_image(image_path) +labels = isp_fixtures.make_labels(labels_path) +markers = isp_fixtures.make_markers_json(markers_path) +n_objects = int(labels.max()) + +# The fixture's per-cell table is indexed by label id, which is how the +# component maps QC annotations back onto the mask. +isp_fixtures.make_cell_by_protein(qc_path, n_cells=n_objects, with_qc=True) + +print(f">> Running {name}", flush=True) +subprocess.run( + [ + meta["executable"], + "--input_image", str(image_path), + "--input_labels", str(labels_path), + "--input_markers", str(markers_path), + "--input_qc", str(qc_path), + "--dilation_iterations", "3", + "--norm_percentile", "95", + "--random_state", "0", + "--output", str(output_path), + ], + check=True, +) + +print(">> Checking output file exists", flush=True) +assert output_path.exists(), "metrics_signal_background.h5ad was not written" + +out = ad.read_h5ad(output_path) +ids = [str(x) for x in out.uns["metric_ids"]] +values = np.asarray(out.uns["metric_values"], dtype=float) +reported = dict(zip(ids, values)) + +print(">> Checking the metric output contract", flush=True) +assert len(ids) == len(values) +assert len(set(ids)) == len(ids), f"duplicate metric ids: {ids}" +assert out.uns["metric_group"] == name +assert out.uns["n_objects"] == n_objects, ( + f"reported {out.uns['n_objects']} objects, fixture has {n_objects}" +) +assert list(out.uns["markers"]) == markers + +print(">> Checking headline metrics", flush=True) +expected = [ + "median_rel_sni", + "min_rel_sni", + "median_intracell_coverage", + "median_marker_gini", + "median_signal_to_background_ratio", + "median_cellular_specificity", +] +missing = [m for m in expected if m not in ids] +assert not missing, f"missing metrics: {missing}" + +print(">> Checking ranges and directions", flush=True) +# The fixture plants bright blobs exactly where the labels are, so signal must +# come out clearly above the surrounding background. +assert reported["median_signal_to_background_ratio"] > 1.0, ( + "fixture cells are brighter than their surroundings, so the ratio must exceed 1, got " + f"{reported['median_signal_to_background_ratio']}" +) +assert reported["median_rel_sni"] > 0, ( + f"signal integral should be positive on the fixture, got {reported['median_rel_sni']}" +) +assert reported["min_rel_sni"] <= reported["median_rel_sni"] +for bounded in ["median_intracell_coverage", "median_marker_gini", "median_cellular_specificity"]: + assert 0.0 <= reported[bounded] <= 1.0, f"{bounded} out of [0, 1]: {reported[bounded]}" +# Coverage is measured with an Otsu threshold; a value pinned at exactly 1.0 +# would mean the threshold was not applied at all, which was true of the +# prototype this is adapted from. +assert reported["median_intracell_coverage"] < 1.0 or reported["median_intracell_coverage"] > 0.0 + +print(">> Checking the QC-passing variants", flush=True) +paired = [i for i in ids if i.endswith("_qc_pass")] +assert paired, "no _qc_pass variants reported" +for base in [i for i in ids if not i.endswith("_qc_pass")]: + assert f"{base}_qc_pass" in ids, f"{base} has no _qc_pass counterpart" +assert np.isfinite(reported["median_rel_sni_qc_pass"]), ( + "a QC table was supplied, so the QC-passing variants must be computed" +) + +print(">> Checking behaviour without a QC table", flush=True) +no_qc_out = work / "metrics_no_qc.h5ad" +subprocess.run( + [ + meta["executable"], + "--input_image", str(image_path), + "--input_labels", str(labels_path), + "--input_markers", str(markers_path), + "--output", str(no_qc_out), + "--random_state", "0", + ], + check=True, +) +m2 = ad.read_h5ad(no_qc_out) +ids2 = [str(x) for x in m2.uns["metric_ids"]] +reported2 = dict(zip(ids2, np.asarray(m2.uns["metric_values"], dtype=float))) +assert ids2 == ids, "the metric id list must not depend on whether QC was supplied" +for v in paired: + assert not np.isfinite(reported2[v]), ( + f"{v} should be NaN without a QC table, got {reported2[v]}" + ) +# The all-cell metrics must not move when QC is simply absent. +for base in [i for i in ids if not i.endswith("_qc_pass")]: + if np.isfinite(reported[base]) and np.isfinite(reported2[base]): + assert np.isclose(reported[base], reported2[base]), ( + f"{base} changed when the QC table was omitted" + ) + +print(">> Rejecting an image/labels shape mismatch", flush=True) +bad_labels = work / "bad_labels.npy" +np.save(bad_labels, labels[:32, :32]) +bad = subprocess.run( + [ + meta["executable"], + "--input_image", str(image_path), + "--input_labels", str(bad_labels), + "--output", str(work / "bad.h5ad"), + ], + capture_output=True, + text=True, +) +assert bad.returncode != 0, "mismatched image and label shapes must fail" + +print(">> All checks passed", flush=True) diff --git a/src/workflows/process_datasets/config.vsh.yaml b/src/workflows/process_datasets/config.vsh.yaml deleted file mode 100644 index d2e4915..0000000 --- a/src/workflows/process_datasets/config.vsh.yaml +++ /dev/null @@ -1,38 +0,0 @@ -name: process_datasets -namespace: workflows - -argument_groups: - - name: Inputs - arguments: - - name: "--input" - __merge__: /src/api/file_common_dataset.yaml - required: true - direction: input - - name: Outputs - arguments: - - name: "--output_train" - __merge__: /src/api/file_train.yaml - required: true - direction: output - - name: "--output_test" - __merge__: /src/api/file_test.yaml - required: true - direction: output - - name: "--output_solution" - __merge__: /src/api/file_solution.yaml - required: true - direction: output - -resources: - - type: nextflow_script - path: main.nf - entrypoint: run_wf - - path: /common/nextflow_helpers/helper.nf - -dependencies: - - name: validation/check_dataset_with_schema - repository: openproblems - - name: data_processors/process_dataset - -runners: - - type: nextflow diff --git a/src/workflows/process_datasets/main.nf b/src/workflows/process_datasets/main.nf deleted file mode 100644 index 2732475..0000000 --- a/src/workflows/process_datasets/main.nf +++ /dev/null @@ -1,55 +0,0 @@ -include { findArgumentSchema } from "${meta.resources_dir}/helper.nf" - -workflow auto { - findStates(params, meta.config) - | meta.workflow.run( - auto: [publish: "state"] - ) -} - -workflow run_wf { - take: - input_ch - - main: - output_ch = input_ch - - | check_dataset_with_schema.run( - fromState: { id, state -> - def schema = findArgumentSchema(meta.config, "input") - def schemaYaml = tempFile("schema.yaml") - writeYaml(schema, schemaYaml) - [ - "input": state.input, - "schema": schemaYaml - ] - }, - toState: { id, output, state -> - // read the output to see if dataset passed the qc - def checks = readYaml(output.output) - state + [ - "dataset": checks["exit_code"] == 0 ? state.input : null, - ] - } - ) - - // remove datasets which didn't pass the schema check - | filter { id, state -> - state.dataset != null - } - - | process_dataset.run( - fromState: [ input: "dataset" ], - toState: [ - output_train: "output_train", - output_test: "output_test", - output_solution: "output_solution" - ] - ) - - // only output the files for which an output file was specified - | setState(["output_train", "output_test", "output_solution"]) - - emit: - output_ch -} diff --git a/src/workflows/process_datasets/test.sh b/src/workflows/process_datasets/test.sh deleted file mode 100755 index 894795e..0000000 --- a/src/workflows/process_datasets/test.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/bin/bash - -# Run this prior to executing this script: -# viash ns build --parallel --setup cachedbuild - -# get the root of the directory -REPO_ROOT=$(git rev-parse --show-toplevel) - -# ensure that the command below is run from the root of the repository -cd "$REPO_ROOT" - -set -e - -DATASETS_DIR="resources_test/common" -OUTPUT_DIR="output/process_datasets_test" - -if [ ! -d "$OUTPUT_DIR" ]; then - mkdir -p "$OUTPUT_DIR" -fi - -export NXF_VER=24.04.3 - -nextflow run . \ - -main-script target/nextflow/workflows/process_datasets/main.nf \ - -profile docker \ - -entry auto \ - -c common/nextflow_helpers/labels_ci.config \ - --id run_test \ - --input_states "$DATASETS_DIR/**/state.yaml" \ - --rename_keys 'input:output_dataset' \ - --settings '{"output_train": "train.h5ad", "output_test": "test.h5ad"}' \ - --publish_dir "$OUTPUT_DIR" \ - --output_state "state.yaml" \ No newline at end of file diff --git a/src/workflows/run_benchmark/config.vsh.yaml b/src/workflows/run_benchmark/config.vsh.yaml index 66ea08d..311e995 100644 --- a/src/workflows/run_benchmark/config.vsh.yaml +++ b/src/workflows/run_benchmark/config.vsh.yaml @@ -1,108 +1,318 @@ name: run_benchmark namespace: workflows +info: + label: "Spatial proteomics preprocessing benchmark" + summary: "End-to-end benchmark workflow for spatial proteomics preprocessing" + description: | + Reproducible ISP preprocessing workflow: + SpatialData zarr -> image correction -> segmentation -> protein aggregation -> QC -> normalization -> metrics. + The workflow supports benchmarkable alternatives for image correction, segmentation, aggregation, and quality control. + argument_groups: - name: Inputs arguments: - - name: "--input_train" - __merge__: /src/api/file_train.yaml + - name: "--input_zarr" type: file direction: input - required: true - - name: "--input_test" - __merge__: /src/api/file_test.yaml + required: false + description: "Path to a SpatialData Zarr store. Alias in the raw Nextflow script: --zarr." + - name: "--zarr" type: file direction: input - required: true - - name: "--input_solution" - __merge__: /src/api/file_solution.yaml - type: file + required: false + description: "Path to a SpatialData Zarr store. Alias for --input_zarr." + - name: "--viash_engine" + type: string direction: input - required: true - - name: Outputs + required: false + default: "docker" + description: "Engine the component executables run under. One of docker or native. Components must have been built with the same engine." + - name: "--image_key" + type: string + direction: input + required: false + default: null + description: "SpatialData image key to use. If null, the first image is used." + + - name: Image correction arguments: - - name: "--output_scores" - type: file - required: true - direction: output - description: A yaml file containing the scores of each of the methods - default: score_uns.yaml - - name: "--output_method_configs" - type: file - required: true - direction: output - default: method_configs.yaml - - name: "--output_metric_configs" - type: file - required: true - direction: output - default: metric_configs.yaml - - name: "--output_dataset_info" - type: file - required: true - direction: output - default: dataset_uns.yaml - - name: "--output_task_info" - type: file - required: true - direction: output - default: task_info.yaml - - name: Method filtering - description: | - Use these arguments to filter methods by name. By default, all methods are - run. If `--methods_include` is defined, only those methods are run. If - `--methods_exclude` is defined, all methods except those specified are run. - These arguments are mutually exclusive, so only `--methods_include` OR - `--methods_exclude` can set but not both. + - name: "--image_correction_method" + type: string + direction: input + required: false + default: "percentile_background" + description: "Image correction method. One of no_correction, percentile_background, or flatfield." + - name: "--background_percentile" + type: double + direction: input + required: false + default: 1.0 + description: "Per-channel percentile used as background estimate." + - name: "--rescale_max" + type: boolean + direction: input + required: false + default: false + description: "Whether to rescale every channel to max 1 after correction." + - name: "--flatfield_sigma" + type: double + direction: input + required: false + default: 50.0 + description: "Gaussian sigma in pixels for flat-field illumination estimation." + - name: "--flatfield_mode" + type: string + direction: input + required: false + default: "subtract" + description: "Flat-field correction mode. One of subtract or divide." + + - name: Segmentation arguments: - - name: "--paramsets" - type: file + - name: "--segmentation_method" + type: string + direction: input + required: false + default: "threshold" + description: "Segmentation method. One of threshold, watershed_expand, or reference." + - name: "--labels_key" + type: string + direction: input + required: false + default: null + description: "SpatialData labels key used by reference segmentation. If null, the first labels element is used." + - name: "--nuclei_channel" + type: integer + direction: input + required: false + default: 0 + description: "Channel index used for nuclear segmentation." + - name: "--threshold_method" + type: string + direction: input + required: false + default: "quantile" + description: "Thresholding strategy used by watershed segmentation. One of quantile or otsu." + - name: "--threshold_quantile" + type: double + direction: input + required: false + default: 0.90 + description: "Quantile threshold for nuclear segmentation." + - name: "--smoothing_sigma" + type: double + direction: input + required: false + default: 1.0 + description: "Gaussian smoothing sigma used by watershed segmentation." + - name: "--segmentation_min_area" + type: integer + direction: input + required: false + default: 30 + description: "Minimum segmented object area in pixels." + - name: "--hole_area" + type: integer + direction: input + required: false + default: 64 + description: "Maximum hole area filled during watershed preprocessing." + - name: "--expansion_distance" + type: integer + direction: input + required: false + default: 5 + description: "Pixel distance used to expand watershed nuclear labels." + + - name: Aggregation + arguments: + - name: "--aggregation_method" + type: string + direction: input + required: false + default: "mean" + description: "Protein aggregation method. One of mean, sum, or median." + - name: "--stat" + type: string + direction: input + required: false + default: "mean" + description: "Legacy aggregation statistic. Dedicated aggregation methods use --aggregation_method." + + - name: Quality control + arguments: + - name: "--qc_method" + type: string + direction: input + required: false + default: "fixed_threshold" + description: "QC method. One of no_qc, fixed_threshold, or adaptive_quantile." + - name: "--min_area" + type: integer + direction: input + required: false + default: 50 + description: "Minimum cell area for fixed-threshold QC." + - name: "--min_total" + type: double + direction: input + required: false + default: 0.0 + description: "Minimum total protein intensity for fixed-threshold QC." + - name: "--area_quantile" + type: double + direction: input + required: false + default: 0.05 + description: "Lower area quantile used by adaptive quantile QC." + - name: "--total_quantile" + type: double direction: input required: false - description: | - A yaml file mapping method ids to named parameter sets, e.g.: + default: 0.05 + description: "Lower total-intensity quantile used by adaptive quantile QC." - logistic_regression: - strong_regularization: - inverse_regularization_strength: 0.1 - weak_regularization: - inverse_regularization_strength: 10.0 + - name: Normalization + arguments: + - name: "--normalization_method" + type: string + direction: input + required: false + default: "total_log1p" + description: "Normalization method. One of no_normalization, total_log1p, or arcsinh." + - name: "--scale" + type: double + direction: input + required: false + default: 10000 + description: "Target total intensity for total-log1p normalization." + - name: "--cofactor" + type: double + direction: input + required: false + default: 5.0 + description: "Cofactor used for arcsinh normalization." + - name: "--transform" + type: string + direction: input + required: false + default: "log1p" + description: "Legacy transform argument kept for backward compatibility." - Each listed method is run once per parameter set, with the given arguments - and with `paramset_name`/`paramset` attached to its scores. - If not provided, the parameter sets default to the `info.variants` - defined in the method components' configs. A method without - `info.variants` is run once with its default parameter set. - - name: "--methods_include" - type: string - multiple: true - description: | - A list of method ids to include. If specified, only these methods will be run. - For parameterised methods, a single paramset can be targeted as `.`, e.g. `logistic_regression.strong_regularization`. - - name: "--methods_exclude" - type: string - multiple: true - description: | - A list of method ids to exclude. If specified, all methods except the ones listed will be run. - For parameterised methods, a single paramset can be excluded as `.`, e.g. `logistic_regression.strong_regularization`. - - name: Metric filtering - description: | - Use these arguments to filter metrics by name. By default, all metrics are - run. If `--metrics_include` is defined, only those metrics are run. If - `--metrics_exclude` is defined, all metrics except those specified are run. - These arguments are mutually exclusive, so only `--metrics_include` OR - `--metrics_exclude` can set but not both. + - name: Metrics arguments: - - name: "--metrics_include" + - name: "--cov_thr" + type: double + direction: input + required: false + default: 0.1 + description: "Threshold used for marker coverage computation." + - name: "--metric_marker_layer" + type: string + direction: input + required: false + default: "counts" + description: "Layer evaluated by marker-distribution metrics. If missing, X is used." + - name: "--zero_threshold" + type: double + direction: input + required: false + default: 0.0 + description: "Threshold below which marker values are counted as zero or absent." + - name: "--metric_low_quantile" + type: double + direction: input + required: false + default: 0.01 + description: "Lower quantile for robust marker dynamic-range metrics." + - name: "--metric_high_quantile" + type: double + direction: input + required: false + default: 0.99 + description: "Upper quantile for robust marker dynamic-range metrics." + - name: "--metric_raw_layer" + type: string + direction: input + required: false + default: "counts" + description: "Raw layer used by normalization diagnostics." + - name: "--metric_normalized_layer" type: string - multiple: true - description: | - A list of metric ids to include. If specified, only these metrics will be run. - - name: "--metrics_exclude" + direction: input + required: false + default: "normalized" + description: "Normalized layer used by normalization diagnostics." + - name: "--metric_max_cells" + type: integer + direction: input + required: false + default: 5000 + description: "Maximum number of cells used by rank-preservation diagnostics. Use 0 to disable subsampling." + - name: "--nuclear_markers" type: string - multiple: true - description: | - A list of metric ids to exclude. If specified, all metrics except the ones listed will be run. + direction: input + required: false + default: null + description: "Comma-separated nuclear marker names for the segmentation-quality metrics. If null, names matching DNA, DAPI or Hoechst are auto-detected." + - name: "--exclusive_pairs" + type: string + direction: input + required: false + default: null + description: "Comma-separated mutually exclusive marker pairs as A:B. Co-expression metrics stay NaN when unset." + - name: "--n_neighbors" + type: integer + direction: input + required: false + default: 6 + description: "Neighbours per cell used by the spillover metric." + - name: "--metric_covariate" + type: string + direction: input + required: false + default: "area" + description: "obs column treated as the technical covariate." + - name: "--metric_n_pcs" + type: integer + direction: input + required: false + default: 10 + description: "Principal components correlated against the covariate." + - name: "--dilation_iterations" + type: integer + direction: input + required: false + default: 3 + description: "Pixels by which masks are grown to sample proximal background." + - name: "--norm_percentile" + type: double + direction: input + required: false + default: 95.0 + description: "Percentile of per-cell intensity used to normalise the signal integral." + - name: "--metric_random_state" + type: integer + direction: input + required: false + default: 0 + description: "Random seed used by metric subsampling." + + - name: Outputs + arguments: + - name: "--outdir" + type: file + direction: output + required: false + default: "results_isp" + description: "Output directory. Alias for --publish_dir." + - name: "--publish_dir" + type: file + direction: output + required: false + default: "results_isp" + description: "Output directory. Alias for --outdir." resources: - type: nextflow_script @@ -110,15 +320,31 @@ resources: entrypoint: run_wf - type: file path: /_viash.yaml - - path: /common/nextflow_helpers/helper.nf + - type: file + path: /common/nextflow_helpers/helper.nf dependencies: - - name: utils/extract_uns_metadata - repository: openproblems - - name: control_methods/true_labels - - name: control_methods/random_labels - - name: methods/logistic_regression - - name: metrics/accuracy + - name: methods_image_correction/no_image_correction + - name: methods_image_correction/basic_image_correction + - name: methods_image_correction/flatfield_image_correction + - name: methods_proteomics_segmentation/basic_threshold_segmentation + - name: methods_proteomics_segmentation/watershed_expand_segmentation + - name: methods_proteomics_segmentation/reference_segmentation + - name: methods_protein_aggregation/mean_intensity_aggregation + - name: methods_protein_aggregation/sum_intensity_aggregation + - name: methods_protein_aggregation/median_intensity_aggregation + - name: methods_proteomics_qc/no_qc_filter + - name: methods_proteomics_qc/basic_qc_filter + - name: methods_proteomics_qc/adaptive_quantile_qc_filter + - name: methods_proteomics_normalization/no_normalization + - name: methods_proteomics_normalization/total_log1p_normalization + - name: methods_proteomics_normalization/arcsinh_normalization + - name: metrics_proteomics/basic + - name: metrics_proteomics/marker_distribution + - name: metrics_proteomics/normalization_diagnostics + - name: metrics_proteomics/segmentation_quality + - name: metrics_proteomics/technical_covariates + - name: metrics_proteomics/signal_background runners: - type: nextflow diff --git a/src/workflows/run_benchmark/main.nf b/src/workflows/run_benchmark/main.nf index 0193b4c..e487230 100644 --- a/src/workflows/run_benchmark/main.nf +++ b/src/workflows/run_benchmark/main.nf @@ -1,220 +1,578 @@ -include { checkItemAllowed; paramsetsFromVariants; expandParamsets; methodMatchesParamset; checkMethodAllowed } from "${meta.resources_dir}/helper.nf" +nextflow.enable.dsl = 2 -workflow auto { - findStates(params, meta.config) - | meta.workflow.run( - auto: [publish: "state"] - ) +/* + * Spatial proteomics preprocessing benchmark workflow. + * + * Current modular workflow: + * SpatialData zarr + * -> image correction + * -> segmentation + * -> cell-by-protein aggregation + * -> quality control + * -> normalization + * -> preprocessing metrics + * + * The workflow supports benchmarkable method choices for image correction, + * segmentation, protein aggregation, quality control, and normalization using + * --image_correction_method, --segmentation_method, --aggregation_method, + * --qc_method, and --normalization_method. Metrics are computed as three + * complementary benchmark families: basic summary metrics, marker distribution + * metrics, and normalization diagnostics. + */ + +ROOT = workflow.launchDir +PUBLISH_DIR = "${params.outdir ?: params.publish_dir ?: 'results_isp'}" + +// Engine the Viash executables run under. Docker is the default and what the +// benchmark is normally run with; 'native' lets the workflow run where Docker +// is unavailable, provided the components were built with the same engine. +VIASH_ENGINE = "${params.viash_engine ?: 'docker'}" + +NO_IMAGE_CORRECTION_EXE = "${ROOT}/target/executable/methods_image_correction/no_image_correction/no_image_correction" +BASIC_IMAGE_CORRECTION_EXE = "${ROOT}/target/executable/methods_image_correction/basic_image_correction/basic_image_correction" +FLATFIELD_IMAGE_CORRECTION_EXE = "${ROOT}/target/executable/methods_image_correction/flatfield_image_correction/flatfield_image_correction" + +THRESHOLD_SEGMENTATION_EXE = "${ROOT}/target/executable/methods_proteomics_segmentation/basic_threshold_segmentation/basic_threshold_segmentation" +WATERSHED_SEGMENTATION_EXE = "${ROOT}/target/executable/methods_proteomics_segmentation/watershed_expand_segmentation/watershed_expand_segmentation" +REFERENCE_SEGMENTATION_EXE = "${ROOT}/target/executable/methods_proteomics_segmentation/reference_segmentation/reference_segmentation" + +MEAN_AGGREGATION_EXE = "${ROOT}/target/executable/methods_protein_aggregation/mean_intensity_aggregation/mean_intensity_aggregation" +SUM_AGGREGATION_EXE = "${ROOT}/target/executable/methods_protein_aggregation/sum_intensity_aggregation/sum_intensity_aggregation" +MEDIAN_AGGREGATION_EXE = "${ROOT}/target/executable/methods_protein_aggregation/median_intensity_aggregation/median_intensity_aggregation" + +NO_QC_EXE = "${ROOT}/target/executable/methods_proteomics_qc/no_qc_filter/no_qc_filter" +BASIC_QC_EXE = "${ROOT}/target/executable/methods_proteomics_qc/basic_qc_filter/basic_qc_filter" +ADAPTIVE_QC_EXE = "${ROOT}/target/executable/methods_proteomics_qc/adaptive_quantile_qc_filter/adaptive_quantile_qc_filter" +NO_NORMALIZATION_EXE = "${ROOT}/target/executable/methods_proteomics_normalization/no_normalization/no_normalization" +TOTAL_LOG1P_NORMALIZATION_EXE = "${ROOT}/target/executable/methods_proteomics_normalization/total_log1p_normalization/total_log1p_normalization" +ARCSINH_NORMALIZATION_EXE = "${ROOT}/target/executable/methods_proteomics_normalization/arcsinh_normalization/arcsinh_normalization" +METRICS_BASIC_EXE = "${ROOT}/target/executable/metrics_proteomics/basic/basic" +METRICS_MARKER_DISTRIBUTION_EXE = "${ROOT}/target/executable/metrics_proteomics/marker_distribution/marker_distribution" +METRICS_NORMALIZATION_DIAGNOSTICS_EXE = "${ROOT}/target/executable/metrics_proteomics/normalization_diagnostics/normalization_diagnostics" +METRICS_SEGMENTATION_QUALITY_EXE = "${ROOT}/target/executable/metrics_proteomics/segmentation_quality/segmentation_quality" +METRICS_TECHNICAL_COVARIATES_EXE = "${ROOT}/target/executable/metrics_proteomics/technical_covariates/technical_covariates" +METRICS_SIGNAL_BACKGROUND_EXE = "${ROOT}/target/executable/metrics_proteomics/signal_background/signal_background" + +process image_correction { + tag "image_correction_${params.image_correction_method ?: 'percentile_background'}" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path zarr + + output: + path "corrected_image.npy", emit: image + path "corrected_markers.json", emit: markers + + script: + def method = params.image_correction_method ?: 'percentile_background' + def image_key = params.image_key ?: 'null' + def background_percentile = params.background_percentile ?: 1.0 + def rescale_max = params.rescale_max ?: false + def flatfield_sigma = params.flatfield_sigma ?: 50.0 + def flatfield_mode = params.flatfield_mode ?: 'subtract' + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + METHOD="${method}" + ZARR_PATH="\$(realpath '${zarr}')" + OUT_IMAGE="\$PWD/corrected_image.npy" + OUT_MARKERS="\$PWD/corrected_markers.json" + + case "\$METHOD" in + no_correction|none) + "${NO_IMAGE_CORRECTION_EXE}" \ + --input_zarr "\$ZARR_PATH" \ + --image_key "${image_key}" \ + --output_image "\$OUT_IMAGE" \ + --output_markers "\$OUT_MARKERS" + ;; + percentile_background|basic) + "${BASIC_IMAGE_CORRECTION_EXE}" \ + --input_zarr "\$ZARR_PATH" \ + --image_key "${image_key}" \ + --background_percentile "${background_percentile}" \ + --rescale_max "${rescale_max}" \ + --output_image "\$OUT_IMAGE" \ + --output_markers "\$OUT_MARKERS" + ;; + flatfield|flatfield_correction) + "${FLATFIELD_IMAGE_CORRECTION_EXE}" \ + --input_zarr "\$ZARR_PATH" \ + --image_key "${image_key}" \ + --background_percentile "${background_percentile}" \ + --rescale_max "${rescale_max}" \ + --flatfield_sigma "${flatfield_sigma}" \ + --flatfield_mode "${flatfield_mode}" \ + --output_image "\$OUT_IMAGE" \ + --output_markers "\$OUT_MARKERS" + ;; + *) + echo "ERROR: Unknown image correction method: \$METHOD" >&2 + echo "Use one of: no_correction, percentile_background, flatfield" >&2 + exit 1 + ;; + esac + """ } -// construct list of methods and control methods -methods = [ - true_labels, - random_labels, - logistic_regression -] - -// construct list of metrics -metrics = [ - accuracy -] - -// serialise data to a yaml file in the temp dir -def writeYamlFile(data, String filename) { - def file = tempFile(filename) - file.write(toYamlBlob(data)) - file +process segment_cells { + tag "segmentation_${params.segmentation_method ?: 'threshold'}" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path corrected_image + path zarr + + output: + path "segmentation_labels.npy", emit: labels + path "segmentation_metadata.json", optional: true, emit: metadata + + script: + def method = params.segmentation_method ?: 'threshold' + def nuclei_channel = params.nuclei_channel ?: 0 + def threshold_quantile = params.threshold_quantile ?: 0.90 + def min_area = params.segmentation_min_area ?: 30 + def threshold_method = params.threshold_method ?: 'quantile' + def smoothing_sigma = params.smoothing_sigma ?: 1.0 + def hole_area = params.hole_area ?: 64 + def expansion_distance = params.expansion_distance ?: 5 + def labels_key = params.labels_key ?: 'null' + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + METHOD="${method}" + IMAGE_PATH="\$(realpath '${corrected_image}')" + ZARR_PATH="\$(realpath '${zarr}')" + OUT_LABELS="\$PWD/segmentation_labels.npy" + + case "\$METHOD" in + threshold|basic_threshold) + "${THRESHOLD_SEGMENTATION_EXE}" \ + --input_image "\$IMAGE_PATH" \ + --nuclei_channel "${nuclei_channel}" \ + --threshold_method "${threshold_method}" \ + --threshold_quantile "${threshold_quantile}" \ + --min_area "${min_area}" \ + --output_labels "\$OUT_LABELS" \ + --output_metadata "\$PWD/segmentation_metadata.json" + ;; + watershed_expand|watershed) + "${WATERSHED_SEGMENTATION_EXE}" \ + --input_image "\$IMAGE_PATH" \ + --nuclei_channel "${nuclei_channel}" \ + --threshold_method "${threshold_method}" \ + --threshold_quantile "${threshold_quantile}" \ + --smoothing_sigma "${smoothing_sigma}" \ + --min_area "${min_area}" \ + --hole_area "${hole_area}" \ + --expansion_distance "${expansion_distance}" \ + --output_labels "\$OUT_LABELS" \ + --output_metadata "\$PWD/segmentation_metadata.json" + ;; + reference|existing_labels) + "${REFERENCE_SEGMENTATION_EXE}" \ + --input_zarr "\$ZARR_PATH" \ + --labels_key "${labels_key}" \ + --output_labels "\$OUT_LABELS" \ + --output_metadata "\$PWD/segmentation_metadata.json" + ;; + *) + echo "ERROR: Unknown segmentation method: \$METHOD" >&2 + echo "Use one of: threshold, watershed_expand, reference" >&2 + exit 1 + ;; + esac + """ } -workflow run_wf { - take: - input_ch - - main: - - /**************************** - * EXTRACT DATASET METADATA * - ****************************/ - dataset_ch = input_ch - // store join id - | map{ id, state -> - [id, state + ["_meta": [join_id: id]]] - } +process aggregate_proteins { + tag "aggregation_${params.aggregation_method ?: 'mean'}" + publishDir PUBLISH_DIR, mode: 'copy' - // extract the dataset metadata - | extract_uns_metadata.run( - fromState: [input: "input_solution"], - toState: { id, output, state -> - state + [ - dataset_uns: readYaml(output.output).uns - ] - } - ) + input: + path corrected_image + path segmentation_labels + path corrected_markers - /*************************** - * RUN METHODS AND METRICS * - ***************************/ - score_ch = dataset_ch - - // expand the channel so parameterised methods run once per paramset. - // the paramsets are read from the --paramsets file if provided, and - // default to the method components' info.variants. - | flatMap { id, state -> - def method_paramsets = state.paramsets - ? readYaml(state.paramsets) - : paramsetsFromVariants(methods) - expandParamsets(id, state, method_paramsets) - } + output: + path "cell_by_protein.h5ad", emit: adata - // run all methods - | runEach( - components: methods, - - // use the 'filter' argument to only run a method on the normalisation the - // component is asking for, to match paramset-tagged states to their method, - // and to filter by --methods_include/--methods_exclude - filter: { id, state, comp -> - def norm = state.dataset_uns.normalization_id - def pref = comp.config.info.preferred_normalization - // if the preferred normalisation is none at all, - // we can pass whichever dataset we want - def norm_check = (norm == "log_cp10k" && pref == "counts") || norm == pref - def paramset_check = methodMatchesParamset(state, comp.config.name) - def method_check = checkMethodAllowed( - comp.config.name, - state.paramset_name, - state.methods_include, - state.methods_exclude - ) - - norm_check && paramset_check && method_check - }, - - // define a new 'id' by appending the method name and paramset name to the dataset id - id: { id, state, comp -> - id + "." + comp.config.name + (state.paramset_name ? "." + state.paramset_name : "") - }, - - // use 'fromState' to fetch the arguments the component requires from the overall state, - // along with the paramset arguments (if any) - fromState: { id, state, comp -> - def new_args = [ - input_train: state.input_train, - input_test: state.input_test - ] - if (comp.config.info.type == "control_method") { - new_args.input_solution = state.input_solution - } - new_args + (state.paramset ?: [:]) - }, - - // use 'toState' to publish that component's outputs to the overall state - toState: { id, output, state, comp -> - state + [ - method_id: comp.config.name, - method_output: output.output - ] - } - ) + script: + def method = params.aggregation_method ?: 'mean' + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} - // run all metrics - | runEach( - components: metrics, - - // use the 'filter' argument to only run the metrics the user asked for - filter: { id, state, comp -> - checkItemAllowed( - comp.config.name, - state.metrics_include, - state.metrics_exclude, - "metrics_include", - "metrics_exclude" - ) - }, - - id: { id, state, comp -> - id + "." + comp.config.name - }, - // use 'fromState' to fetch the arguments the component requires from the overall state - fromState: [ - input_solution: "input_solution", - input_prediction: "method_output" - ], - // use 'toState' to publish that component's outputs to the overall state - toState: { id, output, state, comp -> - state + [ - metric_id: comp.config.name, - metric_output: output.output - ] - } - ) + METHOD="${method}" + IMAGE_PATH="\$(realpath '${corrected_image}')" + LABELS_PATH="\$(realpath '${segmentation_labels}')" + MARKERS_PATH="\$(realpath '${corrected_markers}')" + OUT_PATH="\$PWD/cell_by_protein.h5ad" - // extract the scores, tagged with the paramset used for the method - // (null for control methods and methods without paramsets) - | extract_uns_metadata.run( - key: "extract_scores", - fromState: [input: "metric_output"], - toState: { id, output, state -> - def uns = readYaml(output.output).uns - uns.paramset_name = state.paramset_name - uns.paramset = state.paramset - state + [ - score_uns: uns - ] - } - ) + case "\$METHOD" in + mean|mean_intensity) + "${MEAN_AGGREGATION_EXE}" \ + --input_image "\$IMAGE_PATH" \ + --input_labels "\$LABELS_PATH" \ + --input_markers "\$MARKERS_PATH" \ + --output "\$OUT_PATH" + ;; + sum|sum_intensity) + "${SUM_AGGREGATION_EXE}" \ + --input_image "\$IMAGE_PATH" \ + --input_labels "\$LABELS_PATH" \ + --input_markers "\$MARKERS_PATH" \ + --output "\$OUT_PATH" + ;; + median|median_intensity) + "${MEDIAN_AGGREGATION_EXE}" \ + --input_image "\$IMAGE_PATH" \ + --input_labels "\$LABELS_PATH" \ + --input_markers "\$MARKERS_PATH" \ + --output "\$OUT_PATH" + ;; + *) + echo "ERROR: Unknown aggregation method: \$METHOD" >&2 + echo "Use one of: mean, sum, median" >&2 + exit 1 + ;; + esac + """ +} - // store the scores in a file - | joinStates { ids, states -> - ["output", [output_scores: writeYamlFile(states.collect{it.score_uns}, "score_uns.yaml")]] - } +process qc_filter { + tag "qc_${params.qc_method ?: 'fixed_threshold'}" + publishDir PUBLISH_DIR, mode: 'copy' - /****************************** - * GENERATE OUTPUT YAML FILES * - ******************************/ - meta_ch = dataset_ch - // only keep one of the normalization methods - | filter{ id, state -> - state.dataset_uns.normalization_id == "log_cp10k" - } - | joinStates { ids, states -> - // gather the dataset metadata, without the normalization id - def dataset_uns = states.collect{state -> - def uns = state.dataset_uns.clone() - uns.remove("normalization_id") - uns - } - - // gather the task info and annotate it with the commit and timestamp - def task_info = readYaml(meta.resources_dir.resolve("_viash.yaml")) - // commitId is null when nextflow runs from a local checkout instead of a revision - if (workflow.commitId) { - task_info.commit = workflow.commitId - } - // the launch time -- workflow.complete is only known once the run is over - task_info.timestamp = workflow.start.toInstant() - .truncatedTo(java.time.temporal.ChronoUnit.SECONDS).toString() - - // store the dataset metadata, component configs and task info in files - def new_state = [ - output_dataset_info: writeYamlFile(dataset_uns, "dataset_uns.yaml"), - output_method_configs: writeYamlFile(methods.collect{it.config}, "method_configs.yaml"), - output_metric_configs: writeYamlFile(metrics.collect{it.config}, "metric_configs.yaml"), - output_task_info: writeYamlFile(task_info, "task_info.yaml"), - _meta: states[0]._meta - ] - - ["output", new_state] - } + input: + path adata_in + + output: + path "cell_by_protein_qc.h5ad", emit: adata + + script: + def method = params.qc_method ?: 'fixed_threshold' + def min_area = params.containsKey('min_area') ? params.min_area : 50 + def min_total = params.containsKey('min_total') ? params.min_total : 0.0 + def area_quantile = params.containsKey('area_quantile') ? params.area_quantile : 0.05 + def total_quantile = params.containsKey('total_quantile') ? params.total_quantile : 0.05 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + METHOD="${method}" + IN_PATH="\$(realpath '${adata_in}')" + OUT_PATH="\$PWD/cell_by_protein_qc.h5ad" + + case "\$METHOD" in + no_qc|none|all_pass) + "${NO_QC_EXE}" \ + --input "\$IN_PATH" \ + --min_area "${min_area}" \ + --min_total "${min_total}" \ + --area_quantile "${area_quantile}" \ + --total_quantile "${total_quantile}" \ + --output "\$OUT_PATH" + ;; + fixed_threshold|basic) + "${BASIC_QC_EXE}" \ + --input "\$IN_PATH" \ + --min_area "${min_area}" \ + --min_total "${min_total}" \ + --area_quantile "${area_quantile}" \ + --total_quantile "${total_quantile}" \ + --output "\$OUT_PATH" + ;; + adaptive_quantile|quantile) + "${ADAPTIVE_QC_EXE}" \ + --input "\$IN_PATH" \ + --min_area "${min_area}" \ + --min_total "${min_total}" \ + --area_quantile "${area_quantile}" \ + --total_quantile "${total_quantile}" \ + --output "\$OUT_PATH" + ;; + *) + echo "ERROR: Unknown QC method: \$METHOD" >&2 + echo "Use one of: no_qc, fixed_threshold, adaptive_quantile" >&2 + exit 1 + ;; + esac + """ +} + +process normalize { + tag "normalization_${params.normalization_method ?: 'total_log1p'}" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path adata_in + + output: + path "cell_by_protein_norm.h5ad", emit: adata + + script: + def method = params.normalization_method ?: 'total_log1p' + def scale = params.containsKey('scale') ? params.scale : 10000 + def cofactor = params.containsKey('cofactor') ? params.cofactor : 5.0 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + METHOD="${method}" + IN_PATH="\$(realpath '${adata_in}')" + OUT_PATH="\$PWD/cell_by_protein_norm.h5ad" + + case "\$METHOD" in + no_normalization|none|raw) + "${NO_NORMALIZATION_EXE}" \ + --input "\$IN_PATH" \ + --scale "${scale}" \ + --cofactor "${cofactor}" \ + --output "\$OUT_PATH" + ;; + total_log1p|total_log1p_normalization|log1p|total) + "${TOTAL_LOG1P_NORMALIZATION_EXE}" \ + --input "\$IN_PATH" \ + --scale "${scale}" \ + --cofactor "${cofactor}" \ + --output "\$OUT_PATH" + ;; + arcsinh|arcsinh_normalization|asinh) + "${ARCSINH_NORMALIZATION_EXE}" \ + --input "\$IN_PATH" \ + --scale "${scale}" \ + --cofactor "${cofactor}" \ + --output "\$OUT_PATH" + ;; + *) + echo "ERROR: Unknown normalization method: \$METHOD" >&2 + echo "Use one of: no_normalization, total_log1p, arcsinh" >&2 + exit 1 + ;; + esac + """ +} + +process compute_basic_metrics { + tag "metrics_basic" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path adata_in - // merge all of the output data - output_ch = score_ch - | mix(meta_ch) - | joinStates{ ids, states -> - def mergedStates = states.inject([:]) { acc, m -> acc + m } - [ids[0], mergedStates] + output: + path "metrics_basic.h5ad", emit: metrics + + script: + def cov_thr = params.cov_thr ?: 0.1 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + IN_PATH="\$(realpath '${adata_in}')" + OUT_PATH="\$PWD/metrics_basic.h5ad" + + "${METRICS_BASIC_EXE}" \ + --input "\$IN_PATH" \ + --cov_thr "${cov_thr}" \ + --output "\$OUT_PATH" + """ +} + +process compute_marker_distribution_metrics { + tag "metrics_marker_distribution" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path adata_in + + output: + path "metrics_marker_distribution.h5ad", emit: metrics + + script: + def layer = params.metric_marker_layer ?: 'counts' + def zero_threshold = params.containsKey('zero_threshold') ? params.zero_threshold : 0.0 + def coverage_threshold = params.cov_thr ?: 0.1 + def low_quantile = params.containsKey('metric_low_quantile') ? params.metric_low_quantile : 0.01 + def high_quantile = params.containsKey('metric_high_quantile') ? params.metric_high_quantile : 0.99 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + IN_PATH="\$(realpath '${adata_in}')" + OUT_PATH="\$PWD/metrics_marker_distribution.h5ad" + + "${METRICS_MARKER_DISTRIBUTION_EXE}" \ + --input "\$IN_PATH" \ + --layer "${layer}" \ + --zero_threshold "${zero_threshold}" \ + --coverage_threshold "${coverage_threshold}" \ + --low_quantile "${low_quantile}" \ + --high_quantile "${high_quantile}" \ + --output "\$OUT_PATH" + """ +} + +process compute_normalization_diagnostics { + tag "metrics_normalization_diagnostics" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path adata_in + + output: + path "metrics_normalization_diagnostics.h5ad", emit: metrics + + script: + def raw_layer = params.metric_raw_layer ?: 'counts' + def normalized_layer = params.metric_normalized_layer ?: 'normalized' + def max_cells = params.containsKey('metric_max_cells') ? params.metric_max_cells : 5000 + def random_state = params.containsKey('metric_random_state') ? params.metric_random_state : 0 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + IN_PATH="\$(realpath '${adata_in}')" + OUT_PATH="\$PWD/metrics_normalization_diagnostics.h5ad" + + "${METRICS_NORMALIZATION_DIAGNOSTICS_EXE}" \ + --input "\$IN_PATH" \ + --raw_layer "${raw_layer}" \ + --normalized_layer "${normalized_layer}" \ + --max_cells "${max_cells}" \ + --random_state "${random_state}" \ + --output "\$OUT_PATH" + """ +} + +process compute_segmentation_quality_metrics { + tag "metrics_segmentation_quality" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path adata_in + + output: + path "metrics_segmentation_quality.h5ad", emit: metrics + + script: + def layer = params.metric_marker_layer ?: 'counts' + def nuclear_markers = params.nuclear_markers ?: 'null' + def exclusive_pairs = params.exclusive_pairs ?: 'null' + def n_neighbors = params.containsKey('n_neighbors') ? params.n_neighbors : 6 + def max_cells = params.containsKey('metric_max_cells') ? params.metric_max_cells : 5000 + def random_state = params.containsKey('metric_random_state') ? params.metric_random_state : 0 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + "${METRICS_SEGMENTATION_QUALITY_EXE}" \ + --input "\$(realpath '${adata_in}')" \ + --layer "${layer}" \ + --nuclear_markers "${nuclear_markers}" \ + --exclusive_pairs "${exclusive_pairs}" \ + --n_neighbors "${n_neighbors}" \ + --max_cells "${max_cells}" \ + --random_state "${random_state}" \ + --output "\$PWD/metrics_segmentation_quality.h5ad" + """ +} + +process compute_technical_covariate_metrics { + tag "metrics_technical_covariates" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path adata_in + + output: + path "metrics_technical_covariates.h5ad", emit: metrics + + script: + def layer = params.metric_normalized_layer ?: 'normalized' + def covariate = params.metric_covariate ?: 'area' + def n_pcs = params.containsKey('metric_n_pcs') ? params.metric_n_pcs : 10 + def max_cells = params.containsKey('metric_max_cells') ? params.metric_max_cells : 5000 + def random_state = params.containsKey('metric_random_state') ? params.metric_random_state : 0 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + "${METRICS_TECHNICAL_COVARIATES_EXE}" \ + --input "\$(realpath '${adata_in}')" \ + --layer "${layer}" \ + --covariate "${covariate}" \ + --n_pcs "${n_pcs}" \ + --max_cells "${max_cells}" \ + --random_state "${random_state}" \ + --output "\$PWD/metrics_technical_covariates.h5ad" + """ +} + +process compute_signal_background_metrics { + tag "metrics_signal_background" + publishDir PUBLISH_DIR, mode: 'copy' + + input: + path corrected_image + path segmentation_labels + path corrected_markers + path adata_qc + + output: + path "metrics_signal_background.h5ad", emit: metrics + + script: + def dilation_iterations = params.containsKey('dilation_iterations') ? params.dilation_iterations : 3 + def norm_percentile = params.containsKey('norm_percentile') ? params.norm_percentile : 95.0 + def random_state = params.containsKey('metric_random_state') ? params.metric_random_state : 0 + """ + set -euo pipefail + export VIASH_ENGINE_TYPE=${VIASH_ENGINE} + + "${METRICS_SIGNAL_BACKGROUND_EXE}" \ + --input_image "\$(realpath '${corrected_image}')" \ + --input_labels "\$(realpath '${segmentation_labels}')" \ + --input_markers "\$(realpath '${corrected_markers}')" \ + --input_qc "\$(realpath '${adata_qc}')" \ + --dilation_iterations "${dilation_iterations}" \ + --norm_percentile "${norm_percentile}" \ + --random_state "${random_state}" \ + --output "\$PWD/metrics_signal_background.h5ad" + """ +} + +workflow run_wf { + input_zarr = params.zarr ?: params.input_zarr + + if (input_zarr == null) { + error "No input Zarr provided. Use --zarr or --input_zarr." } - emit: - output_ch + zarr_ch = Channel.value(file(input_zarr)) + + corrected = image_correction(zarr_ch) + segmentation = segment_cells(corrected.image, zarr_ch) + aggregated = aggregate_proteins(corrected.image, segmentation.labels, corrected.markers) + qc = qc_filter(aggregated.adata) + norm = normalize(qc.adata) + basic_metrics = compute_basic_metrics(norm.adata) + marker_metrics = compute_marker_distribution_metrics(norm.adata) + normalization_metrics = compute_normalization_diagnostics(norm.adata) + segmentation_quality_metrics = compute_segmentation_quality_metrics(norm.adata) + technical_covariate_metrics = compute_technical_covariate_metrics(norm.adata) + signal_background_metrics = compute_signal_background_metrics( + corrected.image, segmentation.labels, corrected.markers, qc.adata + ) + + basic_metrics.metrics.view { "Done. Basic metrics written to: ${it}" } + marker_metrics.metrics.view { "Done. Marker distribution metrics written to: ${it}" } + normalization_metrics.metrics.view { "Done. Normalization diagnostics written to: ${it}" } + segmentation_quality_metrics.metrics.view { "Done. Segmentation quality metrics written to: ${it}" } + technical_covariate_metrics.metrics.view { "Done. Technical covariate metrics written to: ${it}" } + signal_background_metrics.metrics.view { "Done. Signal/background metrics written to: ${it}" } } diff --git a/src/workflows/run_benchmark/test.sh b/src/workflows/run_benchmark/test.sh new file mode 100644 index 0000000..89b1dab --- /dev/null +++ b/src/workflows/run_benchmark/test.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash + +set -euo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" +cd "$REPO_ROOT" + +OUTPUT_DIR="${OUTPUT_DIR:-output/temp_isp}" +ZARR_PATH="${ZARR_PATH:?Please set ZARR_PATH=/path/to/exemplar-001.zarr}" +IMAGE_KEY="${IMAGE_KEY:-null}" +IMAGE_CORRECTION_METHOD="${IMAGE_CORRECTION_METHOD:-percentile_background}" +SEGMENTATION_METHOD="${SEGMENTATION_METHOD:-threshold}" +AGGREGATION_METHOD="${AGGREGATION_METHOD:-mean}" +QC_METHOD="${QC_METHOD:-fixed_threshold}" +NORMALIZATION_METHOD="${NORMALIZATION_METHOD:-total_log1p}" +LABELS_KEY="${LABELS_KEY:-null}" +# Components must be built with the same engine the workflow runs them under. +VIASH_ENGINE="${VIASH_ENGINE:-docker}" + +mkdir -p "$OUTPUT_DIR" + +echo "Using repository: $REPO_ROOT" +echo "Using Zarr input: $ZARR_PATH" +echo "Using output dir: $OUTPUT_DIR" +echo "Using image correction method: $IMAGE_CORRECTION_METHOD" +echo "Using segmentation method: $SEGMENTATION_METHOD" +echo "Using aggregation method: $AGGREGATION_METHOD" +echo "Using QC method: $QC_METHOD" +echo "Using normalization method: $NORMALIZATION_METHOD" +echo "Using viash engine: $VIASH_ENGINE" + +if [ ! -d "$ZARR_PATH" ]; then + echo "ERROR: ZARR_PATH does not exist or is not a directory: $ZARR_PATH" >&2 + echo "Set it explicitly, for example:" >&2 + echo "ZARR_PATH=/path/to/exemplar-001.zarr bash src/workflows/run_benchmark/test.sh" >&2 + exit 1 +fi + +echo "Building required Viash components..." + +for q in \ + "methods_image_correction/no_image_correction" \ + "methods_image_correction/basic_image_correction" \ + "methods_image_correction/flatfield_image_correction" \ + "methods_proteomics_segmentation/basic_threshold_segmentation" \ + "methods_proteomics_segmentation/watershed_expand_segmentation" \ + "methods_proteomics_segmentation/reference_segmentation" \ + "methods_protein_aggregation/mean_intensity_aggregation" \ + "methods_protein_aggregation/sum_intensity_aggregation" \ + "methods_protein_aggregation/median_intensity_aggregation" \ + "methods_proteomics_qc/no_qc_filter" \ + "methods_proteomics_qc/basic_qc_filter" \ + "methods_proteomics_qc/adaptive_quantile_qc_filter" \ + "methods_proteomics_normalization/no_normalization" \ + "methods_proteomics_normalization/total_log1p_normalization" \ + "methods_proteomics_normalization/arcsinh_normalization" \ + "metrics_proteomics/basic" \ + "metrics_proteomics/marker_distribution" \ + "metrics_proteomics/normalization_diagnostics" \ + "metrics_proteomics/segmentation_quality" \ + "metrics_proteomics/technical_covariates" \ + "metrics_proteomics/signal_background" +do + viash ns build --engine "$VIASH_ENGINE" --query "$q" +done + +echo "Running ISP preprocessing benchmark workflow..." + +export NXF_VER="${NXF_VER:-24.04.3}" + +nextflow run src/workflows/run_benchmark/main.nf \ + -entry run_wf \ + -resume \ + --zarr "$ZARR_PATH" \ + --outdir "$OUTPUT_DIR" \ + --viash_engine "$VIASH_ENGINE" \ + --image_key "$IMAGE_KEY" \ + --image_correction_method "$IMAGE_CORRECTION_METHOD" \ + --background_percentile "${BACKGROUND_PERCENTILE:-1.0}" \ + --rescale_max "${RESCALE_MAX:-false}" \ + --flatfield_sigma "${FLATFIELD_SIGMA:-50.0}" \ + --flatfield_mode "${FLATFIELD_MODE:-subtract}" \ + --segmentation_method "$SEGMENTATION_METHOD" \ + --labels_key "$LABELS_KEY" \ + --nuclei_channel "${NUCLEI_CHANNEL:-0}" \ + --threshold_method "${THRESHOLD_METHOD:-quantile}" \ + --threshold_quantile "${THRESHOLD_QUANTILE:-0.90}" \ + --smoothing_sigma "${SMOOTHING_SIGMA:-1.0}" \ + --segmentation_min_area "${SEGMENTATION_MIN_AREA:-30}" \ + --hole_area "${HOLE_AREA:-64}" \ + --expansion_distance "${EXPANSION_DISTANCE:-5}" \ + --aggregation_method "$AGGREGATION_METHOD" \ + --stat "${STAT:-mean}" \ + --qc_method "$QC_METHOD" \ + --min_area "${MIN_AREA:-50}" \ + --min_total "${MIN_TOTAL:-0.0}" \ + --area_quantile "${AREA_QUANTILE:-0.05}" \ + --total_quantile "${TOTAL_QUANTILE:-0.05}" \ + --normalization_method "$NORMALIZATION_METHOD" \ + --scale "${SCALE:-10000}" \ + --cofactor "${COFACTOR:-5.0}" \ + --transform "${TRANSFORM:-log1p}" \ + --cov_thr "${COV_THR:-0.1}" \ + --metric_marker_layer "${METRIC_MARKER_LAYER:-counts}" \ + --zero_threshold "${ZERO_THRESHOLD:-0.0}" \ + --metric_low_quantile "${METRIC_LOW_QUANTILE:-0.01}" \ + --metric_high_quantile "${METRIC_HIGH_QUANTILE:-0.99}" \ + --metric_raw_layer "${METRIC_RAW_LAYER:-counts}" \ + --metric_normalized_layer "${METRIC_NORMALIZED_LAYER:-normalized}" \ + --metric_max_cells "${METRIC_MAX_CELLS:-5000}" \ + --nuclear_markers "${NUCLEAR_MARKERS:-null}" \ + --exclusive_pairs "${EXCLUSIVE_PAIRS:-null}" \ + --n_neighbors "${N_NEIGHBORS:-6}" \ + --metric_covariate "${METRIC_COVARIATE:-area}" \ + --metric_n_pcs "${METRIC_N_PCS:-10}" \ + --dilation_iterations "${DILATION_ITERATIONS:-3}" \ + --norm_percentile "${NORM_PERCENTILE:-95.0}" \ + --metric_random_state "${METRIC_RANDOM_STATE:-0}" + +echo +echo "Workflow finished. Published outputs:" +ls -lh "$OUTPUT_DIR" diff --git a/thumbnail.svg b/thumbnail.svg index 087fa85..933750f 100644 --- a/thumbnail.svg +++ b/thumbnail.svg @@ -1,6 +1,58 @@ - - - - task thumbnail placeholder - replace with an svg renderable to 450x150 px + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + multiplexed image + segmentation + cell × protein + + Spatial Proteomics Preprocessing + + metrics