Skip to content

Combine open OpenDDA PRs and guide biologists from upload to analysis - #52

Open
t0mdavid-m wants to merge 27 commits into
mainfrom
claude/project-thread-hepw99
Open

t0mdavid-m wants to merge 27 commits into
mainfrom
claude/project-thread-hepw99

Conversation

@t0mdavid-m

Copy link
Copy Markdown
Member

Requested by Tom · project thread

One PR for all open OpenDDA work, plus a pass over every page for a Biology PhD audience: plain-language explanations, sensible defaults, and a navigation that leads from upload to analysis with the important results up front.

Combined PRs

Merged unchanged (conflicts in app.py, results_filtered.py, results_helpers.py and WorkflowTest.py were import/page-list unions):

Not included because main already has their content or they are stale against it: #38 (superseded by #41), #37 (rest landed in #40), #8, #15, #26, #28, #29. They can be closed with this one, as can #42–#51 and #1–#5.

Navigation

Sidebar section Pages
Start here Home
Step 1 · Set up and run Upload data, Describe experiment, Run analysis
Step 2 · Main results Run summary, Compare groups (new), Volcano plot, Identified proteins
Step 3 · Explore further Sample similarity (PCA), GO enrichment, Clustered heatmap, Protein abundances, Per-sample counts, Heatmap
Export Download results
Fine-tune the comparison 1. Filter proteins, 2. Missing values, 3. Normalize, 4. Statistical test
Quality control and details Identification quality, Peptide-spectrum matches, Mass shifts, Rescored hits (Percolator), Raw search hits (Comet)

The sidebar shows the first ten pages (through GO enrichment) and folds the rest behind "View more", so the expert and QC pages are still there but less prominent.

Guidance on every page

  • src/common/guide.py: page_intro (what the page answers, background in an expander), next_steps footer, needs_results for pages opened before a run.
  • Home rewritten as a three-step guide with a "Which page answers my question?" table.
  • Every page's text rewritten in plain words (PSM, FDR, fold change, adjusted p-value, PCA, GO terms, imputation explained where they appear).

Sensible defaults

  • Compare groups (content/compare_groups.py, src/common/differential.py) runs filtering → imputation → normalization → statistics with the recommended settings, continuing from any step already applied on the Fine-tune pages. Volcano, PCA, heatmaps and GO enrichment compute it on demand instead of telling users to go through four pages first. The page states which direction a positive log2 fold change means (openms-insight subtracts the alphabetically first group).
  • Describe experiment: samples and groups is the first tab, tabs are named by purpose, the 11 TMT tool tabs are regrouped into 6, and the help boxes are rewritten (two of them described the wrong tool).
  • LFQ search defaults to Orbitrap HCD (high_res, 0.01 Da fragment tolerance) instead of ion trap settings; the demo data is Orbitrap HCD.
  • All uploaded runs are preselected, and the demo files get their groups (UPS2_ratio01 / UPS2_ratio10), so a fresh workspace runs the demo end to end without input.

Fixes found on the way

  • Volcano, PCA, heatmap and clustered heatmap stayed on "Loading..." forever (on main too): regenerate_cache=True rebuilt the cache on every rerun, so the component kept requesting data. Caches are now keyed on the plotted data (plot_cache in insight_tables.py).
  • GO enrichment results were drawn only inside the button block and vanished on the next rerun; they are kept in session state and shown in paginated tables.
  • Removed a debug st.write of the results path in the TMT configuration.

Testing

  • pytest tests/ test_gui.py: 213 passed (6 new in tests/test_differential.py).
  • pylint --errors-only: only the existing pyopenms E0611 notices.
  • Ran the demo data end to end with OpenMS 3.5 TOPP tools (bioconda) through Upload → Configure (defaults only) → Run, then opened every page in headless Chromium. Volcano, PCA and both heatmaps render. The demo finds no significant proteins at the default cut-offs because the trimmed demo files contain no quantified UPS2 proteins.
  • GO enrichment was not exercised (needs MyGene.info access).

Screenshots are in the project files under quantms-web-guided/.


Generated by Claude Code

claude added 23 commits October 20, 2026 05:12
…SMs per file

First step towards feature parity with the ProSE web app's QC tab.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D5Wo2FHPpi5a8X6GppNxEk
… PSMs

Second step towards feature parity with the ProSE web app (protein table,
coverage map and drill-down).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D5Wo2FHPpi5a8X6GppNxEk
…ected PSM

Third step towards feature parity with the ProSE web app's PSM view.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D5Wo2FHPpi5a8X6GppNxEk
…esults

A new "Add common contaminants (cRAP)" option (on by default) appends the
116 cRAP proteins to the search database with CON__ accessions. With
generated decoys, DecoyDatabase reads the FASTA and the contaminants
together, so contaminants get decoys too. A FASTA with its own decoys gets
contaminant decoys in its decoy style. The database step is now its own
method, prepare_search_database().

PSM tables gain a Contaminant column. get_abundance_data() leaves
contaminant proteins out by default, so downstream analysis never sees
them. The Abundance page flags them and has a "Show contaminants" toggle.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D5Wo2FHPpi5a8X6GppNxEk
Workflow Results > Overview shows key figures for the last run
(quantified proteins, peptides, target PSMs at the PSM FDR, PSMs per MS2
spectrum, median precursor error, files, samples, conditions). Below them,
a per-file table lists MS2 spectra, target and decoy PSMs, ID rate,
peptides, median precursor mass error, whether Percolator rescored the
file, and CometAdapter's search time.

src/common/overview.py reads only what the run already wrote: the input
mzML files (MS2 spectra counted without peak data), the Percolator and
IDFilter idXMLs, and the run-time log. The page caches it on file mtimes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D5Wo2FHPpi5a8X6GppNxEk
Replace the High Res MS / Low Res MS presets, which only set the
label-free Comet search, with three instrument presets that set the
fragment bins, precursor tolerance and isotope errors for both the
label-free and the TMT search. A test checks every Comet preset against
the valid strings and fragment-bin checks of CometAdapter.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01D5Wo2FHPpi5a8X6GppNxEk
Same change as the streamlit-template PR: the images ship the .NET Framework
ThermoRawFileParser that FileConverter runs through `mono`, but not Mono.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FkEGenJns3nTKscxHae7JJ
A new "Import from PRIDE" tab on the File Upload page looks up a PRIDE
accession through the PRIDE Archive v3 API, shows the project metadata and
its files, and downloads the selected runs, FASTA and SDRF files in a
background process with progress and cancel. mzML files are used as they are,
Thermo RAW files are converted with FileConverter. Downloads go over HTTPS
and resume partial files.

An SDRF file (from the dataset, or uploaded) can then set up the workflow:
the imported files are selected, the analysis mode, LFQ sample groups or TMT
channel groups, and the Comet enzyme, modifications (resolved against the
pyOpenMS ModificationsDB) and tolerances are written to the parameters.

Online deployments cap a dataset import at pride_import.max_download_gb_online
(settings.json, 50 GB by default).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FkEGenJns3nTKscxHae7JJ
…o claude/project-thread-hepw99

# Conflicts:
#	app.py
…claude/project-thread-hepw99

# Conflicts:
#	content/results_filtered.py
#	src/common/results_helpers.py
…claude/project-thread-hepw99

# Conflicts:
#	app.py
…to claude/project-thread-hepw99

# Conflicts:
#	src/WorkflowTest.py
- Navigation follows the user's path: Start here, Set up and run, Main
  results (Run summary, Compare groups, Volcano plot, Identified proteins),
  Explore further; Fine-tune and Quality control pages fold away last.
- New Compare groups page runs filtering, imputation, normalization and
  the statistical test with the recommended settings
  (src/common/differential.py); volcano, PCA, heatmaps and GO enrichment
  compute it on demand instead of sending users to four pages first.
- Every page opens with what it answers in plain words, folds background
  for non-specialists into an expander, and ends with where to go next
  (src/common/guide.py).
- Configure: samples and groups first, tabs named by purpose, TMT tool
  tabs regrouped, plain-language help; LFQ search defaults to Orbitrap
  HCD; all uploaded runs are preselected and the demo files get groups,
  so the demo runs as shipped.
- Volcano, PCA and heatmaps no longer hang on "Loading...": their insight
  caches are keyed on content instead of regenerated on every rerun.
- GO enrichment results survive reruns and use the paginated table.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FkEGemnfPye3pzon8Q9pur
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 57d63dee-c9b6-41ad-a7ce-dc99668c536b
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@t0mdavid-m t0mdavid-m self-assigned this Oct 29, 2026
- Samples or channels without a group are listed as "Unassigned" and left
  out of the comparison. Before, an LFQ sample without a group vanished
  silently, and an Unassigned group became a third group that skewed the
  test and the direction of log2FC.
- Fine-tune pages and PCA take their sample columns from the group map, so
  the TMT report columns n_proteins, n_peptides and protein_score no longer
  count as samples.
- Group changes on Describe experiment refresh the comparison; they apply
  without a new run.
- TMT channels are named by their reporter ion (sample3[127C] instead of
  abundance_sample3[unknown]), in the tables and in the group form.
- The TMT protein table shows one tidy table like LFQ; texts on Upload,
  Compare groups and Missing values cover both experiment types.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FkEGemnfPye3pzon8Q9pur
apt-get ran without DEBIAN_FRONTEND=noninteractive, so when a package
pulled in tzdata the build stopped at the interactive timezone question.
Set it, with TZ=Etc/UTC, in the first stage of every Dockerfile; later
stages inherit it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FkEGemnfPye3pzon8Q9pur
A CRLF checkout (core.autocrlf=true) gives docker/entrypoint.sh a
shebang ending in a carriage return, so the container fails with
"exec /app/entrypoint.sh: no such file or directory". Same rule as
the streamlit-template's .gitattributes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FkEGemnfPye3pzon8Q9pur
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants