Skip to content

Move to anndata 0.13 with sharded zarr - #19

Merged
rcannood merged 6 commits into
mainfrom
anndata-0.13
Nov 19, 2026
Merged

rcannood merged 6 commits into
mainfrom
anndata-0.13

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

Move to anndata 0.13, which writes zarr v3 with sharded arrays. Merge after openproblems-bio/core#51 is released as base images v1.3.0; until then CI still builds on anndata 0.12.

  • _viash.yaml: set ANNDATA_AUTO_SHARD_ZARR_V3=1 and drop ANNDATA_ZARR_WRITE_FORMAT=3, which is deprecated in 0.13 and makes import anndata fail in 0.13.4.
  • Census and NeurIPS 2021/2022 loaders: don't name the var index after the feature_id (or feature_name) column. anndata 0.13 refuses to write a dataframe whose index is named after a column unless pd.Series.equals() holds, and after reading a store the index is str while the column is object, so every component downstream of the loader failed.
  • log_scran_pooling: drop the anndata R package (removed from base_r in core#51) and read and write through Python anndata, since anndataR can't read sharded zarr yet.
  • atac_tfidf: drop the numpy<2 pin, which held it on anndata 0.12 and pandas 2.
  • Regenerated the CELLxGENE test resources with anndata 0.13 (mouse pancreas Ajn1Lo8zZomVzKtQ0005); the arrays are sharded.

Tested locally with the base images from core#51: 43/45 component tests pass (the 2 missing were already disabled).

Not changed here: log_scran_pooling multiplies the counts by the size factors where scran divides by them. It's not part of the new scRNA-seq pipeline, so I left it as is.

Issue ticket number and link

n/a

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
    • Documentation
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

@rcannood
rcannood merged commit d078703 into main Nov 19, 2026
45 of 69 checks passed
@rcannood
rcannood deleted the anndata-0.13 branch November 19, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant