Repository navigation
Move to anndata 0.13 with sharded zarr - #19
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
Move to anndata 0.13, which writes zarr v3 with sharded arrays. Merge after openproblems-bio/core#51 is released as base images v1.3.0; until then CI still builds on anndata 0.12.
_viash.yaml: setANNDATA_AUTO_SHARD_ZARR_V3=1and dropANNDATA_ZARR_WRITE_FORMAT=3, which is deprecated in 0.13 and makesimport anndatafail in 0.13.4.varindex after thefeature_id(orfeature_name) column. anndata 0.13 refuses to write a dataframe whose index is named after a column unlesspd.Series.equals()holds, and after reading a store the index isstrwhile the column isobject, so every component downstream of the loader failed.log_scran_pooling: drop theanndataR package (removed frombase_rin core#51) and read and write through Python anndata, since anndataR can't read sharded zarr yet.atac_tfidf: drop thenumpy<2pin, which held it on anndata 0.12 and pandas 2.Ajn1Lo8zZomVzKtQ0005); the arrays are sharded.Tested locally with the base images from core#51: 43/45 component tests pass (the 2 missing were already disabled).
Not changed here:
log_scran_poolingmultiplies the counts by the size factors where scran divides by them. It's not part of the new scRNA-seq pipeline, so I left it as is.Issue ticket number and link
n/a
Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!