Repository navigation
convert: Visium (Space Ranger) and Xenium spatial outputs - #79
Merged
Merged
Conversation
An outs/ folder (or the run folder holding it) converts with its spot positions in obs and obsm['spatial'], and its images, scale factors and metadata in uns['spatial'][library_id], laid out as squidpy.read.visium lays them out; images stay the PNGs' uint8 pixels. Positions are matched to the matrix's barcodes as they are scanned, and written in its order through core/scatter.py when they come in another. Before this, an outs/ folder was taken for a Cell Ranger 2 MTX layout and refused with 'pick one with --genome'. table.py reads files with no header line (Space Ranger 1) and hands each block's index to a caller during the scan; its column creation and value typing are reusable. Visium HD is refused by name until its Parquet positions are read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bundle (recognised by experiment.xenium) converts with cells.csv.gz's columns in obs, typed as pandas reads them, the centroids in obsm['spatial'] in microns, and Xenium's own analysis: graph-based clusters as obs['graphclust'], UMAP and PCA into obsm. Xenium writes the cell table in the matrix's order, so it is written straight through; a table in another order goes through core/scatter.py, and cells the matrix lacks are left out with their categories. Performance guards cover both paths, and Visium's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Xenium import test_external_readers checks Visium (Space Ranger 1, 2 and 3 with decimal positions) against squidpy.read.visium, values and dtypes, and Xenium against 10x's scanpy recipe, in order and shuffled. import-xenium times adata-cli against that recipe on a bundle built from the benchmark store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every real bundle checked (XOA 3.0 and 4.0) packs analysis/ as analysis.tar.gz, so the clusters, UMAP and PCA were never read without unpacking first. The three tables used are copied out of the archive by name into a temporary folder, never extracted where the archive says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Format pages for both, with their mappings, flags, how they stream and where they differ from squidpy and spatialdata-io; the formats tables, the convert guide and help, Troubleshooting for the new refusals, and AGENTS.md and Developers for core/scatter.py and the FORMATS order. The 10x HDF5 page says that a Space Ranger or Xenium matrix given alone converts without its positions. STATUS records the work and its follow-ups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Contributor
Test Results (py3.13)1 970 tests 1 970 ✅ 5m 4s ⏱️ Results for commit 82a3abe. ♻️ This comment has been updated with latest results. |
Contributor
Test Results (py3.12)1 970 tests 1 970 ✅ 5m 13s ⏱️ Results for commit 82a3abe. ♻️ This comment has been updated with latest results. |
- core/scatter.py: a layout that keeps no source row no longer asks HDF5 for a chunk larger than its empty scratch, which crashed the h5ad write. - A table with a row for none of the matrix's barcodes (positions, cells, an analysis table) is refused as another run's. - Refused at plan time, not in the write after staging began: a coordinate, centroid or projection column that holds text, a clusters table with no Cluster column, a --library-id that cannot be a key of uns (empty, or with '/'). Missing numbers become NaN. - A CSV saved with a byte order mark keeps its header (table.py and the positions' header check). - analysis.tar.gz unpacks beside the output, not into the system's temporary folder, and an archive holding none of the tables says so. - --from xenium refuses a file other than cell_feature_matrix.h5. - graphclust drops categories only cells the matrix lacks used. - The Xenium page says how the archive is read, and when text becomes a categorical; Troubleshooting covers the new refusals. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reads 10x's spatial outputs with their coordinates, not just their matrix: standard Visium from Space Ranger 1–4, and Xenium from XOA 1–4. No new dependency. Visium HD waits for a later release, because its positions exist only as Parquet. Part of #31.
What converts
Visium.
adata convert outs -o section.h5adgives whatsquidpy.read.visiumgives:in_tissue,array_row,array_col;obsm["spatial"]:[x, y]in full-resolution pixels;uns["spatial"][library]: the images as uint8, the scale factors and the metadata.New flags:
--library-idand--no-images.--unfilterednow readsraw_feature_bc_matrix.h5for Visium too.Xenium.
adata convert bundle -o cells.h5adgives:cells.csv.gzcolumns, typed as pandas reads them;obsm["spatial"]: the centroids, in µm;obs["graphclust"],obsm["X_umap"]andobsm["X_pca"], read fromanalysis.tar.gzas Xenium ships it.A bug this fixes. An
outs/folder used to be taken for a Cell Ranger 2 MTX layout and refused with "pick one with --genome".visiumandxeniumnow come before10x-mtxinFORMATS. A bare.h5is still a plain 10x matrix.How it streams
conversion/spatial.py. It matches a table to the matrix's barcodes duringscan_table's pass, through the newon_indexhook.core/scatter.py, the bucket-through-scratch writer generalised from CellBender's.Tests
test_visium.py: Space Ranger 1 headerless, 2, and 3 with decimal positions; every flag and refusal; detection.test_xenium.py: integer and string ids; in order and shuffled; extra and missing cells; analysis from the folder or the archive, including an archive that tries to write outside its folder.test_scatter.pyand newtest_table.pycases.squidpy.read.visium, comparing values and dtypes.import-xeniumcase. At smoke tier, adata-cli took 0.32 s and 75 MB; the scanpy recipe took 1.76 s and 260 MB.uv run pytest -m "not integration" --cov=adata --cov-fail-under=90: 1,956 passed, coverage 95.4%. The local site build passes html-proofer.Docs
reference/formats/visium.mdandxenium.md.--help, Troubleshooting, Developers and AGENTS.md.analysis/for Visium;core/scatter.py.🤖 Generated with Claude Code