diff --git a/STATUS.md b/STATUS.md index 4544224..8f10a52 100644 --- a/STATUS.md +++ b/STATUS.md @@ -7,6 +7,7 @@ moves; the plan of record for the current item is in the section below. | Version | What | Issues | |---|---|---| +| 0.17.1 | Visium HD, one bin size at a time, with the new `spatial` extra (pyarrow) | #81 | | 0.17.0 | `convert` reads Visium (Space Ranger 1–4) and Xenium (XOA 1–4) with positions, images and Xenium's analysis | #79 | | 0.16.2 | Documentation site (guides, troubleshooting, a page per format); `export image` writes PNG to stdout without `-o` | #76, #77 | | 0.16.1 | `subset`, `split` and the MuData export narrow categoricals (and their colours) as anndata's subsetting does | #74 | @@ -20,24 +21,8 @@ The conversion epic's sub-issues (#21–#30) are all done and ticked in #20. ## In progress -### Visium HD (`feat/visium-hd`, for 0.17.1) - -Decisions: -- One bin size per output, `--bin-size` (8 µm by default, as 10x - recommends); a bin folder given directly reads as it is. -- pyarrow comes with a new `spatial` extra, in the image too, and is - imported only by `formats/parquet.py`. -- The Parquet positions are read as a buffered stream, a set of columns per - pass; the join holds barcodes as bytes; images are written in strips as - they are read. 2 µm on 10x's tiny sample: 3.5 s, 470 MB without images. - -Steps: -- [x] `formats/parquet.py`, HD in `visium.py`, `--bin-size` -- [x] Tests: HD fixture, flags and refusals, the missing extra, a performance - guard, a scanpy and pandas comparison -- [x] Real data: 10x's SR 4.0.1 tiny HD sample at 2, 8 and 16 µm -- [x] Docs -- [ ] PR, review, merge; release 0.17.1 +Nothing. Visium HD is out in 0.17.1, completing the spatial import; +Space Ranger 4's cell segmentation is first among the follow-ups below. ## Follow-ups noticed diff --git a/docs/changelog.md b/docs/changelog.md index 230013f..26cc1e6 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -8,6 +8,38 @@ nav_order: 7 Notable changes to `adata-cli`. Versions are `MAJOR.MINOR.PATCH`; tags carry no `v` prefix. +## 0.17.1 + +Visium HD joins standard Visium and Xenium: `convert` reads a Space Ranger +3 or 4 HD output at one bin size, with its bins' positions, images and +scale factors, through a new optional `spatial` extra (pyarrow) for the +Parquet positions. + +### Added + +- **Visium HD** ([#81](https://github.com/cellgeni/adata-cli/pull/81)). + `adata convert outs -o bins.h5ad` reads the 8 µm bins, as 10x recommends + for analysis; `--bin-size 2` or `16` picks another size, and a bin folder + given directly reads as it is. The layout is standard Visium's: obs + `in_tissue`, `array_row`, `array_col`; `obsm["spatial"]` in + full-resolution pixels; the bin folder's images and scale factors in + `uns["spatial"]`. Checked in CI against scanpy with pandas reading the + positions, and on 10x's SR 4.0.1 tiny sample at every bin size. + - The positions are read as a buffered stream, the barcodes and the + numbers in separate passes. On that sample the 2 µm bins (11.2 million + positions, 330,035 in the matrix) convert in 3.5 s with a 470 MB peak + without images, where pandas alone takes about 1 GB to read them. +- **The `spatial` extra**: `pip install 'pyadata-cli[spatial]'` adds + pyarrow, which only Visium HD needs; the container image has it. + +### Changed + +- **Images are written a strip of rows at a time**, as they are read, rather + than loaded before the write starts: Visium HD's hires image is 6,000 + pixels, some 100 MB. +- **Barcodes are matched as bytes**, a quarter of the memory of numpy's + unicode, for Visium, Visium HD and Xenium alike. + ## 0.17.0 `convert` reads 10x's spatial outputs with their coordinates: a Space diff --git a/pyproject.toml b/pyproject.toml index 233ab1e..bea9d45 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -3,7 +3,7 @@ # so the distribution is published as `pyadata-cli`. The import package # and the command are both still `adata`. name = "pyadata-cli" -version = "0.17.0" +version = "0.17.1" description = "Streaming CLI for exploring and editing large AnnData .h5ad and .zarr stores" readme = "README.md" requires-python = ">=3.12" diff --git a/uv.lock b/uv.lock index 0238921..e10a297 100644 --- a/uv.lock +++ b/uv.lock @@ -486,7 +486,7 @@ wheels = [ [[package]] name = "pyadata-cli" -version = "0.17.0" +version = "0.17.1" source = { editable = "." } dependencies = [ { name = "h5py" },