Repository navigation
Conversation
…zarr-developers#4611) * Narrow partial reads of uncompressed chunks within a single row When a selection touches one index along the first axis, apply the same row window along the next axis, and so on. arr[t, y0:y1, x0:x1] then fetches rows y0 to y1 of frame t instead of the whole frame, and a single item is fetched for a point. The fetched block keeps the narrowed axes with length 1, so the shifted selection applies to it unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add release note for zarr-developers#4611 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
…duplicating writes (zarr-developers#4649) * perf(zarr-indexing): group chunk points with an unstable sort and stop deduplicating writes Planning grouped each chunk's points with a stable sort so that every `IndexedSet` and `JointSet` row kept request order, and the writer relied on that order to make the last repeated write win. Both promises are dropped: the order of points within a row is unspecified, and a source cell a write addresses more than once receives one of its values, whole, without saying which. Callers who need a particular value remove repeats from the selection first. Grouping now uses the default sort behind `_group_by_key`, which also skips the sort for already-grouped joint keys, and the writer no longer deduplicates targets before its single fancy assignment or `np.put`. Tests that assumed last-wins now accept any written value, and a new test runs orthogonal, vectorized and mixed repeated writes with the grouping replaced by one that reverses each row. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-indexing): add changelog fragment for unspecified repeated-write order Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-indexing): number the changelog fragment for upstream PR 4649 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(zarr-indexing): group chunk keys with a 16-bit radix sort NumPy's stable sort is a linear-time radix sort for integers of at most 16 bits. `_group_by_key` shifts keys to start at zero and sorts them one 16-bit digit at a time: one pass when their range fits 16 bits, two least-significant-first passes when it fits 32 bits, and the default sort beyond. Grouping 1M random keys over 65,536 chunks drops from about 46 ms to 2 ms. The order within a group stays unspecified. The approach follows zarr-python zarr-developers#4643's `argsort_chunk_indices`, with a shift by the minimum key because grids may have negative chunk IDs. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(zarr-indexing): radix-sort chunk keys of any width `_group_by_key` fell back to the default sort above a 2**32 key range, which was slower than radix passes and, as zarr-python zarr-developers#4643 measured, slower at every key width. It now takes one 16-bit pass per 16 bits of the shifted key range, up to four, reading the shifted keys as uint64 so a range above 2**63 stays exact. Inputs under 128 keys per pass take one stable comparison sort, which beats the passes' fixed overhead. This mirrors zarr-developers#4643's `argsort_chunk_indices`; the order within a group stays unspecified. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-indexing): record why repeated write targets get no promised value Add a "Repeated write targets" design note: the library promises no value for a cell a write addresses more than once, chooses its internal order by performance (today a stable radix sort that happens to keep the last write), carries no order within a planning-table row, and does not deduplicate, because the pass would cost more than the assignment. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…developers#4650) Bumps the python-dependencies group with 5 updates: | Package | From | To | | --- | --- | --- | | [typer](https://github.com/fastapi/typer) | `0.27.3` | `0.29.1` | | [hypothesis](https://github.com/HypothesisWorks/hypothesis) | `6.170.14` | `6.170.15` | | [mkdocs-material[imaging]](https://github.com/squidfunk/mkdocs-material) | `9.8.0` | `9.8.1` | | [markdown-exec[ansi]](https://github.com/pawamoy/markdown-exec) | `1.12.6` | `1.13.0` | | [ruff](https://github.com/astral-sh/ruff) | `0.17.3` | `0.17.4` | Updates `typer` from 0.27.3 to 0.29.1 - [Release notes](https://github.com/fastapi/typer/releases) - [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md) - [Commits](fastapi/typer@0.27.3...0.29.1) Updates `hypothesis` from 6.170.14 to 6.170.15 - [Release notes](https://github.com/HypothesisWorks/hypothesis/releases) - [Commits](HypothesisWorks/hypothesis@v6.170.14...v6.170.15) Updates `mkdocs-material[imaging]` from 9.8.0 to 9.8.1 - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](squidfunk/mkdocs-material@9.8.0...9.8.1) Updates `markdown-exec[ansi]` from 1.12.6 to 1.13.0 - [Release notes](https://github.com/pawamoy/markdown-exec/releases) - [Changelog](https://github.com/pawamoy/markdown-exec/blob/main/CHANGELOG.md) - [Commits](pawamoy/markdown-exec@1.12.6...1.13.0) Updates `ruff` from 0.17.3 to 0.17.4 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.17.3...0.17.4) --- updated-dependencies: - dependency-name: typer dependency-version: 0.29.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: python-dependencies - dependency-name: hypothesis dependency-version: 6.170.15 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: python-dependencies - dependency-name: mkdocs-material[imaging] dependency-version: 9.8.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: python-dependencies - dependency-name: markdown-exec[ansi] dependency-version: 1.13.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: python-dependencies - dependency-name: ruff dependency-version: 0.17.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: python-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
numcodecs deprecates the Pickle codec and warns whenever one is constructed (zarr-developers/numcodecs#905). The tests for array.allow_unsafe_pickle_decode construct the codec on purpose, and pytest turns warnings into errors, so they would fail once numcodecs ships the deprecation. Apply a filterwarnings mark scoped to the deprecation message to those tests only. The filter matches nothing on numcodecs releases that do not warn. Assisted-by: ClaudeCode:claude-opus-5.5
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Ignore numcodecs'
Pickledeprecation warning in the tests that construct the pickle codec on purpose. Without this, those tests fail underfilterwarnings = "error"once numcodecs ships the deprecation (zarr-developers/numcodecs#905).What changes
numcodecs.Pickle.__init__emitDeprecationWarning("The Pickle codec is deprecated and will be removed in numcodecs 0.19.0. ..."). That covers direct construction andget_codecwhen array metadata namespickle.tests/test_v2.pyconstruct the codec to exercise thearray.allow_unsafe_pickle_decodeopt-in from feat: require opt-in to read data encoded with the pickle codec zarr-developers/zarr-python#4465: the five that call_pickle_compressed_array()(one is parametrized twice), plustest_pickle_codec_read_does_not_unpickle, which opens metadata withcompressor: {"id": "pickle"}.ignore_pickle_deprecation = pytest.mark.filterwarnings("ignore:The Pickle codec is deprecated:DeprecationWarning"), and applied it to those tests only. It did not add a globalpyproject.tomlentry, so a newPickleuse elsewhere still fails loudly.numcodecs>=0.16.6,!=0.17.0).Pickleoutside metadata-drivenget_codec.Why now
The "upstream" test environment installs numcodecs from git
main, so it starts failing as soon as zarr-developers/numcodecs#905 merges, before any numcodecs release.Verification
The agent tested against numcodecs built from the zarr-developers/numcodecs#905 branch:
pytest tests/test_v2.py -k picklegives 6 failed, 1 passed. The failures are exactly the six cases listed above, all on the deprecation warning.tests/test_v2.pypasses (35 tests).pytest tests -n 8, Python 3.13) gives 13209 passed, 1306 skipped, 9 xfailed, 0 failed.ruff check,ruff format, codespell and numpydoc-validation (via pre-commit) pass.mypy tests/test_v2.pypasses.Notes for upstreaming
changes/<PR>.misc.mdor apply the no-changelog label, whichever maintainers prefer for test-only changes.🤖 Generated with Claude Code