Repository navigation
fix: return the caller's out from decode(buf, out=out) in every codec - #907
Merged
Merged
Conversation
Add tests/test_codec_contract.py, which runs every registered codec through the behavior the Codec base class promises: encode and decode return buffers, decode(out=...) returns the caller's out, and get_config round-trips through from_config. Known violations are strict xfails, so fixing one forces its entry to be removed. Add tests/typing, where assert_type checks (verified by mypy and pyright in strict mode) sit next to runtime assertions (verified by pytest), and a `typing` CI job that runs both checkers plus the repository's own mypy configuration. The type checkers are pinned in a new `typing` dependency group. Assisted-by: ClaudeCode:claude-opus-5-5
The Codec docstring says decode writes into `out`, and callers expect `out` back. Ten codecs (Zlib, BZ2, LZMA, Base64, Shuffle, ZFPY, PCodec and the three VLen codecs) rebound `out` to a flattened view and returned that, and numcodecs.compat.ndarray_copy did the same for any `out` that was not an ndarray, which every filter and checksum codec inherited. The decoded bytes were already in the caller's buffer; only the returned object changes. JenkinsLookup3.decode now accepts any contiguous writable buffer as `out` and checks its size, and Fletcher32.decode returns a uint8 ndarray instead of a Cython memoryview slice that no type annotation could name. The strict xfails in tests/test_codec_contract.py are removed. Assisted-by: ClaudeCode:claude-opus-5-5
This was referenced Nov 21, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #907 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 27 27
Lines 921 922 +1
=========================================
+ Hits 921 922 +1
🚀 New features to boost your workflow:
|
# Conflicts: # tests/test_codec_contract.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
decode(buf, out=out)now returnsoutitself for every codec, as theCodecdocstring describes. Zlib, BZ2, LZMA, Base64, Shuffle, ZFPY, PCodec, the three VLen codecs, and every codec built onnumcodecs.compat.ndarray_copyreturned a flattened view ofoutinstead, or a new ndarray whenoutwas abytearray. The decoded bytes were already written intoout; only the returned object changes.Summary
ndarray_copyand the ten codecs listed above reboundoutto a flattened view and returned that view. They now return the caller's object. When the agent probed every codec, the bytes always landed inout; only the identity of the return value was wrong.decodeused to callout.view(...), so only an ndarray worked asout. It now accepts any contiguous writable buffer and raisesValueErrorwhenouthas the wrong size.decodewithoutoutreturns auint8ndarray, like the other checksum codecs. It used to return a Cython_memoryviewslice, which is not amemoryviewsubclass, so no annotation could name it.tests/test_codec_contract.pyare removed, and there is a new test for the JenkinsLookup3 size check.Overlap: #891 also validates JenkinsLookup3's
out. Whichever lands second needs a small rebase.Tests
Full suite passes locally (982 passed); the contract tests pass with no xfails.
Stack and merge order
The agent split honest type annotations for numcodecs into a stack of 10 PRs, following an audit of the existing annotations and of the earlier attempts (#443, #698, #701, #702). Each PR is opened from the fork against
main, so until the PRs before it merge, its diff also contains their commits. The change specific to this PR starts at commit 145994e; later commits on its branch are follow-up fixes and merges of the PR below it. Merge in this order:typing/01-harness: test: check the Codec contract for every codec and add static type teststyping/02-out-contract: fix: return the caller's out from decode(buf, out=out) in every codec ← this PRtyping/03-fix-annotations: fix(typing): correct registry, compat and NDArrayLike annotations and add alltyping/04-base-class: feat(abc): annotate the Codec base class and add structural codec protocolstyping/05-cython-stubs: feat(typing): ship .pyi stubs for the Cython extension modulestyping/06a-compressors: feat(typing): annotate the compressor codecs with per-codec config TypedDictstyping/06b-checksums: feat(typing): annotate the checksum codecs and make JenkinsLookup3 configs JSONtyping/06c-filters: feat(typing): annotate the filter codecstyping/06d-object-codecs: feat(typing): annotate the object and array-to-bytes codecstyping/07-py-typed: feat(typing): ship py.typed and check numcodecs with strict mypy🤖 Generated with Claude Code