Repository navigation
Conversation
Add tests/test_codec_contract.py, which runs every registered codec through the behavior the Codec base class promises: encode and decode return buffers, decode(out=...) returns the caller's out, and get_config round-trips through from_config. Known violations are strict xfails, so fixing one forces its entry to be removed. Add tests/typing, where assert_type checks (verified by mypy and pyright in strict mode) sit next to runtime assertions (verified by pytest), and a `typing` CI job that runs both checkers plus the repository's own mypy configuration. The type checkers are pinned in a new `typing` dependency group. Assisted-by: ClaudeCode:claude-opus-5-5
The Codec docstring says decode writes into `out`, and callers expect `out` back. Ten codecs (Zlib, BZ2, LZMA, Base64, Shuffle, ZFPY, PCodec and the three VLen codecs) rebound `out` to a flattened view and returned that, and numcodecs.compat.ndarray_copy did the same for any `out` that was not an ndarray, which every filter and checksum codec inherited. The decoded bytes were already in the caller's buffer; only the returned object changes. JenkinsLookup3.decode now accepts any contiguous writable buffer as `out` and checks its size, and Fletcher32.decode returns a uint8 ndarray instead of a Cython memoryview slice that no type annotation could name. The strict xfails in tests/test_codec_contract.py are removed. Assisted-by: ClaudeCode:claude-opus-5-5
… add __all__ Several annotations described something other than what the code does: - codec_registry holds classes: dict[str, type[Codec]], not dict[str, Codec]. - entries holds single entry points: dict[str, EntryPoint]. - ndarray_copy returned NDArrayLike, but hands back `src` unchanged (often bytes) when `dst` is None. It is now overloaded on `dst`. - NDArrayLike, DType and FlagsObj declared settable attributes, so no numpy.ndarray satisfied NDArrayLike under mypy or pyright. They now declare read-only properties. The isinstance cache moves to module level, because a protocol without annotations of its own would otherwise pick up the metaclass's annotation as a required member. The compat functions annotate their parameters with a new BufferLike alias (collections.abc.Buffer | NDArrayLike), is_ndarray_like returns TypeIs, and get_codec/register_codec are annotated. register_codec now refuses a class without a codec_id instead of registering it under None, and get_codec warns (DeprecationWarning) when an entry point loads a class that does not subclass Codec, so that `-> Codec` can become true. numcodecs/__init__.py defines __all__, so that type checkers treat the re-exported codec classes as public once numcodecs ships py.typed. mypy now checks against Python 3.12, the oldest supported version; doing so caught `typing.TypeIs`, which only exists from 3.13 and now comes from typing_extensions. Assisted-by: ClaudeCode:claude-opus-5-5
…tocols Codec.encode and Codec.decode were unannotated with docstring-only bodies, so mypy saw them as Any and pyright inferred that both return None, which made every typed subclass an "incompatible override" under pyright. - encode/decode take and return BufferLike (collections.abc.Buffer | NDArrayLike), with `buf` positional-only so that subclasses may rename it. - decode is overloaded so that decode(buf, out=out) returns the type of `out`, which every codec now honors. - codec_id is declared `str` without a default instead of `str | None = None`. It is not a ClassVar: with ClassVar, Codec instances do not match protocols that declare codec_id as a plain attribute (zarr-python's Numcodec) or as a property, under mypy or pyright. - get_config returns Mapping[str, Any], so that concrete codecs can later return per-codec TypedDicts, and from_config returns Self. numcodecs.abc also exports CodecProtocol and the three Zarr v3 codec kinds (BytesBytesCodec, ArrayArrayCodec, ArrayBytesCodec) as structural protocols, so consumers can type code that accepts any codec-like object. Assisted-by: ClaudeCode:claude-opus-5-5
blosc, zstd, lz4, vlen, fletcher32, jenkins, compat_ext and _shuffle had no stubs, so type checkers saw Blosc, Zstd, LZ4, Fletcher32 and the VLen codecs as unknown, and numcodecs' own mypy run silently treated them as Any. Each module now has a stub next to its .pyx, installed by meson. The stubs follow probed behavior: the compressors return bytes, or `out` itself when it is given; Fletcher32.decode returns a uint8 ndarray; the VLen codecs encode to bytearray and decode to object arrays. VLenArray.encode takes an object array only, since numpy turns a list of equal-length arrays into a 2-D array. The typing CI job runs mypy.stubtest against the installed extensions, with an allowlist for Cython artifacts (__test__, __pyx_capi__, and the `(self, /)` signature Cython reports for argument-less module functions). Checksum32.checksum is declared positional-only, so that JenkinsLookup3 can use jenkins_lookup3 (whose parameters are named differently) as its checksum; it is wrapped in staticmethod so type checkers do not bind it as a method. It was never bound at runtime, being a builtin function. Assisted-by: ClaudeCode:claude-opus-5-5
…pedDicts Zlib, GZip, BZ2, LZMA and Base64 annotate __init__, encode (-> bytes) and decode (overloaded: bytes without `out`, the type of `out` with it), and Blosc, Zstd and LZ4 get the same through their stubs. Each get_config returns a TypedDict of the codec's parameters; for codecs that inherit the base implementation it is a cast of the inherited dict, so subclasses that add attributes keep them in their configuration as before. numcodecs.abc.CodecConfig is the common base: `id: str`, with PEP 728 `extra_items=ReadOnly[JSON]`, which lets pyright treat any codec configuration as a Mapping[str, JSON] (and as Zarr v2 codec metadata). mypy does not implement PEP 728 yet and sees a plain TypedDict. JSON uses Sequence/Mapping, matching zarr_metadata's JSONValue. `extra_items` needs typing_extensions >= 4.13 at runtime. The Cython modules define their config TypedDicts with the functional syntax, which Cython compiles; the stubs declare the same types with the class syntax. Assisted-by: ClaudeCode:claude-opus-5-5
…nfigs JSON CRC32, Adler32, CRC32C, JenkinsLookup3 and Fletcher32 are annotated, and their configurations are typed: Checksum32Config has an optional `location` (present only when passed to the constructor, which matches the Zarr v3 crc32c codec having no configuration), and JenkinsLookup3Config adds `initval` and `prefix`. Typing the JenkinsLookup3 configuration exposed a bug: with a prefix, get_config() held a numpy array, so the configuration could neither be serialized to JSON nor passed back to get_codec. It now reports the prefix as a list of ints, which the constructor accepts. Checksum32.encode and decode are annotated with the union of what the family returns (uint8 ndarrays for CRC32/Adler32/CRC32C; bytes and a memoryview for JenkinsLookup3, which overrides both). Changing JenkinsLookup3's return types instead would break callers that compare its output with bytes, as its own tests do. Checksum32.decode checks the byte length of its input instead of len(buf), which counted only the first axis of a multidimensional array. This overlaps upstream PR zarr-developers#891. Assisted-by: ClaudeCode:claude-opus-5-5
AsType, Delta, Quantize, FixedScaleOffset, BitRound, PackBits, Shuffle and Categorize annotate their constructors, encode/decode (with the `out` overloads) and get_config, each returning its own CodecConfig subclass. - The numeric filters return ndarrays of the codec's dtypes, which numpy's types cannot express from a runtime dtype, so they are npt.NDArray[Any]. - BitRound returns NDArrayLike: it works on ndarray-likes such as cupy arrays and may return its input unchanged. - PackBits encodes to uint8 and decodes to bool; Shuffle works on uint8 bytes, and its encode also takes `out`. - FixedScaleOffset's `offset` and `scale` are `float`, which covers the ints its configuration echoes back when ints were passed. - Categorize accepts a plain iterable of strings when its dtype is object. Every filter satisfies numcodecs.abc.ArrayArrayCodec. Assisted-by: ClaudeCode:claude-opus-5-5
This was referenced Nov 21, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #913 +/- ##
==========================================
Coverage 100.00% 100.00%
==========================================
Files 27 27
Lines 922 1120 +198
==========================================
+ Hits 922 1120 +198
🚀 New features to boost your workflow:
|
With .pyi stubs next to the .pyx sources, ruff resolves numcodecs.blosc, numcodecs.zstd, numcodecs.fletcher32 and numcodecs.jenkins as first-party modules and wants them grouped with the other numcodecs imports. Assisted-by: ClaudeCode:claude-opus-5-5
…dDicts The Cython modules create their config TypedDicts at runtime with typing.TypedDict, which adds __mutable_keys__ and __readonly_keys__; stubtest does not expect them on the stubs' class-syntax TypedDicts. The allowlist entry was only added in the py.typed change, but it is needed from the commit that introduced the TypedDicts. Assisted-by: ClaudeCode:claude-opus-5-5
# Conflicts: # tests/test_jenkins.py
# Conflicts: # tests/test_codec_contract.py
…ations # Conflicts: # docs/release.md # src/numcodecs/compat.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Annotate AsType, Delta, Quantize, FixedScaleOffset, BitRound, PackBits, Shuffle and Categorize, each with a configuration TypedDict. Every filter satisfies
numcodecs.abc.ArrayArrayCodec.Summary
npt.NDArray[Any].NDArrayLike: it works on ndarray-likes such as cupy arrays, and returns its input unchanged whenkeepbitsis the maximum.uint8and decodes tobool.uint8bytes, and itsencodealso takesout.offsetandscaleare typedfloat, which also covers the ints that its configuration echoes back when ints were passed.object.Tests
Full suite passes locally;
tests/typing/test_filter_types.py.Stack and merge order
The agent split honest type annotations for numcodecs into a stack of 10 PRs, following an audit of the existing annotations and of the earlier attempts (#443, #698, #701, #702). Each PR is opened from the fork against
main, so until the PRs before it merge, its diff also contains their commits. The change specific to this PR starts at commit d8a20b3; later commits on its branch are follow-up fixes and merges of the PR below it. Merge in this order:typing/01-harness: test: check the Codec contract for every codec and add static type teststyping/02-out-contract: fix: return the caller's out from decode(buf, out=out) in every codectyping/03-fix-annotations: fix(typing): correct registry, compat and NDArrayLike annotations and add alltyping/04-base-class: feat(abc): annotate the Codec base class and add structural codec protocolstyping/05-cython-stubs: feat(typing): ship .pyi stubs for the Cython extension modulestyping/06a-compressors: feat(typing): annotate the compressor codecs with per-codec config TypedDictstyping/06b-checksums: feat(typing): annotate the checksum codecs and make JenkinsLookup3 configs JSONtyping/06c-filters: feat(typing): annotate the filter codecs ← this PRtyping/06d-object-codecs: feat(typing): annotate the object and array-to-bytes codecstyping/07-py-typed: feat(typing): ship py.typed and check numcodecs with strict mypy🤖 Generated with Claude Code