Skip to content

test: check the Codec contract for every codec and add static type tests - #906

Merged
d-v-b merged 1 commit into
zarr-developers:mainfrom
d-v-b:typing/01-harness
Nov 21, 2026
Merged

d-v-b merged 1 commit into
zarr-developers:mainfrom
d-v-b:typing/01-harness

Conversation

@d-v-b

@d-v-b d-v-b commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

Check the behavior every codec promises through numcodecs.abc.Codec in a new tests/test_codec_contract.py, and add strict static type tests in tests/typing, run by mypy and pyright in a new typing CI job.

Summary

This PR changes no library code. It adds the checks that the rest of the stack relies on, so that each later annotation is verified against what the code actually does.

  • tests/test_codec_contract.py runs every registered codec through the behavior the base class promises:

    • encode and decode return buffers;
    • decode(buf, out=out) returns the caller's out, for both an ndarray and a bytearray;
    • get_config() round-trips through JSON, get_codec and from_config.

    Known violations are listed as xfail(strict=True), so a fix fails the test until its entry is removed. The next PR removes all of them. A guard test fails when a newly registered codec is missing from the case table.

  • tests/typing/ pairs typing.assert_type (checked by mypy and pyright, both strict) with runtime assertions (checked by pytest). An annotation that the checkers accept but the runtime contradicts fails one of the two. Both checkers read numcodecs from src/.

  • The new typing CI job runs the repository's own mypy configuration over src and tests, plus both checkers over tests/typing.

  • mypy and pyright are pinned in a new typing dependency group.

Tests

Locally on Python 3.13: the full pytest suite passes (83 contract tests pass and 26 are strict xfails). mypy (repository config), mypy strict (tests/typing) and pyright strict (tests/typing) are all clean.

Stack and merge order

The agent split honest type annotations for numcodecs into a stack of 10 PRs, following an audit of the existing annotations and of the earlier attempts (#443, #698, #701, #702). Each PR is opened from the fork against main, so until the PRs before it merge, its diff also contains their commits. The change specific to this PR starts at commit ff36c01; later commits on its branch are follow-up fixes and merges of the PR below it. Merge in this order:

  1. test: check the Codec contract for every codec and add static type tests #906 typing/01-harness: test: check the Codec contract for every codec and add static type tests ← this PR
  2. fix: return the caller's out from decode(buf, out=out) in every codec #907 typing/02-out-contract: fix: return the caller's out from decode(buf, out=out) in every codec
  3. fix(typing): correct registry, compat and NDArrayLike annotations and add __all__ #908 typing/03-fix-annotations: fix(typing): correct registry, compat and NDArrayLike annotations and add all
  4. feat(abc): annotate the Codec base class and add structural codec protocols #909 typing/04-base-class: feat(abc): annotate the Codec base class and add structural codec protocols
  5. feat(typing): ship .pyi stubs for the Cython extension modules #910 typing/05-cython-stubs: feat(typing): ship .pyi stubs for the Cython extension modules
  6. feat(typing): annotate the compressor codecs with per-codec config TypedDicts #911 typing/06a-compressors: feat(typing): annotate the compressor codecs with per-codec config TypedDicts
  7. feat(typing): annotate the checksum codecs and make JenkinsLookup3 configs JSON #912 typing/06b-checksums: feat(typing): annotate the checksum codecs and make JenkinsLookup3 configs JSON
  8. feat(typing): annotate the filter codecs #913 typing/06c-filters: feat(typing): annotate the filter codecs
  9. feat(typing): annotate the object and array-to-bytes codecs #914 typing/06d-object-codecs: feat(typing): annotate the object and array-to-bytes codecs
  10. feat(typing): ship py.typed and check numcodecs with strict mypy #915 typing/07-py-typed: feat(typing): ship py.typed and check numcodecs with strict mypy

🤖 Generated with Claude Code

Add tests/test_codec_contract.py, which runs every registered codec through
the behavior the Codec base class promises: encode and decode return buffers,
decode(out=...) returns the caller's out, and get_config round-trips through
from_config. Known violations are strict xfails, so fixing one forces its
entry to be removed.

Add tests/typing, where assert_type checks (verified by mypy and pyright in
strict mode) sit next to runtime assertions (verified by pytest), and a
`typing` CI job that runs both checkers plus the repository's own mypy
configuration. The type checkers are pinned in a new `typing` dependency
group.

Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b
d-v-b merged commit 28983da into zarr-developers:main Nov 21, 2026
36 checks passed
d-v-b added a commit to d-v-b/numcodecs that referenced this pull request Nov 21, 2026
Resolve the conflict in src/numcodecs/pickles.py by keeping the typed
signatures from zarr-developers#915 and the deprecation warning in Pickle.__init__.

Two tests added upstream construct the Pickle codec and now see the
deprecation warning: test_config_round_trip[pickle] (zarr-developers#906), which
rebuilds the codec from its config, and the static type test
test_pickle (zarr-developers#915). Both get a filterwarnings mark scoped to the
deprecation message.

Assisted-by: ClaudeCode:claude-opus-5.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant