Skip to content

FIX Preserve NumPy masked-array fill values - #563

Open
ananya-kaul wants to merge 1 commit into
skops-dev:mainfrom
ananya-kaul:ananya/fix-masked-array-fill-value
Open

ananya-kaul wants to merge 1 commit into
skops-dev:mainfrom
ananya-kaul:ananya/fix-masked-array-fill-value

Conversation

@ananya-kaul

Copy link
Copy Markdown

FIX Preserve NumPy masked-array fill values

Saving and loading a masked array currently discards its custom fill value. For example, an integer array with fill_value=-99 uses 999999 after loading, changing the result of filled().

Persist the fill value using the existing NumPy node and restore it when constructing the masked array. Older files without this field retain their default behavior. The new child is included in the type audit and restricted to NdArrayNode.

Validation: 12 round-trip cases fail before the fix and pass afterward, covering integer, floating-point and string data, masked/unmasked arrays, and file/bytes APIs. A compatibility test covers files without the new field, and an audit regression rejects an unexpected child type. Persistence, legacy persistence and audit suites: 1083 passed, 1 skipped, 1 xpassed. Ruff lint/format and focused pyrefly check pass. Tested on macOS with Python 3.13.5; the full project suite and other platforms were not run.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The fill-value node restriction rejects supported object-dtype masked arrays with Python string fills, breaking round-trip loading and untrusted-type inspection.

1 open issue
What changed in this PR

This PR updates NumPy masked-array persistence to preserve custom fill values, while retaining default behavior for older files without a saved fill value.

Changes:

  • Save and restore fill values and include them in type auditing.
  • Add round-trip, backward-compatibility, and unexpected-child-type tests.
  • Document the fix in the changelog.
File Description
skops/​io/​tests/​test_masked_array.py Adds fill-value round-trip and backward-compatibility tests.
skops/​io/​tests/​test_audit.py Adds rejection coverage for unexpected fill-value node types.
skops/​io/​_numpy.py Serializes, audits, and restores masked-array fill values.
docs/​changes.rst Records the persistence fix.

Review effort: Balanced

Comment thread skops/io/_numpy.py
state["content"]["fill_value"],
load_context,
trusted=trusted,
allowed_types=(NdArrayNode,),

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants