Skip to content

fix: handle numpy arrays nested in dicts/lists for auto serializer#52

Closed
27Bslash6 wants to merge 2 commits intomainfrom
fix/50-nested-numpy-serialization
Closed

fix: handle numpy arrays nested in dicts/lists for auto serializer#52
27Bslash6 wants to merge 2 commits intomainfrom
fix/50-nested-numpy-serialization

Conversation

@27Bslash6
Copy link
Contributor

Summary

  • serializer="auto" only handled top-level ndarray; nested arrays in dicts/lists hit _auto_default() which had no numpy handler, raising TypeError
  • Added __ndarray__ marker to _auto_default() and corresponding deserialization in _auto_object_hook() for roundtrip support
  • Added 19 unit tests covering nested 1D/2D/3D arrays, mixed types, deeply nested structures, edge cases (empty, 0-d scalar), integrity checking, and corruption detection

Test plan

  • Dict containing 1D, 2D, float64, and multiple numpy arrays
  • Lists containing numpy arrays
  • Deeply nested structures (dict → list → dict → array)
  • Mixed types (numpy + datetime + UUID + set in same structure)
  • Edge cases: empty array, 0-d scalar, 3D tensor
  • With and without ByteStorage integrity checking
  • Malformed __ndarray__ markers raise SerializationError
  • All pre-commit hooks pass (ruff lint, ruff format, basedpyright)

Closes #50

@codecov
Copy link

codecov bot commented Feb 22, 2026

Codecov Report

❌ Patch coverage is 0% with 8 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/cachekit/serializers/auto_serializer.py 0.00% 7 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

- bytes 1.10.1 → 1.11.1 (RUSTSEC-2026-0007: integer overflow in BytesMut::reserve)
- filelock 3.20.1 → 3.24.3 (GHSA-qmgc-5h2g-mvrw: TOCTOU symlink vuln)
- orjson 3.11.4 → 3.11.7 (GHSA-hx9q-6w63-j58v: unbounded recursion)
- pip 25.3 → 26.0.1 (GHSA-6vgw-5pg2-w6jp: path traversal in wheel extraction)
- test_ttl_enforced: increase TTL 1s→3s to prevent CI timing flakiness
_auto_default() had no handler for ndarray, so msgpack.packb() raised
TypeError when traversing container types with nested numpy arrays.

Added __ndarray__ marker to _auto_default/_auto_object_hook for
roundtrip serialization of nested arrays via the msgpack fallback path.

Closes #50
@27Bslash6 27Bslash6 force-pushed the fix/50-nested-numpy-serialization branch from 5d36d78 to 6a8c744 Compare February 22, 2026 02:44
@27Bslash6
Copy link
Contributor Author

Superseded by combined fix PR

@27Bslash6 27Bslash6 closed this Feb 22, 2026
@27Bslash6 27Bslash6 deleted the fix/50-nested-numpy-serialization branch February 22, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serializer="auto" fails on numpy arrays nested in dicts

1 participant