fix: byte-order handling for structured dtypes in the bytes codec

[d-v-b]

🤖 AI text below 🤖

Summary

Fixes zarr-developers#4141: the bytes codec silently corrupted structured-dtype data whose field byte order differed from the stored one.

Two defects, both in BytesCodec:

• Encode decided whether to byte-swap by comparing self.endian against the buffer's top-level byteorder, but numpy reports '|' for void dtypes, so struct fields were never swapped — big-endian input was written raw under endian: little metadata, corrupting even a pure zarr write/read roundtrip.
• Decode only honored endian for scalar (HasEndianness) dtypes and viewed struct chunks in the fields' native byte order — the silent-corruption path for virtual references to external big-endian data (e.g. FITS binary tables via VirtualiZarr).

The fix keeps stored and in-memory byte order independent, per the struct data type spec (field data types carry no byte order; the codec's endian governs all multi-byte fields):

• Encode detects mismatches by comparing full dtypes via newbyteorder(self.endian), converting to the stored order when they differ.
• Decode reinterprets raw bytes in the stored order, then converts to the data type's declared byte order when they differ. This also applies to the scalar path, which previously returned a storage-order view and relied on the downstream out-buffer assignment to cast.

Note: v3 metadata carries no per-field byte order, so a reopened array always declares native-endian fields and reads now return correctly byte-swapped native values. Per-field endianness of the dtype declaration still doesn't survive a metadata roundtrip — that would be a metadata/API feature, distinct from this corruption fix.

For reviewers

The decode-side contract change (decoded buffers now always come back in the declared dtype's byte order, converting when needed) is the part worth a second look — it slightly changes what _decode_sync returns for non-native scalar dtypes, though the user-visible result was already cast by the out-buffer assignment. Encode-side and the tests are straightforward.

Author attestation

• I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions (n/a — no user-facing API change)
• New/modified features documented in docs/user-guide/*.md (n/a)
• Changes documented as a new file in changes/
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

🤖 Generated with Claude Code