From 8da13336dfa60c2b4412097bc5e58488aebbb66c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 31 May 2026 19:28:07 +0200 Subject: [PATCH 01/11] chore(deps): bump the actions group across 1 directory with 8 updates (#176) Bumps the actions group with 8 updates in the / directory: | Package | From | To | | --- | --- | --- | | [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` | | [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` | | [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` | | [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` | | [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` | | [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` | | [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` | Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6 - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](https://github.com/prefix-dev/setup-pixi/compare/1b2de7f3351f171c8b4dfeb558c639cb58ed4ec0...5185adfbffb4bd703da3010310260805d89ebb11) Updates `codecov/codecov-action` from 6.0.0 to 6.0.1 - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/57e3a136b779b570ffcdbf80b3bdc90e7fab3de2...e79a6962e0d4c0c17b229090214935d2e33f8354) Updates `github/issue-metrics` from 4.2.2 to 4.2.7 - [Release notes](https://github.com/github/issue-metrics/releases) - [Commits](https://github.com/github/issue-metrics/compare/c9e9838147fd355dace335ba787f01b6641a400a...1e38d5e62363e14db8019ed7d106b9855bdba6cc) Updates `j178/prek-action` from 2.0.3 to 2.0.4 - [Release notes](https://github.com/j178/prek-action/releases) - [Commits](https://github.com/j178/prek-action/compare/6ad80277337ad479fe43bd70701c3f7f8aa74db3...bdca6f102f98e2b4c7029491a53dfd366469e33d) Updates `actions/upload-artifact` from 7.0.0 to 7.0.1 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v7...043fb46d1a93c77aae656e7c1c64a875d1fc6a0a) Updates `actions/download-artifact` from 7.0.0 to 8.0.1 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](https://github.com/actions/download-artifact/compare/v7...3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c) Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0 - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](https://github.com/pypa/gh-action-pypi-publish/compare/v1.13.0...cef221092ed1bacb1cc03d23a2d87d1d172e277b) Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6 - [Release notes](https://github.com/zizmorcore/zizmor-action/releases) - [Commits](https://github.com/zizmorcore/zizmor-action/compare/b1d7e1fb5de872772f31590499237e7cce841e8e...5f14fd08f7cf1cb1609c1e344975f152c7ee938d) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.9.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: codecov/codecov-action dependency-version: 6.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: github/issue-metrics dependency-version: 4.2.7 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: j178/prek-action dependency-version: 2.0.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/download-artifact dependency-version: 8.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.14.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions - dependency-name: zizmorcore/zizmor-action dependency-version: 0.5.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/downstream.yml | 2 +- .github/workflows/gpu_test.yml | 2 +- .github/workflows/hypothesis.yaml | 2 +- .github/workflows/issue-metrics.yml | 2 +- .github/workflows/lint.yml | 2 +- .github/workflows/test.yml | 4 ++-- .github/workflows/zarr-metadata-release.yml | 12 ++++++------ .github/workflows/zizmor.yml | 2 +- 8 files changed, 14 insertions(+), 14 deletions(-) diff --git a/.github/workflows/downstream.yml b/.github/workflows/downstream.yml index 74026233c4..3eb6898895 100644 --- a/.github/workflows/downstream.yml +++ b/.github/workflows/downstream.yml @@ -34,7 +34,7 @@ jobs: persist-credentials: false - name: Set up pixi - uses: prefix-dev/setup-pixi@1b2de7f3351f171c8b4dfeb558c639cb58ed4ec0 # v0.9.5 + uses: prefix-dev/setup-pixi@5185adfbffb4bd703da3010310260805d89ebb11 # v0.9.6 with: manifest-path: xarray/pixi.toml diff --git a/.github/workflows/gpu_test.yml b/.github/workflows/gpu_test.yml index 403441b306..333769cb9e 100644 --- a/.github/workflows/gpu_test.yml +++ b/.github/workflows/gpu_test.yml @@ -76,7 +76,7 @@ jobs: hatch env run --env "$HATCH_ENV" run-coverage - name: Upload coverage - uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0 + uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1 with: token: ${{ secrets.CODECOV_TOKEN }} flags: gpu diff --git a/.github/workflows/hypothesis.yaml b/.github/workflows/hypothesis.yaml index 4f9467be7d..a456b2aa0a 100644 --- a/.github/workflows/hypothesis.yaml +++ b/.github/workflows/hypothesis.yaml @@ -93,7 +93,7 @@ jobs: key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }} - name: Upload coverage - uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0 + uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1 with: token: ${{ secrets.CODECOV_TOKEN }} flags: tests diff --git a/.github/workflows/issue-metrics.yml b/.github/workflows/issue-metrics.yml index 14fba5b9ec..510849ef3e 100644 --- a/.github/workflows/issue-metrics.yml +++ b/.github/workflows/issue-metrics.yml @@ -33,7 +33,7 @@ jobs: echo "last_month=$first_day..$last_day" >> "$GITHUB_ENV" - name: Run issue-metrics tool - uses: github/issue-metrics@c9e9838147fd355dace335ba787f01b6641a400a # v4.2.2 + uses: github/issue-metrics@1e38d5e62363e14db8019ed7d106b9855bdba6cc # v4.2.7 env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} SEARCH_QUERY: 'repo:zarr-developers/zarr-python is:issue created:${{ env.last_month }} -reason:"not planned"' diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 768e660ec2..fec211b4dd 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -30,4 +30,4 @@ jobs: uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 with: enable-cache: true - - uses: j178/prek-action@6ad80277337ad479fe43bd70701c3f7f8aa74db3 # v2.0.3 + - uses: j178/prek-action@bdca6f102f98e2b4c7029491a53dfd366469e33d # v2.0.4 diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 03143d3e5b..62e571856b 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -78,7 +78,7 @@ jobs: hatch env run --env "$HATCH_ENV" run-coverage - name: Upload coverage if: ${{ matrix.dependency-set == 'optional' && matrix.os == 'ubuntu-latest' }} - uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0 + uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1 with: token: ${{ secrets.CODECOV_TOKEN }} flags: tests @@ -125,7 +125,7 @@ jobs: run: | hatch env run --env "$HATCH_ENV" run-coverage - name: Upload coverage - uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0 + uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1 with: token: ${{ secrets.CODECOV_TOKEN }} flags: tests diff --git a/.github/workflows/zarr-metadata-release.yml b/.github/workflows/zarr-metadata-release.yml index 809d502f16..9639fcfdd3 100644 --- a/.github/workflows/zarr-metadata-release.yml +++ b/.github/workflows/zarr-metadata-release.yml @@ -35,7 +35,7 @@ jobs: - name: Build run: hatch build - - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: zarr-metadata-dist path: packages/zarr-metadata/dist @@ -45,7 +45,7 @@ jobs: needs: [build] runs-on: ubuntu-latest steps: - - uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: name: zarr-metadata-dist path: dist @@ -76,7 +76,7 @@ jobs: id-token: write # required for OIDC trusted publishing attestations: write # required for artifact attestations steps: - - uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: name: zarr-metadata-dist path: dist @@ -87,7 +87,7 @@ jobs: subject-path: dist/* - name: Publish package to PyPI - uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0 + uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0 upload_testpypi: name: Upload to TestPyPI @@ -101,7 +101,7 @@ jobs: id-token: write attestations: write steps: - - uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: name: zarr-metadata-dist path: dist @@ -112,6 +112,6 @@ jobs: subject-path: dist/* - name: Publish package to TestPyPI - uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0 + uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0 with: repository-url: https://test.pypi.org/legacy/ diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index da19f22421..7ac4fe5d0e 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -32,4 +32,4 @@ jobs: persist-credentials: false - name: Run zizmor - uses: zizmorcore/zizmor-action@b1d7e1fb5de872772f31590499237e7cce841e8e # v0.5.3 + uses: zizmorcore/zizmor-action@5f14fd08f7cf1cb1609c1e344975f152c7ee938d # v0.5.6 From 6ab5ffc8e0481df2c7b800d7ae03c043d9ae2033 Mon Sep 17 00:00:00 2001 From: Max Jones <14077947+maxrjones@users.noreply.github.com> Date: Fri, 5 Jun 2026 13:53:17 -0400 Subject: [PATCH 02/11] docs: add AGENTS.md for AI-assisted development --- AGENTS.md | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..527c78007a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,103 @@ +# AGENTS.md + +Guidance for AI coding agents working in this repository. Human contributors should also read `docs/contributing.md`, especially the **AI-assisted contributions** section: the human submitting the PR must understand and being able to explain every change, and PR descriptions / review responses must be in the human's own words. Keep diffs small and reviewable. + +## Project overview + +`zarr-python` (PyPI package `zarr`) implements chunked, compressed, N-dimensional arrays for Python. This is the 3.x line, which reads and writes both Zarr format v2 and v3 data. Requires Python >= 3.12. The public API is re-exported from `src/zarr/__init__.py` (`Array`, `Group`, `create_array`, `open`, etc.). + +## Environment and common commands + +Development uses **hatch** (with `uv` as the installer) for managed environments, and `uv` directly for ad-hoc commands. The canonical test environments are named `test.py3.{12,13,14}-{minimal,optional}`; `optional` pulls in remote stores (fsspec, obstore, s3fs), the CLI, and universal-pathlib. + +```bash +# Run the full test suite in a managed env (benchmarks excluded by default) +hatch env run --env test.py3.12-optional run + +# Run with coverage (XML report); coverage must reach 100% for CI to pass +hatch env run --env test.py3.12-optional run-coverage + +# Ad-hoc test runs with uv (faster iteration than spinning up a hatch env). +# Prefer uv run pytest for narrow runs; reach for hatch envs for full/coverage runs. +uv run pytest tests/test_array.py # one test file +uv run pytest tests/test_array.py::test_name # one test function +uv run pytest tests/test_array.py -k "expr" # tests matching a -k expression +uv run pytest "tests/test_array.py::test_name[param-id]" # one parametrized case +uv run pytest tests/test_array.py -x --lf # stop on first failure, rerun last-failed + +# Run tests in parallel across CPUs with pytest-xdist (-n). Big speedup for the +# full suite; for a handful of tests the worker startup cost usually isn't worth it. +uv run pytest -n auto tests/ # auto = one worker per core +uv run pytest -n 4 tests/test_codecs/ # fixed worker count + +# Type-check (strict mypy over src + tests) — this is what CI's Lint job runs +uv run --frozen mypy + +# Hypothesis property tests (slow; opt in) +hatch env run --env test.py3.12-optional run-hypothesis + +# Docs: live-reloading server / strict build +hatch --env docs run serve +hatch --env docs run check +``` + +Note: `pytest` `testpaths` include `src`, `tests`, and `docs/user-guide`, and `--doctest-modules` is on — docstrings in `src/` are executed as tests, and the user-guide markdown is doctested. `xfail_strict = true` and `filterwarnings = ["error", ...]` mean an unexpected pass or an unfiltered warning fails the suite. + +## Linting and pre-commit + +The project uses [`prek`](https://github.com/j178/prek) (a drop-in pre-commit runner) against `.pre-commit-config.yaml`. Ruff (lint + format, line length 100) and `mypy` run here, plus codespell, numpydoc validation, towncrier-check, and zizmor. + +```bash +prek run --all-files # all hooks +prek run --last-commit # only files changed in the last commit +``` + +Two local rules worth knowing because they will reject otherwise-valid code: +- **No `.lstrip("...")` / `.rstrip("...")` with multi-char string args** (`ban-lstrip-rstrip`) — these are character-set operations and almost always a bug where `removeprefix`/`removesuffix` was intended. +- **numpydoc validation** is enforced on docstrings (Parameters/Returns sections must match signatures). + +## Changelog (required for most PRs) + +Every user-facing change needs a news fragment in `changes/` named `{issue-or-pr-number}.{type}.md`, where type is one of `feature`, `bugfix`, `doc`, `removal`, `misc`. Generated into `docs/release-notes.md` by towncrier at release time. `towncrier create` scaffolds one. The `towncrier-check` pre-commit hook will flag PRs missing a fragment. + +## Architecture + +### Async core with a generated sync facade + +Every storage and array operation is fundamentally **async**. The two-layer pattern is pervasive and important to preserve: + +- `AsyncArray` / `AsyncGroup` (`core/array.py`, `core/group.py`) hold the real async implementations. +- `Array` / `Group` are thin synchronous wrappers that drive the async methods through `sync()` (`core/sync.py`), which runs coroutines on a single dedicated background event loop (a module-global loop in its own thread). Never call `asyncio.run` or create new loops in library code — route through `sync()`. +- The user-facing top-level functions split the same way: `api/asynchronous.py` (async) and `api/synchronous.py` (sync wrappers). `__init__.py` re-exports the synchronous ones. + +When adding functionality, implement it on the `Async*` class / in `api/asynchronous.py` first, then add the sync wrapper. + +### Store abstraction + +`abc/store.py` defines the abstract `Store` (async `get`/`set`/`delete`/`list`/`exists`, partial reads via `RangeByteRequest`/`OffsetByteRequest`/`SuffixByteRequest`). Concrete stores live in `storage/`: `_local.py`, `_memory.py`, `_zip.py`, `_fsspec.py` (any fsspec filesystem), `_obstore.py` (Rust `obstore` backend), plus `_wrapper.py` and `_logging.py` decorators. `StorePath` (`storage/_common.py`) couples a store with a key prefix and is what arrays/groups actually hold. + +### Codecs and the codec pipeline + +Reading/writing a chunk runs it through an ordered pipeline of codecs, typed by what they transform (`abc/codec.py`): `ArrayArrayCodec` → `ArrayBytesCodec` (exactly one, e.g. `bytes`) → `BytesBytesCodec` (e.g. compressors). `core/codec_pipeline.py` (`BatchedCodecPipeline`) orchestrates encode/decode and batches/concurrency-limits chunk I/O. Built-in codecs are in `codecs/` (`blosc`, `gzip`, `zstd`, `crc32c`, `transpose`, `sharding`, `vlen_utf8`, `scale_offset`, ...); `codecs/numcodecs/` and `codecs/_v2.py` bridge to numcodecs for v2 filters/compressors. **Sharding** (`codecs/sharding.py`) is itself an `ArrayBytesCodec` that packs a grid of sub-chunks into one store object — the most intricate codec. + +### Metadata and v2/v3 duality + +`AsyncArray` is generic over `ArrayV2Metadata | ArrayV3Metadata` (`core/metadata/`). Most of the codebase branches on format version through this metadata type rather than ad-hoc version flags. v2 uses `.zarray`/`.zattrs`/`.zgroup`; v3 uses `zarr.json`. `metadata/migrate_v3.py` and `core/metadata/` hold the conversion and parsing logic. When touching read/write paths, check both metadata classes. + +### dtype system + +`core/dtype/` is a pluggable dtype layer (not raw NumPy dtypes) that maps Zarr data types to/from NumPy and handles fill values, endianness, and v2-vs-v3 dtype naming. `dtype.py` at the package root re-exports it. + +### Registry and config + +`registry.py` is the extension mechanism: codecs, pipelines, buffers, ndbuffers, and chunk-key-encodings are registered by name and discovered via entry points (`zarr.codecs`, `zarr.codec_pipelines`, etc.), so third-party packages can plug in. `core/config.py` uses `donfig` (the `zarr.config` object) for runtime settings — default codecs, concurrency limits, the active buffer/pipeline implementation. Resolving "which class implements X" generally goes through config → registry. + +### Buffers and the GPU path + +`abc/buffer.py` + `core/buffer/` abstract `Buffer` (1-D bytes) and `NDBuffer` (N-D array) so the same code runs on CPU (`core/buffer/cpu.py`, NumPy) or GPU (`core/buffer/gpu.py`, CuPy). GPU-only tests are marked `@pytest.mark.gpu`. Don't hardcode `np.ndarray`; go through the buffer prototype. + +## Conventions + +- **Public API surface:** anything added to `__all__` in `__init__.py` / `api/synchronous.py` is public and must have a numpydoc docstring and an entry under `docs/api/*.md`. New user-facing behavior also belongs in `docs/user-guide/`. +- **Experimental features** go under `src/zarr/experimental/`, are documented in `docs/user-guide/experimental.md`, and carry no stability guarantees (may be removed in any release). The team aims to promote or remove them within ~6 months. +- **Versioning is EffVer, not SemVer** — breaking changes are possible in minor (and rarely patch) releases, judged by upgrade effort. Prefer backwards-compatible changes and deprecation warnings over hard breaks. From 18b7d30b06285df54b1b7f4a5473c05e4a940d3e Mon Sep 17 00:00:00 2001 From: Max Jones <14077947+maxrjones@users.noreply.github.com> Date: Fri, 5 Jun 2026 16:05:19 -0400 Subject: [PATCH 03/11] Update AGENTS.md Co-authored-by: Davis Bennett --- AGENTS.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 527c78007a..72437de94f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,7 +5,12 @@ Guidance for AI coding agents working in this repository. Human contributors sho ## Project overview `zarr-python` (PyPI package `zarr`) implements chunked, compressed, N-dimensional arrays for Python. This is the 3.x line, which reads and writes both Zarr format v2 and v3 data. Requires Python >= 3.12. The public API is re-exported from `src/zarr/__init__.py` (`Array`, `Group`, `create_array`, `open`, etc.). +## Related projects +`zarr-python` depends on the contents of the Zarr [v2](https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html) and Zarr [v3](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html) storage specifications. We are committed to compliance with the specs, and also consistency with other Zarr implementations, namely: +- [TensorStore](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html) (C++ / Python) +- [Zarrs](https://github.com/zarrs/zarrs) (Rust) +- [Zarrita](https://github.com/manzt/zarrita.js/) (Javascript) ## Environment and common commands Development uses **hatch** (with `uv` as the installer) for managed environments, and `uv` directly for ad-hoc commands. The canonical test environments are named `test.py3.{12,13,14}-{minimal,optional}`; `optional` pulls in remote stores (fsspec, obstore, s3fs), the CLI, and universal-pathlib. From 5b12eaf094e753351e2ae44615c045f115bcb9f4 Mon Sep 17 00:00:00 2001 From: Max Jones <14077947+maxrjones@users.noreply.github.com> Date: Wed, 1 Jul 2026 10:56:00 -0400 Subject: [PATCH 04/11] Gitignore agentic files --- .gitignore | 38 ++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 1 + 2 files changed, 39 insertions(+) create mode 100644 CLAUDE.md diff --git a/.gitignore b/.gitignore index 3284865d6c..db301c12f1 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,9 @@ htmlcov/ .cache coverage.xml *,cover +.pytest_cache/ +.ruff_cache/ +.mypy_cache/ # Translations *.mo @@ -94,3 +97,38 @@ zarr.egg-info/ # zarr-metadata package lockfile (a library, not an app) packages/zarr-metadata/uv.lock + +# ─── Agentic coding: local/personal state (provider-agnostic) ─── +# Commit shared instruction files (AGENTS.md, CLAUDE.md, .cursor/rules/) intentionally; +# ignore only per-user/local state, session data, and caches below. + +# Local overrides & personal instruction variants +CLAUDE.local.md +AGENTS.local.md +AGENTS.override.md +*.local.md + +# Agent tool state directories +.claude/ +.windsurf/ +.aider* +.continue/ +.codeium/ + +# Agent session transcripts, logs, scratch +.agent/ +.agent-workspace/ +.agent-scratch/ +agent-logs/ +*.session.json +*.transcript.* + +# MCP configs (frequently contain tokens/credentials) +.mcp.local.json +*.mcp.local.json + +# Agent caches / embeddings / vector stores +.agent-cache/ +.context-cache/ +.embeddings/ +.vectorstore/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000000..43c994c2d3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md From a0c0c24eed34453c0e54c30559013e1575eb8022 Mon Sep 17 00:00:00 2001 From: Max Jones <14077947+maxrjones@users.noreply.github.com> Date: Wed, 1 Jul 2026 11:41:53 -0400 Subject: [PATCH 05/11] chore: add changelog fragment --- changes/4041.doc.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 changes/4041.doc.md diff --git a/changes/4041.doc.md b/changes/4041.doc.md new file mode 100644 index 0000000000..0aae3b102a --- /dev/null +++ b/changes/4041.doc.md @@ -0,0 +1 @@ +Add guidance for agents to AGENT.md and CLAUDE.md and gitignore agent state. \ No newline at end of file From 43dcd3dbd96aa9d0c4dec0d9f9bb6dd7c40678f9 Mon Sep 17 00:00:00 2001 From: Max Jones <14077947+maxrjones@users.noreply.github.com> Date: Wed, 1 Jul 2026 12:18:22 -0400 Subject: [PATCH 06/11] remove architecture section --- AGENTS.md | 36 ------------------------------------ 1 file changed, 36 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 72437de94f..8d9113e042 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -65,42 +65,6 @@ Two local rules worth knowing because they will reject otherwise-valid code: Every user-facing change needs a news fragment in `changes/` named `{issue-or-pr-number}.{type}.md`, where type is one of `feature`, `bugfix`, `doc`, `removal`, `misc`. Generated into `docs/release-notes.md` by towncrier at release time. `towncrier create` scaffolds one. The `towncrier-check` pre-commit hook will flag PRs missing a fragment. -## Architecture - -### Async core with a generated sync facade - -Every storage and array operation is fundamentally **async**. The two-layer pattern is pervasive and important to preserve: - -- `AsyncArray` / `AsyncGroup` (`core/array.py`, `core/group.py`) hold the real async implementations. -- `Array` / `Group` are thin synchronous wrappers that drive the async methods through `sync()` (`core/sync.py`), which runs coroutines on a single dedicated background event loop (a module-global loop in its own thread). Never call `asyncio.run` or create new loops in library code — route through `sync()`. -- The user-facing top-level functions split the same way: `api/asynchronous.py` (async) and `api/synchronous.py` (sync wrappers). `__init__.py` re-exports the synchronous ones. - -When adding functionality, implement it on the `Async*` class / in `api/asynchronous.py` first, then add the sync wrapper. - -### Store abstraction - -`abc/store.py` defines the abstract `Store` (async `get`/`set`/`delete`/`list`/`exists`, partial reads via `RangeByteRequest`/`OffsetByteRequest`/`SuffixByteRequest`). Concrete stores live in `storage/`: `_local.py`, `_memory.py`, `_zip.py`, `_fsspec.py` (any fsspec filesystem), `_obstore.py` (Rust `obstore` backend), plus `_wrapper.py` and `_logging.py` decorators. `StorePath` (`storage/_common.py`) couples a store with a key prefix and is what arrays/groups actually hold. - -### Codecs and the codec pipeline - -Reading/writing a chunk runs it through an ordered pipeline of codecs, typed by what they transform (`abc/codec.py`): `ArrayArrayCodec` → `ArrayBytesCodec` (exactly one, e.g. `bytes`) → `BytesBytesCodec` (e.g. compressors). `core/codec_pipeline.py` (`BatchedCodecPipeline`) orchestrates encode/decode and batches/concurrency-limits chunk I/O. Built-in codecs are in `codecs/` (`blosc`, `gzip`, `zstd`, `crc32c`, `transpose`, `sharding`, `vlen_utf8`, `scale_offset`, ...); `codecs/numcodecs/` and `codecs/_v2.py` bridge to numcodecs for v2 filters/compressors. **Sharding** (`codecs/sharding.py`) is itself an `ArrayBytesCodec` that packs a grid of sub-chunks into one store object — the most intricate codec. - -### Metadata and v2/v3 duality - -`AsyncArray` is generic over `ArrayV2Metadata | ArrayV3Metadata` (`core/metadata/`). Most of the codebase branches on format version through this metadata type rather than ad-hoc version flags. v2 uses `.zarray`/`.zattrs`/`.zgroup`; v3 uses `zarr.json`. `metadata/migrate_v3.py` and `core/metadata/` hold the conversion and parsing logic. When touching read/write paths, check both metadata classes. - -### dtype system - -`core/dtype/` is a pluggable dtype layer (not raw NumPy dtypes) that maps Zarr data types to/from NumPy and handles fill values, endianness, and v2-vs-v3 dtype naming. `dtype.py` at the package root re-exports it. - -### Registry and config - -`registry.py` is the extension mechanism: codecs, pipelines, buffers, ndbuffers, and chunk-key-encodings are registered by name and discovered via entry points (`zarr.codecs`, `zarr.codec_pipelines`, etc.), so third-party packages can plug in. `core/config.py` uses `donfig` (the `zarr.config` object) for runtime settings — default codecs, concurrency limits, the active buffer/pipeline implementation. Resolving "which class implements X" generally goes through config → registry. - -### Buffers and the GPU path - -`abc/buffer.py` + `core/buffer/` abstract `Buffer` (1-D bytes) and `NDBuffer` (N-D array) so the same code runs on CPU (`core/buffer/cpu.py`, NumPy) or GPU (`core/buffer/gpu.py`, CuPy). GPU-only tests are marked `@pytest.mark.gpu`. Don't hardcode `np.ndarray`; go through the buffer prototype. - ## Conventions - **Public API surface:** anything added to `__all__` in `__init__.py` / `api/synchronous.py` is public and must have a numpydoc docstring and an entry under `docs/api/*.md`. New user-facing behavior also belongs in `docs/user-guide/`. From 1e3c43777d2c5bb736925b2b07e5690c78ad8116 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 10 Jul 2026 15:04:33 +0200 Subject: [PATCH 07/11] docs: restructure contribution guidelines around authorship vs composition Separate accountability (never delegable to a tool) from composition (delegable, if disclosed), and derive the AI policy from that split rather than maintaining a parallel AI-specific section. Machine-composed text is now permitted where labeled, provided a pull request opens with a sentence the author wrote themselves. Review responses must still be in the author's own words. Assisted-by: ClaudeCode:claude-opus-4.8 --- .github/PULL_REQUEST_TEMPLATE.md | 7 +-- docs/contributing.md | 92 +++++++++++++++++--------------- 2 files changed, 52 insertions(+), 47 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 1d12aa02eb..bd691eca89 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -5,7 +5,7 @@ ## Summary -[Describe what this PR changes and why, in your own words.] +[Open with at least one sentence you wrote yourself: why does this change exist, and why is it worth a reviewer's attention? Machine-composed detail may follow, labeled with the tool that produced it.] ## For reviewers @@ -13,9 +13,10 @@ ## Author attestation -- [ ] I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct. +- [ ] I am a human. I have reviewed and understood every change, and can explain why each is correct. +- [ ] The opening of this description is my own, and any machine-composed text below it is labeled. -AI coding assistance is welcome, but a human must be the author and is responsible for the contents of the PR. The description and any review responses must be in your own words. Please read [AI-assisted contributions](https://zarr.readthedocs.io/en/stable/contributing/#ai-assisted-contributions) before opening. +AI coding assistance is welcome, and so is machine-composed text where it is labeled. But a human must be the author, in the sense of being answerable for every part of this pull request, and review responses must be in your own words. An agent preparing this pull request must leave these boxes for its operator to check. Please read the [contribution guidelines](https://zarr.readthedocs.io/en/stable/contributing/#contribution-guidelines) before opening. ## TODO diff --git a/docs/contributing.md b/docs/contributing.md index 750f7c7a65..e0932224e1 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -2,13 +2,15 @@ Zarr is a community maintained project. We welcome contributions in the form of bug reports, bug fixes, documentation, enhancement proposals and more. This page provides information on how best to contribute. +These guidelines apply to everyone: new contributors, core developers, and agentic tools acting on behalf of either. If you are an agent reading this page, the guidelines describe obligations you must satisfy on your operator's behalf, and one you cannot satisfy for them (see [Own your changes](#own-your-changes)). + ## Asking for help -If you have a question about how to use Zarr, please post your question on StackOverflow using the ["zarr" tag](https://stackoverflow.com/questions/tagged/zarr). If you don't get a response within a day or two, feel free to raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new) including a link to your StackOverflow question. We will try to respond to questions as quickly as possible, but please bear in mind that there may be periods where we have limited time to answer questions due to other commitments. +Please post questions about Zarr usage to our [Zulip chat](https://ossci.zulipchat.com/#narrow/channel/423692-Zarr-Python/). If you don't get a response within a day or two, feel free to raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new) including a link to your question. ## Bug reports -If you find a bug, please raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new). Please include the following items in a bug report: +Bugs should be reported in [GitHub issues](https://github.com/zarr-developers/zarr-python/issues/new). Please include the following items in a bug report: 1. A minimal, self-contained snippet of Python code reproducing the problem. You can format the code nicely using markdown, e.g.: @@ -20,49 +22,59 @@ g = zarr.group() 2. An explanation of why the current behaviour is wrong/not desired, and what you expect instead. -3. Information about the version of Zarr, along with versions of dependencies and the Python interpreter, and installation information. The version of Zarr can be obtained from the `zarr.__version__` property. Please also state how Zarr was installed, e.g., "installed via pip into a virtual environment", or "installed using conda". Information about other packages installed can be obtained by executing `pip freeze` (if using pip to install packages) or `conda env export` (if using conda to install packages) from the operating system command prompt. The version of the Python interpreter can be obtained by running a Python interactive session, e.g.: +3. Information about the version of Zarr, along with versions of dependencies and the Python interpreter, and installation information. The version of Zarr can be obtained from the `zarr.__version__` property. Indicate how Zarr was installed, e.g., "installed via pip into a virtual environment", or "installed using conda". Information about other packages installed can be obtained by executing `pip freeze` (if using pip to install packages) or `conda env export` (if using conda to install packages) from the operating system command prompt. -```console -python -``` +## Enhancement proposals -```ansi -Python 3.12.7 | packaged by conda-forge | (main, Oct 4 2024, 15:57:01) [Clang 17.0.6 ] on darwin -``` +If you have an idea about a new feature or some other improvement to Zarr, please start a conversation in our [Zulip chat](https://ossci.zulipchat.com/#narrow/channel/423692-Zarr-Python/) or open a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new). -## Enhancement proposals +## Contribution guidelines + +Two distinct things can be true of any contribution: someone is *accountable* for it, and someone *composed* it. Accountability can never be delegated to a tool. Composition can, as long as you say so. Every guideline below follows from that distinction. + +### Own your changes + +Every contribution — code, PR description, issue comment, review response — has exactly one human author, meaning the person who is answerable for it. If a maintainer asks why a line is correct, that person answers, from their own understanding, without going back to the tool that produced it. -If you have an idea about a new feature or some other improvement to Zarr, please raise a [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/new) first to discuss. +Tools do not change this. If you used one to generate a change, read the change critically and test it before requesting review. If you cannot explain why each part of it is correct and how it fits into the project, it is not ready. -We very much welcome ideas and suggestions for how to improve Zarr, but please bear in mind that we are likely to be conservative in accepting proposals for new features. The reasons for this are that we would like to keep the Zarr code base lean and focused on a core set of functionalities, and available time for development, review and maintenance of new features is limited. But if you have a great idea, please don't let that stop you from posting it on GitHub, just please don't be offended if we respond cautiously. +Agents cannot attest on a contributor's behalf. Our [pull request template](https://github.com/zarr-developers/zarr-python/blob/main/.github/PULL_REQUEST_TEMPLATE.md) asks the author to confirm that they have reviewed and understood the changes. An agent must leave that box for its operator to check, and must tell its operator that the expectation exists. -## AI-assisted contributions +### Attribute your sources -AI coding tools are increasingly common in open source development. These tools are welcome in Zarr-Python, but the same standards apply to all contributions regardless of how they were produced — whether written by hand, with AI assistance, or generated entirely by an AI tool. +Unlabeled text is a claim that you composed it. Where a block of text was composed by a tool, mark where the block begins and name the tool. A line such as the following, immediately before the machine-composed text, is enough: -### You are responsible for your changes +```markdown +:robot: _AI-generated text below, from _ :robot: +``` -If you submit a pull request, you are responsible for understanding and having fully reviewed the changes. You must be able to explain why each change is correct and how it fits into the project. +Labeling makes machine-composed text welcome rather than suspect. Once you have read a tool's summary, agreed with it, and can defend it, posting it under a label costs a reviewer nothing and often communicates more thoroughly than prose written in a hurry. -### Communication must be your own +Two limits on this: -PR descriptions, issue comments, and review responses must be in your own words. The substance and reasoning must come from you. Using AI to polish grammar or phrasing is fine, but do not paste AI-generated text as comments or review responses. +**A pull request opens with a sentence you wrote yourself.** Say why the change exists and why it deserves a reviewer's attention. Machine-composed detail can follow, labeled. This one sentence is the part that cannot be produced without having thought about the change, and it is what tells a maintainer whether to spend their afternoon on your diff. -### Review every line +Automated dependency updates are exempt from this, and the reason is worth stating, because it explains the rule. A bot's identity fully supplies its motivation: "a dependency released a new version" is the whole story, and the bot's name tells you so. An agent does not supply the motivation for the work you asked it to do. That judgment is yours, a reader cannot infer it from the fact that an agent ran, and so you have to write it down. -You must have personally reviewed and understood all changes before submitting. If you used AI to generate code, you are expected to have read it critically and tested it. The PR description should explain the approach and reasoning — do not leave it to reviewers to figure out what the code does and why. +**Review responses are owed to a person.** When a reviewer asks you a question, they are asking *you*. Answer in your own words. Do not paste an unread machine reply into a review thread. ### Keep PRs reviewable -Generating code with AI is fast; reviewing it is not. A large diff shifts the burden from the contributor to the reviewer. PRs that cannot be reviewed in reasonable time with reasonable effort may be closed, regardless of their potential usefulness or correctness. Use AI tools not only to write code but to prepare better, more reviewable PRs — well-structured commits, clear descriptions, and minimal scope. +Generating a change is fast; reviewing it is not. A large diff shifts the burden from you to the reviewer, and a reviewer's time is the scarcest resource this project has. Pull requests that cannot be reviewed in reasonable time with reasonable effort may be closed, however useful or correct they might be. -If you are planning a large AI-assisted contribution (e.g., a significant refactor or a new subsystem), **open an issue first** to discuss the scope and approach with maintainers. Maintainers may also request that large changes be broken into smaller, reviewable pieces. +Tools make it cheap to produce a broad, flat set of unrelated fixes. Such a pull request is hard to review and, because its title cannot describe its contents, it is invisible to the people who would have wanted to weigh in on any one part of it. Prefer to split by topic, even when each individual change is safe. Use tools not only to write code but to prepare a better pull request: coherent commits, a clear description, and minimal scope. -### Documentation +If you plan a large contribution — a significant refactor, a new subsystem — discuss the scope and approach with maintainers first, on our [Zulip chat](https://ossci.zulipchat.com/#narrow/channel/423692-Zarr-Python/) or in an issue. Maintainers may ask that a large change be broken into smaller pieces. + +### Write commit messages for the reader + +A pull request is merged by squashing, so its title becomes a permanent line in the history of `main`. Write it as [a Conventional Commit](https://www.conventionalcommits.org/en/v1.0.0/) — `fix: handle 0-d arrays in save_array` — describing what changed, not which files you touched. Someone reading [the commit list](https://github.com/zarr-developers/zarr-python/commits/main/) should come away with an accurate picture of how the project is developing. -The same principles apply to documentation. Zarr has domain-specific semantics (chunked storage, codec pipelines, Zarr v2/v3 format details) that AI tools frequently get wrong. Do not submit documentation that you haven't carefully read and verified. +### Documentation is held to the same standard -## Contributing code and/or documentation +Zarr has domain-specific semantics — chunked storage, codec pipelines, the differences between Zarr formats 2 and 3 — that language models frequently get wrong, confidently and plausibly. Do not submit documentation you have not read and verified against the behavior of the code. + +## How to contribute ### Forking the repository @@ -80,7 +92,7 @@ git remote add upstream git@github.com:zarr-developers/zarr-python.git ### Creating a development environment -To work with the Zarr source code, it is recommended to use [hatch](https://hatch.pypa.io/latest/index.html) to create and manage development environments. Hatch will automatically install all Zarr dependencies using the same versions as are used by the core developers and continuous integration services. Assuming you have a Python 3 interpreter already installed, and you have cloned the Zarr source code and your current working directory is the root of the repository, you can do something like the following: +To work with the Zarr source code, it is recommended to use [hatch](https://hatch.pypa.io/latest/index.html) to create and manage development environments. Hatch automatically installs the correct Zarr dependencies. Assuming you have a Python 3 interpreter already installed, and you have cloned the Zarr source code and your current working directory is the root of the repository, you can do something like the following: ```bash pip install hatch @@ -125,20 +137,16 @@ Again, any conflicts need to be resolved before submitting a pull request. ### Running the test suite -Zarr includes a suite of unit tests. The simplest way to run the unit tests is to activate your development environment (see [creating a development environment](#creating-a-development-environment) above) and invoke: +Zarr's tests are in the `tests` directory. We use [`pytest`](https://docs.pytest.org/en/stable/). To run the test suite, initialize your development environment (see [creating a development environment](#creating-a-development-environment) above) and invoke: ```bash hatch env run --env test.py3.12-optional run ``` -All tests are automatically run via GitHub Actions for every pull request and must pass before code can be accepted. Test coverage is also collected automatically via the Codecov service. - -> **Note:** Previous versions of Zarr-Python made extensive use of doctests. These tests were not maintained during the 3.0 refactor but may be brought back in the future. See issue #2614 for more details. +All tests are automatically run via GitHub Actions for every pull request. Automated checks must pass before changes are accepted. Test coverage is also collected automatically via the Codecov service. ### Code standards - using prek -All code must conform to the PEP8 standard. Regarding line length, lines up to 100 characters are allowed, although please try to keep under 90 wherever possible. - `Zarr` uses a set of git hooks managed by [`prek`](https://github.com/j178/prek), a fast, Rust-based pre-commit hook manager that is fully compatible with `.pre-commit-config.yaml` files. `prek` can be installed locally by running: ```bash @@ -181,19 +189,17 @@ To list all available hooks: prek list ``` -If you would like to skip the failing checks and push the code for further discussion, use the `--no-verify` option with `git commit`. +To skip failing checks and push the code for further discussion, use the `--no-verify` option with `git commit`. ### Test coverage -> **Note:** Test coverage for Zarr-Python 3 is currently not at 100%. This is a known issue and help is welcome to bring test coverage back to 100%. See issue #2613 for more details. - -Zarr strives to maintain 100% test coverage under the latest Python stable release. Both unit tests and docstring doctests are included when computing coverage. Running: +Zarr strives to maintain high test coverage. Running ```bash hatch env run --env test.py3.12-optional run-coverage ``` -will automatically run the test suite with coverage and produce an XML coverage report. This should be 100% before code can be accepted into the main code base. +will automatically run the test suite with coverage and produce an XML coverage report. You can also generate an HTML coverage report by running: @@ -201,13 +207,13 @@ You can also generate an HTML coverage report by running: hatch env run --env test.py3.12-optional run-coverage-html ``` -When submitting a pull request, coverage will also be collected across all supported Python versions via the Codecov service, and will be reported back within the pull request. Codecov coverage must also be 100% before code can be accepted. +When submitting a pull request, coverage will also be collected across all supported Python versions via the Codecov service, and will be reported back within the pull request. ### Documentation Docstrings for user-facing classes and functions should follow the [numpydoc](https://numpydoc.readthedocs.io/en/stable/format.html#docstring-standard) standard, including sections for Parameters and Examples. All examples should run and pass as doctests under Python 3.12. -Zarr uses mkdocs for documentation, hosted on readthedocs.org. Documentation is written in the Markdown markup language (.md files) in the `docs` folder. The documentation consists both of prose and API documentation. All user-facing classes and functions are included in the API documentation, under the `docs/api` folder using the [mkdocstrings](https://mkdocstrings.github.io/) extension. Add any new public functions or classes to the relevant markdown file in `docs/api/*.md`. Any new features or important usage information should be included in the user-guide (`docs/user-guide`). Any changes should also be included as a new file in the `changes` directory. +Zarr uses mkdocs for documentation, hosted on readthedocs.org. Documentation is written in Markdown (.md files) in the `docs` folder. The documentation consists both of prose and API documentation. All user-facing classes and functions are included in the API documentation, under the `docs/api` folder using the [mkdocstrings](https://mkdocstrings.github.io/) extension. Add new public functions or classes to the relevant markdown file in `docs/api/*.md`. ew features or important usage information should be included in the user-guide (`docs/user-guide`). Any changes should also be included as a new file in the `changes` directory. The documentation can be built locally by running: @@ -255,10 +261,8 @@ print("Hello world") #### Validating code blocks: `exec` vs `test` -Every Python code block in the documentation is checked by a test -(`tests/test_docs.py`) so that examples cannot quietly rot — the bug that motivated -this was an example calling `zarr.create_array(..., mode="w")`, an argument that does -not exist, which went unnoticed because nothing ran it. A block declares *how* it is +Python code blocks in the documentation are checked by tests +(`tests/test_docs.py`). A block declares *how* it is validated using one of two independent attributes: - **`exec="true"`** — Markdown Exec runs the block **at docs-build time to render its @@ -317,7 +321,7 @@ Sometimes, you may want the documentation to build quicker. You can disable code ### Changelog -zarr-python uses [towncrier](https://towncrier.readthedocs.io/en/stable/tutorial.html) to manage release notes. Most pull requests should include at least one news fragment describing the changes. To add a release note, you'll need the GitHub issue or pull request number and the type of your change (`feature`, `bugfix`, `doc`, `removal`, `misc`). With that, run `towncrier create` with your development environment, which will prompt you for the issue number, change type, and the news text: +Zarr Python uses [towncrier](https://towncrier.readthedocs.io/en/stable/tutorial.html) to manage release notes. Most pull requests should include at least one news fragment describing the changes. To add a release note, you'll need the GitHub issue or pull request number and the type of your change (`feature`, `bugfix`, `doc`, `removal`, `misc`). With that, run `towncrier create` with your development environment, which will prompt you for the issue number, change type, and the news text: ```bash towncrier create From e4c33dad5732927aae2b07e3494c51663f827593 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 10 Jul 2026 15:10:04 +0200 Subject: [PATCH 08/11] docs: add commit-message guidance and changelog fragment Correct the squash-merge description: a single-commit PR takes its subject from the commit, not the PR title. Assisted-by: ClaudeCode:claude-opus-4.8 --- changes/4140.doc.md | 1 + docs/contributing.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changes/4140.doc.md diff --git a/changes/4140.doc.md b/changes/4140.doc.md new file mode 100644 index 0000000000..adb20a3d9b --- /dev/null +++ b/changes/4140.doc.md @@ -0,0 +1 @@ +Restructure the contribution guidelines around the distinction between authorship (who is accountable for a contribution, which a tool can never be) and composition (who wrote the text, which a tool may do if it is labeled). Machine-composed text is now explicitly permitted in pull request descriptions where it is labeled with the tool that produced it, provided the description opens with a sentence the author wrote themselves. Review responses must still be in the author's own words. The separate "AI-assisted contributions" section is removed, as its rules now follow from the general guidelines, which apply equally to contributors, core developers, and agents. diff --git a/docs/contributing.md b/docs/contributing.md index f0ce963d87..d3e681f497 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -68,7 +68,7 @@ If you plan a large contribution — a significant refactor, a new subsystem — ### Write commit messages for the reader -A pull request is merged by squashing, so its title becomes a permanent line in the history of `main`. Write it as [a Conventional Commit](https://www.conventionalcommits.org/en/v1.0.0/) — `fix: handle 0-d arrays in save_array` — describing what changed, not which files you touched. Someone reading [the commit list](https://github.com/zarr-developers/zarr-python/commits/main/) should come away with an accurate picture of how the project is developing. +Pull requests are merged by squashing, so each one leaves a single permanent line in the history of `main` — taken from the pull request title, or from the commit subject when there is only one commit. Write that line as [a Conventional Commit](https://www.conventionalcommits.org/en/v1.0.0/) — `fix: handle 0-d arrays in save_array` — describing what changed, not which files you touched. Someone reading [the commit list](https://github.com/zarr-developers/zarr-python/commits/main/) should come away with an accurate picture of how the project is developing. ### Documentation is held to the same standard From 8185b6dca96b4bf1a2cec5bc140e45c1fc85308f Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 10 Jul 2026 15:29:26 +0200 Subject: [PATCH 09/11] docs: reconcile AGENTS.md with the new contribution guidelines Max's AGENTS.md referenced the "AI-assisted contributions" section that this branch removes, and restated the old rule requiring PR descriptions to be in the human's own words. Point it at the Contribution guidelines section instead, and give agents the short imperative rules that bind them directly, rather than duplicating policy prose that agents follow unreliably. Also, from a survey of comparable projects: - Labels now record whether a human read the text before it was posted, not just which tool composed it (after apache/airflow). Provenance alone doesn't tell a reviewer what they need to know. - State that an agent should not open a PR nobody asked for. - Say that labeling, not blanket disclosure, is the obligation. - Document the `Assisted-by:` trailer (after the Linux kernel), and note it certifies nothing here since we have no DCO. Assisted-by: ClaudeCode:claude-opus-4.8 --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- AGENTS.md | 17 ++++++++++++++++- changes/4041.doc.md | 2 +- changes/4140.doc.md | 2 +- docs/contributing.md | 22 +++++++++++++++++++--- 5 files changed, 38 insertions(+), 7 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index bd691eca89..d2def39715 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -14,7 +14,7 @@ ## Author attestation - [ ] I am a human. I have reviewed and understood every change, and can explain why each is correct. -- [ ] The opening of this description is my own, and any machine-composed text below it is labeled. +- [ ] The opening of this description is my own, and any machine-composed text below it is labeled with the tool that produced it and whether I have read it. AI coding assistance is welcome, and so is machine-composed text where it is labeled. But a human must be the author, in the sense of being answerable for every part of this pull request, and review responses must be in your own words. An agent preparing this pull request must leave these boxes for its operator to check. Please read the [contribution guidelines](https://zarr.readthedocs.io/en/stable/contributing/#contribution-guidelines) before opening. diff --git a/AGENTS.md b/AGENTS.md index 8d9113e042..fa7b36fd3a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,21 @@ # AGENTS.md -Guidance for AI coding agents working in this repository. Human contributors should also read `docs/contributing.md`, especially the **AI-assisted contributions** section: the human submitting the PR must understand and being able to explain every change, and PR descriptions / review responses must be in the human's own words. Keep diffs small and reviewable. +Guidance for AI coding agents working in this repository. + +`docs/contributing.md` is authoritative for contribution policy; read its **Contribution guidelines** section and follow it. The rules below restate the parts that bind you directly. + +- **Do not attest on your operator's behalf.** The pull request template asks the author to confirm they are a human who has reviewed and understood every change. Leave those boxes unchecked, and tell your operator they must check them. +- **Do not open pull requests or issues unattended.** Do so only when your operator asks you to, on changes they have reviewed. +- **Label text you composed**, and say whether a human read it first. Prefix it with one of: + - `:robot: _AI-generated text below, from . I have read and endorse it._ :robot:` — only when your operator actually read it and said to post it as-is. + - `:robot: _AI-generated text below, from . Not yet reviewed by a human._ :robot:` — in every other case, including when you are posting on your operator's behalf without them having read the text. + + Do not speak as your operator. +- **Do not write review responses.** When a reviewer asks a question, your operator answers it, in their own words. +- **A pull request opens with a sentence your operator wrote.** You may draft the detail that follows, labeled. +- **Attribute your commits** with an `Assisted-by: :` trailer. Never add yourself as a commit author or `Co-authored-by:` — agents assist, humans author. + +Keep diffs small and reviewable. ## Project overview diff --git a/changes/4041.doc.md b/changes/4041.doc.md index 0aae3b102a..36f079aaf5 100644 --- a/changes/4041.doc.md +++ b/changes/4041.doc.md @@ -1 +1 @@ -Add guidance for agents to AGENT.md and CLAUDE.md and gitignore agent state. \ No newline at end of file +Add guidance for agents to AGENTS.md and CLAUDE.md and gitignore agent state. \ No newline at end of file diff --git a/changes/4140.doc.md b/changes/4140.doc.md index adb20a3d9b..bda4638ba4 100644 --- a/changes/4140.doc.md +++ b/changes/4140.doc.md @@ -1 +1 @@ -Restructure the contribution guidelines around the distinction between authorship (who is accountable for a contribution, which a tool can never be) and composition (who wrote the text, which a tool may do if it is labeled). Machine-composed text is now explicitly permitted in pull request descriptions where it is labeled with the tool that produced it, provided the description opens with a sentence the author wrote themselves. Review responses must still be in the author's own words. The separate "AI-assisted contributions" section is removed, as its rules now follow from the general guidelines, which apply equally to contributors, core developers, and agents. +Restructure the contribution guidelines around the distinction between authorship (who is accountable for a contribution, which a tool can never be) and composition (who wrote the text, which a tool may do if it is labeled). Machine-composed text is now explicitly permitted in pull request descriptions where it is labeled with the tool that produced it and with whether a human has read it, provided the description opens with a sentence the author wrote themselves. Review responses must still be in the author's own words. Commits may carry an `Assisted-by:` trailer, but a tool is never an author. The separate "AI-assisted contributions" section is removed, as its rules now follow from the general guidelines, which apply equally to contributors, core developers, and agents; `AGENTS.md` restates the agent-facing rules and defers to the contributing guide. diff --git a/docs/contributing.md b/docs/contributing.md index d3e681f497..4cbf233ab6 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -38,18 +38,26 @@ Every contribution — code, PR description, issue comment, review response — Tools do not change this. If you used one to generate a change, read the change critically and test it before requesting review. If you cannot explain why each part of it is correct and how it fits into the project, it is not ready. -Agents cannot attest on a contributor's behalf. Our [pull request template](https://github.com/zarr-developers/zarr-python/blob/main/.github/PULL_REQUEST_TEMPLATE.md) asks the author to confirm that they have reviewed and understood the changes. An agent must leave that box for its operator to check, and must tell its operator that the expectation exists. +Agents cannot attest on a contributor's behalf. Our [pull request template](https://github.com/zarr-developers/zarr-python/blob/main/.github/PULL_REQUEST_TEMPLATE.md) asks the author to confirm that they have reviewed and understood the changes. An agent must leave that box for its operator to check, and must tell its operator that the expectation exists. For the same reason, an agent should not open a pull request that no human has asked for and read: a pull request nobody is answerable for is one we would have to close. + +Agent-specific instructions live in [`AGENTS.md`](https://github.com/zarr-developers/zarr-python/blob/main/AGENTS.md), which restates the rules below in a form an agent can follow directly. This page remains authoritative. ### Attribute your sources -Unlabeled text is a claim that you composed it. Where a block of text was composed by a tool, mark where the block begins and name the tool. A line such as the following, immediately before the machine-composed text, is enough: +Unlabeled text is a claim that you composed it. Where a block of text was composed by a tool, mark where the block begins, name the tool, and say whether you read the text before posting it. A line such as one of these, immediately before the machine-composed text, is enough: ```markdown -:robot: _AI-generated text below, from _ :robot: +:robot: _AI-generated text below, from . I have read and endorse it._ :robot: + +:robot: _AI-generated text below, from . Not yet reviewed by a human._ :robot: ``` +That second clause is the one that carries information. Whether a tool or a person typed the words changes little; whether a person read them before they reached a reviewer changes everything. A reviewer who knows which they are looking at can spend their attention accordingly. + Labeling makes machine-composed text welcome rather than suspect. Once you have read a tool's summary, agreed with it, and can defend it, posting it under a label costs a reviewer nothing and often communicates more thoroughly than prose written in a hurry. +We ask for labeling rather than a blanket disclosure of every use of a tool. Where a tool helped you think, or fixed your grammar, or you rewrote its output until it became yours, there is nothing to label and nothing to declare. The obligation attaches to text a reviewer might otherwise mistake for yours. + Two limits on this: **A pull request opens with a sentence you wrote yourself.** Say why the change exists and why it deserves a reviewer's attention. Machine-composed detail can follow, labeled. This one sentence is the part that cannot be produced without having thought about the change, and it is what tells a maintainer whether to spend their afternoon on your diff. @@ -70,6 +78,14 @@ If you plan a large contribution — a significant refactor, a new subsystem — Pull requests are merged by squashing, so each one leaves a single permanent line in the history of `main` — taken from the pull request title, or from the commit subject when there is only one commit. Write that line as [a Conventional Commit](https://www.conventionalcommits.org/en/v1.0.0/) — `fix: handle 0-d arrays in save_array` — describing what changed, not which files you touched. Someone reading [the commit list](https://github.com/zarr-developers/zarr-python/commits/main/) should come away with an accurate picture of how the project is developing. +Where a tool wrote a meaningful part of a commit, record it with a trailer naming the harness and the model: + +``` +Assisted-by: ClaudeCode:claude-opus-4.8 +``` + +Do not list a tool as an author or `Co-authored-by:`. This project has no Developer Certificate of Origin, so the trailer certifies nothing; it is attribution, and it lets us see later how the project was built. Authorship stays with the person who signed up to answer for the change. + ### Documentation is held to the same standard Zarr has domain-specific semantics — chunked storage, codec pipelines, the differences between Zarr formats 2 and 3 — that language models frequently get wrong, confidently and plausibly. Do not submit documentation you have not read and verified against the behavior of the code. From ce34fe65fe546b486b6a64c6781691cba02e4305 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 10 Jul 2026 15:49:25 +0200 Subject: [PATCH 10/11] docs: require trimming endorsed machine-composed text Two borrowings from ghostty's AI_POLICY.md, which frames this better than anyone else surveyed. Its rule is "reviewed _and edited_ by a human", with the reason that "AI is very good at being overly verbose and including noise that distracts from the main point." Our label asked only whether a human had read the text. Reading it does not spare the reviewer the padding; cutting it does. Endorsing machine-composed text now means having trimmed it, and AGENTS.md tells agents to write less. Also open the section by naming the harm the way ghostty does: posting unread text at the boundary where a person meets your work shifts the burden of validating it onto them. That is a discourtesy, not a procedural violation, and saying so is more persuasive than a rule. Deliberately not adopted: ghostty's maintainer exemption (contrary to what this branch exists to settle) and its public denouncement list (calibrated for a flood of anonymous drive-bys we do not have). Assisted-by: ClaudeCode:claude-opus-4.8 --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- AGENTS.md | 3 ++- changes/4140.doc.md | 2 +- docs/contributing.md | 6 +++++- 4 files changed, 9 insertions(+), 4 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index d2def39715..021ad4ba29 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -14,7 +14,7 @@ ## Author attestation - [ ] I am a human. I have reviewed and understood every change, and can explain why each is correct. -- [ ] The opening of this description is my own, and any machine-composed text below it is labeled with the tool that produced it and whether I have read it. +- [ ] The opening of this description is my own. Any machine-composed text below it is labeled with the tool that produced it and whether I have read it — and where I endorsed it, I trimmed it to what a reviewer needs. AI coding assistance is welcome, and so is machine-composed text where it is labeled. But a human must be the author, in the sense of being answerable for every part of this pull request, and review responses must be in your own words. An agent preparing this pull request must leave these boxes for its operator to check. Please read the [contribution guidelines](https://zarr.readthedocs.io/en/stable/contributing/#contribution-guidelines) before opening. diff --git a/AGENTS.md b/AGENTS.md index fa7b36fd3a..521ab7294f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,10 +7,11 @@ Guidance for AI coding agents working in this repository. - **Do not attest on your operator's behalf.** The pull request template asks the author to confirm they are a human who has reviewed and understood every change. Leave those boxes unchecked, and tell your operator they must check them. - **Do not open pull requests or issues unattended.** Do so only when your operator asks you to, on changes they have reviewed. - **Label text you composed**, and say whether a human read it first. Prefix it with one of: - - `:robot: _AI-generated text below, from . I have read and endorse it._ :robot:` — only when your operator actually read it and said to post it as-is. + - `:robot: _AI-generated text below, from . I have read and endorse it._ :robot:` — only when your operator actually read it, edited it, and said to post it as-is. - `:robot: _AI-generated text below, from . Not yet reviewed by a human._ :robot:` — in every other case, including when you are posting on your operator's behalf without them having read the text. Do not speak as your operator. +- **Write less.** A human reads everything you post here. Say what changed and why it is correct; cut restatement of the diff, hedging, and summary of your own process. If your operator has to trim it before posting, you wrote too much. - **Do not write review responses.** When a reviewer asks a question, your operator answers it, in their own words. - **A pull request opens with a sentence your operator wrote.** You may draft the detail that follows, labeled. - **Attribute your commits** with an `Assisted-by: :` trailer. Never add yourself as a commit author or `Co-authored-by:` — agents assist, humans author. diff --git a/changes/4140.doc.md b/changes/4140.doc.md index bda4638ba4..4f36218be2 100644 --- a/changes/4140.doc.md +++ b/changes/4140.doc.md @@ -1 +1 @@ -Restructure the contribution guidelines around the distinction between authorship (who is accountable for a contribution, which a tool can never be) and composition (who wrote the text, which a tool may do if it is labeled). Machine-composed text is now explicitly permitted in pull request descriptions where it is labeled with the tool that produced it and with whether a human has read it, provided the description opens with a sentence the author wrote themselves. Review responses must still be in the author's own words. Commits may carry an `Assisted-by:` trailer, but a tool is never an author. The separate "AI-assisted contributions" section is removed, as its rules now follow from the general guidelines, which apply equally to contributors, core developers, and agents; `AGENTS.md` restates the agent-facing rules and defers to the contributing guide. +Restructure the contribution guidelines around the distinction between authorship (who is accountable for a contribution, which a tool can never be) and composition (who wrote the text, which a tool may do if it is labeled). Machine-composed text is now explicitly permitted in pull request descriptions where it is labeled with the tool that produced it and with whether a human has read it, provided the description opens with a sentence the author wrote themselves. Endorsing such text means having trimmed it, not merely read it. Review responses must still be in the author's own words. Commits may carry an `Assisted-by:` trailer, but a tool is never an author. The separate "AI-assisted contributions" section is removed, as its rules now follow from the general guidelines, which apply equally to contributors, core developers, and agents; `AGENTS.md` restates the agent-facing rules and defers to the contributing guide. diff --git a/docs/contributing.md b/docs/contributing.md index 4cbf233ab6..ce99b9fb87 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -44,6 +44,8 @@ Agent-specific instructions live in [`AGENTS.md`](https://github.com/zarr-develo ### Attribute your sources +There are humans here. Every issue, pull request, and review comment is read by a person, and it is the point where your work meets their attention. Arriving at that boundary with text nobody has read is not a procedural violation; it is a discourtesy, because it moves the work of finding out whether the text is any good from you onto them. The rules below exist to keep that from happening, not to police how you write. + Unlabeled text is a claim that you composed it. Where a block of text was composed by a tool, mark where the block begins, name the tool, and say whether you read the text before posting it. A line such as one of these, immediately before the machine-composed text, is enough: ```markdown @@ -54,7 +56,9 @@ Unlabeled text is a claim that you composed it. Where a block of text was compos That second clause is the one that carries information. Whether a tool or a person typed the words changes little; whether a person read them before they reached a reviewer changes everything. A reviewer who knows which they are looking at can spend their attention accordingly. -Labeling makes machine-composed text welcome rather than suspect. Once you have read a tool's summary, agreed with it, and can defend it, posting it under a label costs a reviewer nothing and often communicates more thoroughly than prose written in a hurry. +Reading it is not enough — cut it. Machine-composed text runs long. It restates the diff, hedges, and pads a finding out to three paragraphs. If you post it under the endorsing label, you are saying you have not only read it but trimmed it to what a reviewer needs, and you are answerable for every sentence that survived. Text you have not edited is text you have not really read, so label it as unreviewed and say so plainly. + +Labeling makes machine-composed text welcome rather than suspect. Once you have read a tool's summary, cut it down, agreed with what remains, and can defend it, posting it under a label costs a reviewer nothing and often communicates more thoroughly than prose written in a hurry. We ask for labeling rather than a blanket disclosure of every use of a tool. Where a tool helped you think, or fixed your grammar, or you rewrote its output until it became yours, there is nothing to label and nothing to declare. The obligation attaches to text a reviewer might otherwise mistake for yours. From 4c40ecfe7a1950267079c4ea7ade0178f02d9bce Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 10 Jul 2026 15:58:57 +0200 Subject: [PATCH 11/11] docs: forbid agents from drafting the author's opening sentence The rule said a pull request opens with a sentence the author wrote, but never said an agent must not supply one. An agent drafting a sentence for its operator to paste satisfies the letter and defeats the purpose: the sentence is evidence that a human thought about the change, and evidence you were handed is not evidence. State it in contributing.md, which is authoritative, and restate it as an imperative in AGENTS.md and in the template's Summary prompt. Keep the phrasing tool-neutral -- the rule binds any agent, not one vendor. Assisted-by: ClaudeCode:claude-opus-4.8 --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- AGENTS.md | 2 +- docs/contributing.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 021ad4ba29..58b36d3eeb 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -5,7 +5,7 @@ ## Summary -[Open with at least one sentence you wrote yourself: why does this change exist, and why is it worth a reviewer's attention? Machine-composed detail may follow, labeled with the tool that produced it.] +[Open with at least one sentence you wrote yourself: why does this change exist, and why is it worth a reviewer's attention? No agent may write this sentence for you. Machine-composed detail may follow, labeled with the tool that produced it.] ## For reviewers diff --git a/AGENTS.md b/AGENTS.md index 521ab7294f..90369a937b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,7 +13,7 @@ Guidance for AI coding agents working in this repository. Do not speak as your operator. - **Write less.** A human reads everything you post here. Say what changed and why it is correct; cut restatement of the diff, hedging, and summary of your own process. If your operator has to trim it before posting, you wrote too much. - **Do not write review responses.** When a reviewer asks a question, your operator answers it, in their own words. -- **A pull request opens with a sentence your operator wrote.** You may draft the detail that follows, labeled. +- **A pull request opens with a sentence your operator wrote.** Do not write it, and do not draft one for them to paste — the point is that they thought about the change, and a sentence you supplied proves nothing. Leave a placeholder saying so. You may draft the detail that follows, labeled. - **Attribute your commits** with an `Assisted-by: :` trailer. Never add yourself as a commit author or `Co-authored-by:` — agents assist, humans author. Keep diffs small and reviewable. diff --git a/docs/contributing.md b/docs/contributing.md index ce99b9fb87..9326845d1b 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -64,7 +64,7 @@ We ask for labeling rather than a blanket disclosure of every use of a tool. Whe Two limits on this: -**A pull request opens with a sentence you wrote yourself.** Say why the change exists and why it deserves a reviewer's attention. Machine-composed detail can follow, labeled. This one sentence is the part that cannot be produced without having thought about the change, and it is what tells a maintainer whether to spend their afternoon on your diff. +**A pull request opens with a sentence you wrote yourself.** Say why the change exists and why it deserves a reviewer's attention. Machine-composed detail can follow, labeled. This one sentence is the part that cannot be produced without having thought about the change, and it is what tells a maintainer whether to spend their afternoon on your diff. No agent may write it for you, or draft one for you to paste: a sentence you did not think of proves nothing, which is the only thing this rule is for. Automated dependency updates are exempt from this, and the reason is worth stating, because it explains the rule. A bot's identity fully supplies its motivation: "a dependency released a new version" is the whole story, and the bot's name tells you so. An agent does not supply the motivation for the work you asked it to do. That judgment is yours, a reader cannot infer it from the fact that an agent ran, and so you have to write it down.