feat(api): add URL-pipeline-native create_v3_array, create_v3_group, create_v2_array and create_v2_group

changes/+create-v3.feature.md

@@ -0,0 +1,12 @@
+Added `zarr.create_v3_array`, `zarr.create_v3_group`, `zarr.create_v2_array` and
+`zarr.create_v2_group`: creation functions that are native to URL pipelines and to one Zarr
+format each. Their keyword arguments are that format's metadata fields (for V3 `shape`,
+`data_type`, `chunk_grid`, `codecs`, `chunk_key_encoding`, `fill_value`, `attributes`,
+`dimension_names`, `storage_transformers`; for V2 `shape`, `dtype`, `chunks`, `fill_value`,
+`order`, `dimension_separator`, `compressor`, `filters`, `attributes`), with no translation
+layer between formats. Fields left at the new `zarr.AUTO` sentinel are computed from the
+shape, the data type and the configuration, while `None` keeps the meaning the metadata
+gives it (for example no compressor). A string location is always a URL pipeline such as
+`"file:/data/example.zarr|zarr3:group/array"`, so its root must carry a scheme and a literal
+`|` in a local path is spelled `%7C`; a `URLPipeline`, a `Store` or a `StorePath` is accepted
+as well. Creation fails if a node already exists at the location unless `overwrite=True`.

changes/4192.feature.md

@@ -22,8 +22,8 @@ Behavior notes:
   parameter mirroring `Group.open`.
 - Inside a pipeline, `memory:` and `file:` roots follow the URL pipeline spec's spelling
   rules (`memory:` ≡ `memory:/` ≡ `memory://`; `file:` must carry an absolute path with
-  at most a `localhost` authority). Percent-escapes are not decoded, matching
-  `LocalStore`. A pipeline's `memory:` root is zarr's `ManagedMemoryStore`, which does
+  at most a `localhost` authority). Percent-escapes in these roots are decoded, as in
+  any URL. A pipeline's `memory:` root is zarr's `ManagedMemoryStore`, which does
   not share data with the fsspec in-memory filesystem a plain `memory://name` string
   resolves to.
 - Wrapper adapters can open a local *file* root only in mode `"r"` (via

docs/api/zarr/functions/create_v2_array.md

@@ -0,0 +1,5 @@
+---
+title: zarr.create_v2_array
+---
+
+::: zarr.create_v2_array

docs/api/zarr/functions/create_v2_group.md

@@ -0,0 +1,5 @@
+---
+title: zarr.create_v2_group
+---
+
+::: zarr.create_v2_group

docs/api/zarr/functions/create_v3_array.md

@@ -0,0 +1,5 @@
+---
+title: zarr.create_v3_array
+---
+
+::: zarr.create_v3_array

docs/api/zarr/functions/create_v3_group.md

@@ -0,0 +1,5 @@
+---
+title: zarr.create_v3_group
+---
+
+::: zarr.create_v3_group

docs/user-guide/storage.md

@@ -135,9 +135,26 @@ print(zarr.open_url("memory://pipeline-demo|zarr2:").metadata.zarr_format)
 Only these two entry points read a string as a pipeline. Plain string store
 specifications keep their existing meaning everywhere else, so a local path that
 happens to contain `|` still names a directory, whatever adapters are installed.
-Unlike `zarr.open`, `zarr.open_url` never creates or overwrites: to create a node at a
-pipeline-addressed location, pass the `URLPipeline` to `zarr.create_array` or
-`zarr.create_group`.
+Unlike `zarr.open`, `zarr.open_url` never creates or overwrites. To create a node at a
+pipeline-addressed location, use [`zarr.create_v3_array`][] / [`zarr.create_v3_group`][]
+(or their Zarr format 2 counterparts [`zarr.create_v2_array`][] / [`zarr.create_v2_group`][]),
+which read a string location as a pipeline and take that format's metadata fields as
+keyword arguments, or pass the `URLPipeline` to `zarr.create_array` / `zarr.create_group`:
+
+```python exec="true" session="storage" source="above" result="ansi"
+arr = zarr.create_v3_array(
+    "memory://pipeline-demo|zarr3:temperature",
+    shape=(365, 180, 360),
+    data_type="float32",
+    chunk_grid=(30, 180, 360),
+    dimension_names=("time", "lat", "lon"),
+)
+print(arr)
+print(zarr.open_url("memory://pipeline-demo|zarr3:temperature").metadata.dimension_names)
+```
+
+In these functions a string is *always* a pipeline, so its root must carry a URL scheme and
+a literal `|` in a local path is spelled `%7C`, as in `file:/data/a%7Cb.zarr`.
 
 A trailing `zarr2:` or `zarr3:` segment is a *format segment*. It is not an adapter:
 zarr-python reads it from the pipeline itself and uses it as the `zarr_format` of the
@@ -147,16 +164,16 @@ open. A `zarr_format` argument that conflicts with the format segment raises
 and `Group.open`) need an explicit `zarr_format=None` to defer to the pipeline.
 
 `storage_options` apply to the *root* sub-URL (e.g. fsspec options for `s3://...`);
-adapters may consume adapter-specific, namespaced keys. No percent-escape is decoded
-anywhere in a pipeline. Registered adapters cannot intercept zarr's native `file:` and
-`memory:` root schemes.
+adapters may consume adapter-specific, namespaced keys. Percent-escapes are decoded in
+zarr's native `file:` and `memory:` roots, as in any URL; adapter segments receive their
+bodies verbatim. Registered adapters cannot intercept zarr's native `file:` and `memory:`
+root schemes.
 
 Inside a pipeline, a `memory:` root is zarr's managed in-memory store (`memory:`,
 `memory:/` and `memory://` are equivalent, and `memory:name` selects a named store).
 When fsspec is installed, a plain `memory://name` string store specification is still
 routed to fsspec's in-memory filesystem, which is a different store. A `file:` root
-must carry an absolute path; percent-escapes are not decoded, matching the
-[local store](#local-store). The root always carries a scheme: a local path is spelled
+must carry an absolute path. The root always carries a scheme: a local path is spelled
 as a `file:` URL, never as a bare path.
 
 ## Explicit Store Creation

mkdocs.yml

@@ -62,6 +62,10 @@ nav:
       - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_array</code>': api/zarr/functions/create_array.md
       - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_group</code>': api/zarr/functions/create_group.md
       - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_hierarchy</code>': api/zarr/functions/create_hierarchy.md
+      - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_v2_array</code>': api/zarr/functions/create_v2_array.md
+      - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_v2_group</code>': api/zarr/functions/create_v2_group.md
+      - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_v3_array</code>': api/zarr/functions/create_v3_array.md
+      - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.create_v3_group</code>': api/zarr/functions/create_v3_group.md
       - '<code class="doc-symbol doc-symbol-toc doc-symbol-module"></code> <code>zarr.dtype</code>': api/zarr/dtype.md
       - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.empty</code>': api/zarr/functions/empty.md
       - '<code class="doc-symbol doc-symbol-toc doc-symbol-function"></code> <code>zarr.empty_like</code>': api/zarr/functions/empty_like.md

pyproject.toml

@@ -159,7 +159,7 @@ dev = [
     {include-group = "remote-tests"},
     {include-group = "docs"},
     "universal-pathlib",
-    "mypy==2.3.1",
+    "mypy==2.4.0",
 ]
 
 [tool.coverage.report]

src/zarr/__init__.py

@@ -13,6 +13,10 @@
     create_array,
     create_group,
     create_hierarchy,
+    create_v2_array,
+    create_v2_group,
+    create_v3_array,
+    create_v3_group,
     empty,
     empty_like,
     from_array,
@@ -36,6 +40,7 @@
     zeros_like,
 )
 from zarr.core.array import Array, AsyncArray
+from zarr.core.common import AUTO
 from zarr.core.config import config
 from zarr.core.group import AsyncGroup, Group
 
@@ -147,6 +152,7 @@ def set_format(log_format: str) -> None:
 
 
 __all__ = [
+    "AUTO",
     "Array",
     "AsyncArray",
     "AsyncGroup",
@@ -162,6 +168,10 @@ def set_format(log_format: str) -> None:
     "create_array",
     "create_group",
     "create_hierarchy",
+    "create_v2_array",
+    "create_v2_group",
+    "create_v3_array",
+    "create_v3_group",
     "empty",
     "empty_like",
     "from_array",

src/zarr/abc/url_pipeline.py

@@ -16,11 +16,12 @@
 
 from __future__ import annotations
 
-import enum
 from abc import ABC, abstractmethod
 from dataclasses import dataclass
 from typing import TYPE_CHECKING, Any
 
+from typing_extensions import Sentinel
+
 from zarr.errors import URLPipelineError
 
 if TYPE_CHECKING:
@@ -35,11 +36,8 @@
 ]
 
 
-class _Unset(enum.Enum):
-    token = 0
-
-
-_UNSET = _Unset.token
+_UNSET = Sentinel("_UNSET")
+"""Marks a `resolve_preceding` argument the adapter did not override."""
 
 
 @dataclass(frozen=True)
@@ -164,8 +162,8 @@ def preceding_url(self) -> str:
     async def resolve_preceding(
         self,
         *,
-        mode: AccessModeLiteral | _Unset | None = _UNSET,
-        storage_options: dict[str, Any] | _Unset | None = _UNSET,
+        mode: AccessModeLiteral | _UNSET | None = _UNSET,
+        storage_options: dict[str, Any] | _UNSET | None = _UNSET,
     ) -> AdapterResolution:
         """
         Resolve the preceding pipeline into a store.
@@ -208,9 +206,9 @@ async def resolve_preceding(
             )
         return await _resolve(
             self.preceding,
-            mode=self.mode if isinstance(mode, _Unset) else mode,
+            mode=self.mode if mode is _UNSET else mode,
             storage_options=(
-                self.storage_options if isinstance(storage_options, _Unset) else storage_options
+                self.storage_options if storage_options is _UNSET else storage_options
             ),
         )