docs: add roadmap page outlining future plans#4149
Draft
d-v-b wants to merge 3 commits into
Draft
Conversation
Adds a Roadmap page to the documentation describing the goals for the
next major cycle of work ("v4"), the intended changes per theme, and
the three-stream release model (additive minors, deprecations, one
minimal removals-only major).
Assisted-by: ClaudeCode:claude-fable-5
Contributor
Author
|
cc @ilan-gold @maxrjones, opening this as a draft for visibility. lmk if there's anything more or less we should say here |
Contributor
This doesn't lead me where I thought it would, somehow linking to an old page |
ilan-gold
reviewed
Jul 16, 2026
| Zarr-Python around the stack, so that each level is something you can depend | ||
| on, conform to, or replace, without buying every other level: | ||
|
|
||
| - **A focused package per level** — `zarr-metadata`, `zarr-store`, |
Contributor
There was a problem hiding this comment.
Would maybe note the zarrs example here
Comment on lines
+58
to
+60
| complementary rather than competitive: Zarr-Python aims to be the best | ||
| pure-Python Zarr implementation *and* the best wrapper around the | ||
| compiled-language implementations, so that users who need native throughput can |
Contributor
There was a problem hiding this comment.
I'm really conflicted on this, because I think baking fragmentation into the ecosystem is not great, but the flipside is that languages may have different features/advantages. It would be great to have a concrete reason (or list of reasons) as to why we would design around this level of extensibility
Contributor
Author
There was a problem hiding this comment.
good points, I'd say we want the backend to be an extension point for a few reasons:
- we always want a traditional python impl. IMO it's our job as the python implementation, and it keeps the ecosystem healthy to have multiple implementations
- we can't guarantee uniform, complete feature support across all implementations. As long as we allow people to hack on stuff in python, there will be things the python impl does that e.g. a rust impl doesn't support, and that's OK. For now, a compiled backend is effectively an external dep, and we simply can't promise what external deps will do in perpetuity.
- IMO a full backend is just an refined version of our current model, which requires extenders manage buffer prototypes, data type implementations, codec implementations, and store implementations. Consolidating all of this pluggability on a single surface should make extending Zarr Python easier even for other python backends (like a cupy backend, or a pysparse backend)
|
|
||
| ### Codecs | ||
|
|
||
| The current codec API wraps every codec in an unnecessary async layer (a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Roadmap page to the documentation describing the goals for the
next major cycle of work ("v4"), the intended changes per theme, and
the three-stream release model (additive minors, deprecations, one
minimal removals-only major), with an option for a
legacymodule.Assisted-by: ClaudeCode:claude-fable-5
Author attestation