-
Notifications
You must be signed in to change notification settings - Fork 95
Implement Zarr spatial and proj conventions support #883
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
emmanuelmathot
wants to merge
25
commits into
corteva:master
Choose a base branch
from
emmanuelmathot:zarr_conventions
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
19b41ae
FEAT: Implement Zarr spatial and proj conventions support with parsin…
emmanuelmathot 608f3a9
FEAT: Enhance spatial dimension handling to prioritize 'spatial:dimen…
emmanuelmathot 4a88399
FEAT: Implement Zarr conventions declaration and validation functions…
emmanuelmathot 606a7d7
REFAC: Simplify PROJJSON handling in crs_from_user_input function
emmanuelmathot 9fc918d
Add support for Zarr spatial and proj conventions
emmanuelmathot 773b50a
docs: Update history and documentation for Zarr support
emmanuelmathot 350efad
FEAT: Update Zarr conventions to default to WKT2 format and enhance c…
emmanuelmathot 6c9708a
FEAT: Update convention handling to default to None and prefer CF whe…
emmanuelmathot 6bcec17
FEAT: Enhance Zarr convention handling with new write_conventions met…
emmanuelmathot 3d1340f
FEAT: Enhance fallback handling for Zarr conventions in dimension and…
emmanuelmathot cee37b9
FEAT: Update Zarr conventions handling in writing functions and impro…
emmanuelmathot 8cdb7bb
FEAT: Update documentation and examples for Zarr conventions handling…
emmanuelmathot e15cffc
Implement feature X to enhance user experience and optimize performance
emmanuelmathot 6480bed
FEAT: Update tests for Zarr convention handling and improve assertion…
emmanuelmathot 008b325
FEAT: Refactor Zarr convention functions for improved readability and…
emmanuelmathot 4682037
FEAT: Update example notebook to include execution counts and output …
emmanuelmathot 429e767
FEAT: Update documentation to include links for Zarr conventions
emmanuelmathot b7d1c7d
FEAT: Update documentation and examples for Zarr conventions and impr…
emmanuelmathot 6b6070b
REFAC: Simplify error handling and improve readability in spatial met…
emmanuelmathot 41b5983
REFAC: Remove unnecessary blank lines in test_convention_architecture…
emmanuelmathot cad8ecb
REFAC: Update convention checks to use 'is' for identity comparison i…
emmanuelmathot f0389ca
REFAC: Remove redundant import of EXPORT_GRID_MAPPING and get_option …
emmanuelmathot 500e781
REFAC: Rename spatial metadata functions for clarity and consistency …
emmanuelmathot 062ed4b
REFAC: Remove unused format parameters and streamline CRS writing in …
emmanuelmathot 8b01209
Refactor code structure for improved readability and maintainability
emmanuelmathot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| Geospatial Metadata Conventions | ||
| =============================== | ||
|
|
||
| Overview | ||
| -------- | ||
|
|
||
| rioxarray supports two geospatial metadata conventions for storing coordinate reference system (CRS) and transform information: | ||
|
|
||
| 1. **CF (Climate and Forecasts) Convention** - NetCDF convention using grid_mapping coordinates | ||
| 2. **Zarr Spatial and Proj Conventions** - Cloud-native conventions using direct attributes | ||
|
|
||
| Convention Selection | ||
| -------------------- | ||
|
|
||
| rioxarray uses CF conventions by default. When convention is set to ``None`` (the default), rioxarray uses CF conventions but will fallback to reading Zarr conventions if they are explicitly declared in the data. | ||
|
|
||
| Global Setting | ||
| ~~~~~~~~~~~~~~ | ||
|
|
||
| Set the default convention globally using ``set_options``: | ||
|
|
||
| .. code-block:: python | ||
| import rioxarray | ||
| from rioxarray import Convention | ||
| # Use CF convention with Zarr fallback (default) | ||
| rioxarray.set_options(convention=None) | ||
| # Use CF conventions exclusively | ||
| rioxarray.set_options(convention=Convention.CF) | ||
| # Use Zarr conventions exclusively | ||
| rioxarray.set_options(convention=Convention.Zarr) | ||
| Per-Method Override | ||
| ~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Override the global setting for individual method calls: | ||
|
|
||
| .. code-block:: python | ||
| # Write CRS using CF convention (default) | ||
| data.rio.write_crs("EPSG:4326") | ||
| # Write CRS using Zarr convention | ||
| data.rio.write_crs("EPSG:4326", convention=Convention.Zarr) | ||
| # Write transform using Zarr convention | ||
| data.rio.write_transform(transform, convention=Convention.Zarr) | ||
| CF Convention | ||
| ------------- | ||
|
|
||
| The CF (Climate and Forecasts) convention: | ||
|
|
||
| - CRS information stored in a grid_mapping coordinate variable | ||
| - Transform information stored as ``GeoTransform`` attribute on the grid_mapping coordinate | ||
| - Compatible with NetCDF and GDAL tools | ||
| - Verbose but widely supported | ||
|
|
||
| Example: | ||
|
|
||
| .. code-block:: python | ||
| import rioxarray | ||
| from rioxarray import Convention | ||
| # Write using CF convention | ||
| data_cf = data.rio.write_crs("EPSG:4326", convention=Convention.CF) | ||
| data_cf = data_cf.rio.write_transform(transform, convention=Convention.CF) | ||
| # Results in: | ||
| # - Grid mapping coordinate with CRS attributes | ||
| # - GeoTransform attribute with space-separated transform values | ||
| Zarr Conventions | ||
| ---------------- | ||
|
|
||
| The Zarr spatial and proj conventions provide a cloud-native approach: | ||
|
|
||
| - CRS information stored as direct attributes (``proj:code``, ``proj:wkt2``, ``proj:projjson``) | ||
| - Transform stored as ``spatial:transform`` numeric array attribute | ||
| - Spatial metadata in ``spatial:dimensions``, ``spatial:shape``, ``spatial:bbox`` | ||
| - Lightweight and efficient for cloud storage | ||
|
|
||
| Example: | ||
|
|
||
| .. code-block:: python | ||
| import rioxarray | ||
| from rioxarray import Convention | ||
| # Write using Zarr conventions | ||
| data_zarr = data.rio.write_crs("EPSG:4326", convention=Convention.Zarr) | ||
| data_zarr = data_zarr.rio.write_transform(transform, convention=Convention.Zarr) | ||
| # Write both CRS and transform using Zarr conventions | ||
| data_zarr = data.rio.write_crs("EPSG:4326", convention=Convention.Zarr) | ||
| data_zarr = data_zarr.rio.write_transform(transform, convention=Convention.Zarr) | ||
| Writing Zarr Conventions | ||
| ------------------------ | ||
|
|
||
| To write data using Zarr conventions, use the ``convention`` parameter: | ||
|
|
||
| .. code-block:: python | ||
| from affine import Affine | ||
| from rioxarray import Convention | ||
| # Write CRS using Zarr conventions | ||
| data = data.rio.write_crs("EPSG:4326", convention=Convention.Zarr) | ||
| # Write transform using Zarr conventions | ||
| transform = Affine(1.0, 0.0, 0.0, 0.0, -1.0, 100.0) | ||
| data = data.rio.write_transform(transform, convention=Convention.Zarr) | ||
| # Results in: | ||
| # - proj:wkt2: CRS as WKT2 string | ||
| # - spatial:transform: [1.0, 0.0, 0.0, 0.0, -1.0, 100.0] | ||
| # - spatial:dimensions: ["y", "x"] | ||
| # - spatial:shape: [height, width] | ||
| # - zarr_conventions: Convention declarations | ||
| Reading Behavior | ||
| ---------------- | ||
|
|
||
| When reading geospatial metadata, rioxarray follows this priority order based on the global convention setting: | ||
|
|
||
| - **None (default)**: CF conventions first, with Zarr conventions as fallback if explicitly declared | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would like both conventions attempted when reading files regardless of the convention setting. The convention setting changing the search priority is fine. This way you can read in files from either format at the same time with the convention setting set. |
||
| - **Convention.CF**: CF conventions only (grid_mapping coordinates and CF attributes) | ||
| - **Convention.Zarr**: Zarr conventions only (spatial: and proj: attributes) | ||
|
|
||
| The fallback behavior ensures that CF remains the primary convention while allowing Zarr conventions to be read when they are the only available metadata. | ||
|
|
||
| Convention Declaration | ||
| ---------------------- | ||
|
|
||
| According to the `Zarr conventions specification <https://github.com/zarr-conventions/zarr-conventions-spec>`, conventions must be explicitly declared in the ``zarr_conventions`` array. rioxarray automatically handles this when writing Zarr conventions: | ||
|
|
||
| .. code-block:: python | ||
| data_zarr = data.rio.write_crs("EPSG:4326", convention=Convention.Zarr) | ||
| # Automatically adds to zarr_conventions: | ||
| print(data_zarr.attrs["zarr_conventions"]) | ||
| # [{"name": "proj:", "uuid": "f17cb550-5864-4468-aeb7-f3180cfb622f", ...}] | ||
| References | ||
| ---------- | ||
|
|
||
| - `CF Conventions <https://github.com/cf-convention/cf-conventions>`_ | ||
| - `Zarr Spatial Convention <https://github.com/zarr-conventions/spatial>`_ | ||
| - `Zarr Geo-Proj Convention <https://github.com/zarr-experimental/geo-proj>`_ | ||
| - `Zarr Conventions Specification <https://github.com/zarr-conventions/zarr-conventions-spec>`_ | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't modify this in this PR.