Route remote sidecar parse failures through the shared warn/raise policy#3025
Merged
Conversation
discover_remote_sidecar swallowed download/parse failures of a probed .ovr sidecar and fell back to base-only silently. The eager local and fsspec readers already route the same failure through handle_sidecar_parse_failure, which warns on a malformed sidecar and raises (chaining the real parse error) when an explicit overview_level needs the sidecar. Thread overview_level into discover_remote_sidecar and replace the silent except with handle_sidecar_parse_failure so the remote/HTTP dask metadata path matches the local policy. CloudSizeLimitError still propagates unchanged. Wire overview_level through the _cog_http caller.
brendancol
commented
Jun 7, 2026
Contributor
Author
brendancol
left a comment
There was a problem hiding this comment.
PR Review: Route remote sidecar parse failures through the shared warn/raise policy
Blockers (must fix before merge)
None.
Suggestions (should fix, not blocking)
None.
Nits (optional improvements)
xrspatial/geotiff/_cog_http.py:238: three mock HTTP tests (test_read_cog_http_uses_coalesced_fetches,test_read_cog_http_perf_with_mock_rtt,test_dask_http_parses_ifds_once) now emit a RuntimeWarning because their mockread_all()rejects themax_byteskwarg, soload_sidecarraises a TypeError that the new policy surfaces as a warning. The tests still pass and the warning is the intended behavior change. If the noise is unwanted those mocks could accept**kwargs, but that is out of scope here.
What looks good
- The fix reuses the existing
handle_sidecar_parse_failurehelper instead of duplicating the warn/raise logic, so the eager CPU/GPU, fsspec metadata, and remote/HTTP dask metadata paths now share one policy. base_ifd_count=len(base_ifds)matches the gating the local paths use, so a base TIFF with its own internal overviews can still servelevel=1without forcing a raise when the sidecar is corrupt.CloudSizeLimitErrorstill re-raises before the new handler, so the byte-budget contract is preserved.- The release contract (
reader.sidecar_ovrrow) already promised this behavior for the dask metadata path; the code was the outlier, so no doc change is needed. - Tests cover both new branches: warn-and-fall-back for an implicit level, and the chained parse error for an explicit
overview_level=1against a single-IFD base.
Checklist
- Algorithm matches reference/paper: N/A (reader robustness fix, no kernel)
- All implemented backends produce consistent results: N/A (no spatial-op backends touched)
- NaN handling is correct: N/A
- Edge cases are covered by tests: yes
- Dask chunk boundaries handled correctly: N/A (metadata path only)
- No premature materialization or unnecessary copies: yes
- Benchmark exists or is not needed: not needed
- README feature matrix updated: N/A (no new public API)
- Docstrings present and accurate: yes, docstring updated to document the new behavior and the ValueError path
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3022
What changed
discover_remote_sidecarnow routes a probed-but-unreadable.ovrsidecar through the existinghandle_sidecar_parse_failurehelper instead of swallowing the exception. A malformed sidecar warns and falls back to base-only; an explicitoverview_levelonly the sidecar could serve re-raises the parse error (chained), matching the eager local and fsspec readers.overview_levelintodiscover_remote_sidecarand wired it from the_cog_httpmetadata caller.CloudSizeLimitErrorbudget breaches still propagate unchanged.Backend coverage
This touches the GeoTIFF reader's remote/HTTP and dask metadata path only. No spatial-op kernels, so the numpy/cupy/dask matrix does not apply. The fix aligns the dask+HTTP metadata path with the eager CPU/GPU and fsspec paths that already used the shared policy.
Test plan
test_discover_remote_sidecar_falls_back_when_load_failsupdated to assert theRuntimeWarningnow fires on fallback.test_discover_remote_sidecar_surfaces_parse_error_for_sidecar_levelcovers an explicitoverview_level=1against a single-IFD base raising the chained parse error.test_discover_remote_sidecar_propagates_cloud_size_limitstill passes (budget breach re-raises).test_sidecar.py+test_http_sources.pysuites pass (416 passed).