Skip to content

Route remote sidecar parse failures through the shared warn/raise policy#3025

Merged
brendancol merged 2 commits into
mainfrom
issue-3022
Jun 7, 2026
Merged

Route remote sidecar parse failures through the shared warn/raise policy#3025
brendancol merged 2 commits into
mainfrom
issue-3022

Conversation

@brendancol
Copy link
Copy Markdown
Contributor

Closes #3022

What changed

  • discover_remote_sidecar now routes a probed-but-unreadable .ovr sidecar through the existing handle_sidecar_parse_failure helper instead of swallowing the exception. A malformed sidecar warns and falls back to base-only; an explicit overview_level only the sidecar could serve re-raises the parse error (chained), matching the eager local and fsspec readers.
  • Threaded overview_level into discover_remote_sidecar and wired it from the _cog_http metadata caller.
  • CloudSizeLimitError budget breaches still propagate unchanged.

Backend coverage

This touches the GeoTIFF reader's remote/HTTP and dask metadata path only. No spatial-op kernels, so the numpy/cupy/dask matrix does not apply. The fix aligns the dask+HTTP metadata path with the eager CPU/GPU and fsspec paths that already used the shared policy.

Test plan

  • test_discover_remote_sidecar_falls_back_when_load_fails updated to assert the RuntimeWarning now fires on fallback.
  • New test_discover_remote_sidecar_surfaces_parse_error_for_sidecar_level covers an explicit overview_level=1 against a single-IFD base raising the chained parse error.
  • test_discover_remote_sidecar_propagates_cloud_size_limit still passes (budget breach re-raises).
  • Full test_sidecar.py + test_http_sources.py suites pass (416 passed).

discover_remote_sidecar swallowed download/parse failures of a probed
.ovr sidecar and fell back to base-only silently. The eager local and
fsspec readers already route the same failure through
handle_sidecar_parse_failure, which warns on a malformed sidecar and
raises (chaining the real parse error) when an explicit overview_level
needs the sidecar.

Thread overview_level into discover_remote_sidecar and replace the
silent except with handle_sidecar_parse_failure so the remote/HTTP dask
metadata path matches the local policy. CloudSizeLimitError still
propagates unchanged. Wire overview_level through the _cog_http caller.
@github-actions github-actions Bot added the performance PR touches performance-sensitive code label Jun 7, 2026
Copy link
Copy Markdown
Contributor Author

@brendancol brendancol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: Route remote sidecar parse failures through the shared warn/raise policy

Blockers (must fix before merge)

None.

Suggestions (should fix, not blocking)

None.

Nits (optional improvements)

  • xrspatial/geotiff/_cog_http.py:238: three mock HTTP tests (test_read_cog_http_uses_coalesced_fetches, test_read_cog_http_perf_with_mock_rtt, test_dask_http_parses_ifds_once) now emit a RuntimeWarning because their mock read_all() rejects the max_bytes kwarg, so load_sidecar raises a TypeError that the new policy surfaces as a warning. The tests still pass and the warning is the intended behavior change. If the noise is unwanted those mocks could accept **kwargs, but that is out of scope here.

What looks good

  • The fix reuses the existing handle_sidecar_parse_failure helper instead of duplicating the warn/raise logic, so the eager CPU/GPU, fsspec metadata, and remote/HTTP dask metadata paths now share one policy.
  • base_ifd_count=len(base_ifds) matches the gating the local paths use, so a base TIFF with its own internal overviews can still serve level=1 without forcing a raise when the sidecar is corrupt.
  • CloudSizeLimitError still re-raises before the new handler, so the byte-budget contract is preserved.
  • The release contract (reader.sidecar_ovr row) already promised this behavior for the dask metadata path; the code was the outlier, so no doc change is needed.
  • Tests cover both new branches: warn-and-fall-back for an implicit level, and the chained parse error for an explicit overview_level=1 against a single-IFD base.

Checklist

  • Algorithm matches reference/paper: N/A (reader robustness fix, no kernel)
  • All implemented backends produce consistent results: N/A (no spatial-op backends touched)
  • NaN handling is correct: N/A
  • Edge cases are covered by tests: yes
  • Dask chunk boundaries handled correctly: N/A (metadata path only)
  • No premature materialization or unnecessary copies: yes
  • Benchmark exists or is not needed: not needed
  • README feature matrix updated: N/A (no new public API)
  • Docstrings present and accurate: yes, docstring updated to document the new behavior and the ValueError path

@brendancol brendancol merged commit aadcfe7 into main Jun 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance PR touches performance-sensitive code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remote sidecar parse failures are swallowed instead of warning or surfacing the parse error

1 participant