Fix reductions on empty dimensions returning NaN/NaT instead of raising - #11555
Fix reductions on empty dimensions returning NaN/NaT instead of raising#11555karlhillx wants to merge 1 commit into
Conversation
Several xarray reduction methods raised on empty input (size-0 axis):
- numpy 'max', 'min', 'median', 'std', 'var' raise ValueError
'zero-size array to reduction operation fmax which has no identity'
- 'mean' on datetime64/timedelta64 raises because internal offset
computation hits np.min/np.max on empty float-converted array
- 'median' on datetime64 raises UFuncTypeError for the same reason
Bring behaviour in line with pandas, which returns NaN for numeric and
NaT for datetime/timedelta on empty reductions.
Adds _empty_axis_fill_value to compute the post-reduction shape and the
appropriate fill value (NaT preserves datetime dtype, NaN with float64
promotion preserves other numeric dtype). Inserts a values.size == 0
short-circuit in _create_nan_agg_method so max/min/median/std/var/mean/
sum/prod all return the fill value uniformly. Also short-circuits
_datetime_nanreduce on empty input so the public mean() path yields
NaT consistently.
Closes pydata#11554
There was a problem hiding this comment.
🟡 Changes recommended
The new empty-input short-circuit currently risks changing semantics for non-target ops (e.g., sum/prod identities and cumsum/cumprod behavior) and needs keepdims/backend-preservation fixes to avoid incorrect shapes/types.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates xarray’s duck-array reduction helpers so reductions over an empty dimension return appropriate fill values (NaN / NaT) instead of raising, aligning behavior with pandas and existing xarray reductions.
Changes:
- Add an empty-input short-circuit to reduction dispatch in
xarray/core/duck_array_ops.py. - Add regression tests for empty reductions (numeric + datetime) in
xarray/tests/test_duck_array_ops.py. - Document the user-visible change in
doc/whats-new.rst.
File summaries
| File | Description |
|---|---|
xarray/core/duck_array_ops.py |
Adds empty-input handling for reductions and datetime nan-reductions. |
xarray/tests/test_duck_array_ops.py |
Adds tests asserting NaN/NaT results for empty reductions. |
doc/whats-new.rst |
Notes the behavior change in the release notes. |
Review details
Suppressed comments (2)
xarray/core/duck_array_ops.py:534
- _empty_axis_fill_value currently always uses NumPy broadcast_to and returns a NumPy array even when the input is a non-NumPy duck array (e.g., CuPy). That breaks backend preservation for empty reductions.
dtype = values.dtype
if dtypes.is_datetime_like(dtype):
scalar = np.array("NaT", dtype=dtype)
elif np.issubdtype(dtype, np.floating) or np.issubdtype(dtype, np.complexfloating):
scalar = np.array(np.nan, dtype=dtype)
xarray/core/duck_array_ops.py:564
- The empty-input short-circuit currently applies to every _create_nan_agg_method user, which would change semantics for operations with identities or non-reduction semantics (e.g., sum/prod should return 0/1; cumsum/cumprod should return an empty array; argmin/argmax should still raise). Limit the short-circuit to the reductions that are supposed to return NaN/NaT, and pass keepdims through when requested.
# Handle reductions over an empty axis. Several numpy reductions
# raise on zero-size input (e.g. max, min, median, std, var);
# xarray instead returns the appropriate fill value to match
# pandas. See GH #11554.
if values.size == 0:
return _empty_axis_fill_value(values, axis)
- Files reviewed: 3/3 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| def _empty_axis_fill_value(values, axis): | ||
| """Return a fill-value array for a reduction over an empty axis. | ||
|
|
||
| For datetime/timedelta dtypes the result preserves dtype with a NaT | ||
| fill. For floating/complex dtypes the result preserves dtype with a | ||
| NaN fill. For integer (and other numeric) dtypes the result is | ||
| float64 with a NaN fill, matching pandas / NumPy mean-on-empty | ||
| behaviour. | ||
|
|
||
| See https://github.com/pydata/xarray/issues/11554. | ||
| """ | ||
| if axis is None: | ||
| out_shape: tuple[int, ...] = () | ||
| elif isinstance(axis, (tuple, list)): | ||
| axes = tuple(a if a >= 0 else values.ndim + a for a in axis) | ||
| out_shape = tuple(s for i, s in enumerate(values.shape) if i not in axes) | ||
| else: | ||
| a = axis if axis >= 0 else values.ndim + axis | ||
| out_shape = tuple(s for i, s in enumerate(values.shape) if i != a) | ||
|
|
| :py:meth:`DataArray.var`, :py:meth:`DataArray.max` and :py:meth:`DataArray.min` | ||
| no longer raise when reducing over an empty dimension. They now return ``NaN`` | ||
| for numeric dtypes and ``NaT`` for ``datetime64`` / ``timedelta64`` dtypes, | ||
| matching :py:func:`pandas.Series` behaviour (:issue:`11554`). |
| for f in (duck_array_ops.max, duck_array_ops.min): | ||
| assert np.isnan(f(empty_f8)) | ||
| assert np.isnan(f(empty_i8)) | ||
|
|
Description
Closes #11554
max,min, and datetimemean/medianraise on an empty dimension instead of returning the fill value that the other reductions already return (mean/median/std/varon numeric data returnNaN;timedelta64medianreturnsNaT).The root cause is that the reduction helpers in
duck_array_ops.pynever short-circuit on empty input, so they fall through to the underlying numpy reduction and raise.The fix adds a short-circuit for empty arrays in two places:
_create_nan_agg_method— afterasarray+coerce_strings, before the reduction call, whenvalues.size == 0. This coversmax,min,median,std,var,mean,sum, andproduniformly. A new helper_empty_axis_fill_value(values, axis)returns the correctly-shaped fill array:NaNfor numeric,NaNpromoted tofloat64for integer, and dtype-preservingNaTfor datetime/timedelta. It computes the output shape correctly foraxis=None, a scalar axis, and a tuple/list axis._datetime_nanreduce— returnsNaTwith the original dtype for empty input, keeping the publicmean()path consistent (datetimemeanroutes throughmin/maxoffset →datetime_to_numeric→_mean; with both short-circuits it producesNaTend-to-end).The TLE/
twoline2rvpath is untouched.sumandprodalready have identities and are unaffected.Before / after
Scope note
Cftime object-dtype empty arrays still return
None(existing behaviour, untouched). The issue scopes to datetimemean/median, and cftime has noNaTequivalent — leaving that for a follow-up rather than expanding review burden.Checklist
max,minand datetimemean/medianraise on an empty dimension #11554whats-new.rstAI Disclosure
Tools: OpenClaw (Claude/Codex-backed coding agent). I iterated with the agent to produce the implementation, then read and validated every line and ran the full test suite myself.