Skip to content

Fix reductions on empty dimensions returning NaN/NaT instead of raising - #11555

Open
karlhillx wants to merge 1 commit into
pydata:mainfrom
karlhillx:xarray-issue-11554
Open

Fix reductions on empty dimensions returning NaN/NaT instead of raising#11555
karlhillx wants to merge 1 commit into
pydata:mainfrom
karlhillx:xarray-issue-11554

Conversation

@karlhillx

Copy link
Copy Markdown

Description

Closes #11554

max, min, and datetime mean/median raise on an empty dimension instead of returning the fill value that the other reductions already return (mean/median/std/var on numeric data return NaN; timedelta64 median returns NaT).

The root cause is that the reduction helpers in duck_array_ops.py never short-circuit on empty input, so they fall through to the underlying numpy reduction and raise.

The fix adds a short-circuit for empty arrays in two places:

  1. _create_nan_agg_method — after asarray + coerce_strings, before the reduction call, when values.size == 0. This covers max, min, median, std, var, mean, sum, and prod uniformly. A new helper _empty_axis_fill_value(values, axis) returns the correctly-shaped fill array: NaN for numeric, NaN promoted to float64 for integer, and dtype-preserving NaT for datetime/timedelta. It computes the output shape correctly for axis=None, a scalar axis, and a tuple/list axis.

  2. _datetime_nanreduce — returns NaT with the original dtype for empty input, keeping the public mean() path consistent (datetime mean routes through min/max offset → datetime_to_numeric_mean; with both short-circuits it produces NaT end-to-end).

The TLE/twoline2rv path is untouched. sum and prod already have identities and are unaffected.

Before / after

[float64]    mean=nan, median=nan, max=nan, min=nan, std=nan, var=nan
[int64]      mean=nan, median=nan, max=nan, min=nan, std=nan, var=nan
[datetime64] mean=NaT, median=NaT, max=NaT, min=NaT
[timedelta64] mean=NaT, median=NaT, max=NaT, min=NaT

Scope note

Cftime object-dtype empty arrays still return None (existing behaviour, untouched). The issue scopes to datetime mean/median, and cftime has no NaT equivalent — leaving that for a follow-up rather than expanding review burden.

Checklist

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested any AI-generated content in my PR.
    • I take responsibility for any AI-generated content in my PR.
      Tools: OpenClaw (Claude/Codex-backed coding agent). I iterated with the agent to produce the implementation, then read and validated every line and ran the full test suite myself.

Several xarray reduction methods raised on empty input (size-0 axis):
  - numpy 'max', 'min', 'median', 'std', 'var' raise ValueError
    'zero-size array to reduction operation fmax which has no identity'
  - 'mean' on datetime64/timedelta64 raises because internal offset
    computation hits np.min/np.max on empty float-converted array
  - 'median' on datetime64 raises UFuncTypeError for the same reason

Bring behaviour in line with pandas, which returns NaN for numeric and
NaT for datetime/timedelta on empty reductions.

Adds _empty_axis_fill_value to compute the post-reduction shape and the
appropriate fill value (NaT preserves datetime dtype, NaN with float64
promotion preserves other numeric dtype). Inserts a values.size == 0
short-circuit in _create_nan_agg_method so max/min/median/std/var/mean/
sum/prod all return the fill value uniformly. Also short-circuits
_datetime_nanreduce on empty input so the public mean() path yields
NaT consistently.

Closes pydata#11554
Copilot AI lite review requested due to automatic review settings September 1, 2026 22:18
@github-actions github-actions Bot added the topic-arrays related to flexible array support label Sep 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new empty-input short-circuit currently risks changing semantics for non-target ops (e.g., sum/prod identities and cumsum/cumprod behavior) and needs keepdims/backend-preservation fixes to avoid incorrect shapes/types.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates xarray’s duck-array reduction helpers so reductions over an empty dimension return appropriate fill values (NaN / NaT) instead of raising, aligning behavior with pandas and existing xarray reductions.

Changes:

  • Add an empty-input short-circuit to reduction dispatch in xarray/core/duck_array_ops.py.
  • Add regression tests for empty reductions (numeric + datetime) in xarray/tests/test_duck_array_ops.py.
  • Document the user-visible change in doc/whats-new.rst.
File summaries
File Description
xarray/core/duck_array_ops.py Adds empty-input handling for reductions and datetime nan-reductions.
xarray/tests/test_duck_array_ops.py Adds tests asserting NaN/NaT results for empty reductions.
doc/whats-new.rst Notes the behavior change in the release notes.
Review details

Suppressed comments (2)

xarray/core/duck_array_ops.py:534

  • _empty_axis_fill_value currently always uses NumPy broadcast_to and returns a NumPy array even when the input is a non-NumPy duck array (e.g., CuPy). That breaks backend preservation for empty reductions.
    dtype = values.dtype
    if dtypes.is_datetime_like(dtype):
        scalar = np.array("NaT", dtype=dtype)
    elif np.issubdtype(dtype, np.floating) or np.issubdtype(dtype, np.complexfloating):
        scalar = np.array(np.nan, dtype=dtype)

xarray/core/duck_array_ops.py:564

  • The empty-input short-circuit currently applies to every _create_nan_agg_method user, which would change semantics for operations with identities or non-reduction semantics (e.g., sum/prod should return 0/1; cumsum/cumprod should return an empty array; argmin/argmax should still raise). Limit the short-circuit to the reductions that are supposed to return NaN/NaT, and pass keepdims through when requested.
        # Handle reductions over an empty axis. Several numpy reductions
        # raise on zero-size input (e.g. max, min, median, std, var);
        # xarray instead returns the appropriate fill value to match
        # pandas. See GH #11554.
        if values.size == 0:
            return _empty_axis_fill_value(values, axis)
  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +510 to +529
def _empty_axis_fill_value(values, axis):
"""Return a fill-value array for a reduction over an empty axis.

For datetime/timedelta dtypes the result preserves dtype with a NaT
fill. For floating/complex dtypes the result preserves dtype with a
NaN fill. For integer (and other numeric) dtypes the result is
float64 with a NaN fill, matching pandas / NumPy mean-on-empty
behaviour.

See https://github.com/pydata/xarray/issues/11554.
"""
if axis is None:
out_shape: tuple[int, ...] = ()
elif isinstance(axis, (tuple, list)):
axes = tuple(a if a >= 0 else values.ndim + a for a in axis)
out_shape = tuple(s for i, s in enumerate(values.shape) if i not in axes)
else:
a = axis if axis >= 0 else values.ndim + axis
out_shape = tuple(s for i, s in enumerate(values.shape) if i != a)

Comment thread doc/whats-new.rst
:py:meth:`DataArray.var`, :py:meth:`DataArray.max` and :py:meth:`DataArray.min`
no longer raise when reducing over an empty dimension. They now return ``NaN``
for numeric dtypes and ``NaT`` for ``datetime64`` / ``timedelta64`` dtypes,
matching :py:func:`pandas.Series` behaviour (:issue:`11554`).
Comment on lines +272 to +275
for f in (duck_array_ops.max, duck_array_ops.min):
assert np.isnan(f(empty_f8))
assert np.isnan(f(empty_i8))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic-arrays related to flexible array support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

max, min and datetime mean/median raise on an empty dimension

2 participants