Skip to content

unnest of a struct ignores the struct's null mask #25293

Description

@neilconway

Describe the bug

Unnesting a struct column treats the child arrays as independent of the struct's own validity, in both the planner and the executor:

  • get_unnested_columns copies each child's declared nullability into the output schema without checking whether the parent struct is nullable.
  • flatten_struct_cols returns struct_arr.columns() verbatim, without applying the struct's null buffer.

To Reproduce

This can result in unnesting values from a "null" struct:

-- One row holding a fully valid struct.
create table base as select named_struct('a', 1, 'b', 'x') as s;

-- nullif on a column goes through arrow's nullif kernel, which only flips the
-- struct's validity bit. The child arrays still hold 1 and 'x'.
create table t2 as select nullif(s, s) as s from base;

select s from t2;
-- NULL

select s is null from t2;
-- true

select unnest(s) from t2;
-- 1 x
select unnest(arrow_cast(NULL, 'Struct("a": non-null Int32)'));

yields:

Arrow error: Invalid argument error: Column '__unnest_placeholder(arrow_cast(NULL,Utf8("Struct("a": non-null Int32)"))).a' is declared as non-nullable but contains null values

Expected behavior

  • Unnesting a NULL struct row yields NULL for every output column.
  • Every output column of a struct unnest is nullable whenever the parent struct is nullable, matching the rule get_field already follows for nested access.

i.e., first query above should return NULL NULL, second query should return NULL instead of an error.

Additional context

Similar to #25120

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions