Skip to content

[C++][Compute] Vector replace kernels (replace_with_mask, fill_null) do not support string_view, binary_view, decimal32, or decimal64 #55256

Open

Description

@Awesome06

Describe the bug, including details regarding any error messages, version, and platform.

The vector replacement compute functions (replace_with_mask, fill_null_forward, and fill_null_backward) fail with arrow::Status::NotImplemented when invoked with:

  • utf8_view (string_view) or binary_view
  • Modern 32-bit and 64-bit decimal types (decimal32, decimal64)

While classic variable-length binary types (utf8, large_utf8, binary, large_binary), primitives, and Decimal128 / Decimal256 are supported, modern view types and small decimals are omitted from the kernel registration table.

In cpp/src/arrow/compute/kernels/vector_replace.cc, RegisterVectorFunction registers fixed-size binary, legacy decimals, and BaseBinaryTypes():

AddKernel(Type::FIXED_SIZE_BINARY,
          Functor<FixedSizeBinaryType>::GetSignature(Type::FIXED_SIZE_BINARY),
          Functor<FixedSizeBinaryType>::Exec, ChunkedFunctor<FixedSizeBinaryType>::Exec,
          registry, func.get());
AddKernel(Type::DECIMAL128,
          Functor<FixedSizeBinaryType>::GetSignature(Type::DECIMAL128),
          Functor<FixedSizeBinaryType>::Exec, ChunkedFunctor<FixedSizeBinaryType>::Exec,
          registry, func.get());
AddKernel(Type::DECIMAL256,
          Functor<FixedSizeBinaryType>::GetSignature(Type::DECIMAL256),
          Functor<FixedSizeBinaryType>::Exec, ChunkedFunctor<FixedSizeBinaryType>::Exec,
          registry, func.get());
for (const auto& ty : BaseBinaryTypes()) {
  AddKernel(ty->id(), Functor<FixedSizeBinaryType>::GetSignature(ty->id()),
            GenerateTypeAgnosticVarBinaryBase<Functor>(*ty),
            GenerateTypeAgnosticVarBinaryBase<ChunkedFunctor>(*ty), registry,
            func.get());
}

Because replace_with_mask, fill_null_forward, and fill_null_backward all instantiate kernels via this template helper, none of them support:

  1. BinaryViewType or StringViewType (BaseBinaryTypes() only covers binary, large_binary, utf8, large_utf8).
  2. Decimal32Type or Decimal64Type (only DECIMAL128 and DECIMAL256 are registered).

Reproduction

The issue reproduces in C++ (and PyArrow):

TEST(TestVectorReplace, StringViewAndSmallDecimals) {
  // 1. replace_with_mask on utf8_view
  auto view_arr = ArrayFromJSON(utf8_view(), "[\"apple\", null, \"cherry\"]");
  auto mask = ArrayFromJSON(boolean(), "[false, true, false]");
  auto replacements = ArrayFromJSON(utf8_view(), "[\"banana\"]");

  auto res1 = arrow::compute::ReplaceWithMask(view_arr, mask, replacements);
  ASSERT_OK(res1.status());

  // 2. fill_null_forward on utf8_view
  auto res2 = arrow::compute::FillNullForward(view_arr);
  ASSERT_OK(res2.status());

  // 3. replace_with_mask on decimal32
  auto dec32_type = decimal32(7, 2);
  auto dec_arr = ArrayFromJSON(dec32_type, "[\"12.34\", null, \"56.78\"]");
  auto dec_repl = ArrayFromJSON(dec32_type, "[\"99.99\"]");

  auto res3 = arrow::compute::ReplaceWithMask(dec_arr, mask, dec_repl);
  ASSERT_OK(res3.status());
}

Or in Python:

import pyarrow as pa
import pyarrow.compute as pc

# 1. string_view replace_with_mask
views = pa.array(["apple", None, "cherry"], type=pa.string_view())
mask = pa.array([False, True, False])
repl = pa.array(["banana"], type=pa.string_view())

pc.replace_with_mask(views, mask, repl)

# 2. string_view fill_null_forward
pc.fill_null_forward(views)

# 3. decimal32 replace_with_mask
dec_arr = pa.array([1234, None, 5678], type=pa.decimal32(7, 2))
dec_repl = pa.array([9999], type=pa.decimal32(7, 2))
pc.replace_with_mask(dec_arr, mask, dec_repl)

Expected:

  • replace_with_mask(views, mask, repl) returns string_view array ["apple", "banana", "cherry"].
  • fill_null_forward(views) returns string_view array ["apple", "apple", "cherry"].
  • replace_with_mask(dec_arr, mask, dec_repl) returns decimal32(7, 2) array ["12.34", "99.99", "56.78"].

Actual:

ArrowNotImplementedError: Function 'replace_with_mask' has no kernel matching input types (string_view, boolean, string_view)
ArrowNotImplementedError: Function 'fill_null_forward' has no kernel matching input types (string_view)
ArrowNotImplementedError: Function 'fill_null_backward' has no kernel matching input types (string_view)
ArrowNotImplementedError: Function 'replace_with_mask' has no kernel matching input types (decimal32(7, 2), boolean, decimal32(7, 2))

Reproduced on current main across all platforms. This is an unfiled type-matrix parity gap in Arrow C++ compute.

Suggested fix

  1. For decimal32 and decimal64:
    In RegisterVectorFunction in cpp/src/arrow/compute/kernels/vector_replace.cc, register Type::DECIMAL32 and Type::DECIMAL64 matching the existing DECIMAL128 fixed-width pattern:

    AddKernel(Type::DECIMAL32,
              Functor<FixedSizeBinaryType>::GetSignature(Type::DECIMAL32),
              Functor<FixedSizeBinaryType>::Exec, ChunkedFunctor<FixedSizeBinaryType>::Exec,
              registry, func.get());
    AddKernel(Type::DECIMAL64,
              Functor<FixedSizeBinaryType>::GetSignature(Type::DECIMAL64),
              Functor<FixedSizeBinaryType>::Exec, ChunkedFunctor<FixedSizeBinaryType>::Exec,
              registry, func.get());
  2. For string_view and binary_view:
    Add view-aware execution functors to vector_replace.cc and copy_data_internal.h:

    • Copy the 16-byte view descriptors (StringViewType::c_type) during element replacement or propagation.
    • For replace_with_mask, merge the data buffers of the replacement array/scalar into the output array's data->buffers.
    • For fill_null_forward / fill_null_backward, forward the 16-byte descriptors directly and retain the input array's data buffers.
  3. Add Unit Tests:
    Add test coverage in cpp/src/arrow/compute/kernels/vector_replace_test.cc verifying:

    • Sliced view arrays with non-zero offsets.
    • Runs of multiple consecutive nulls with fill_null_forward and fill_null_backward.
    • Array-array and array-scalar replacement permutations for utf8_view, binary_view, decimal32, and decimal64.

Component(s)

C++

Activity

  1. huong8373tt-beep commented on Oct 26, 2026

    @huong8373tt-beep

    I would like to work on this issue, starting with the decimal32/decimal64 registration gap and verifying whether the view-type support can be safely split into a follow-up if it needs separate buffer-handling machinery.

    I will add focused C++ regression coverage for the supported scope, validate the behavior on current main, and open a PR once the change is ready.

    Disclosure: Slai Agent Runtime is assisting with analysis, implementation, testing, and preparation. The account owner authorizes and reviews the work and takes full responsibility for the submission.

  2. Awesome06 commented on Oct 26, 2026

    @Awesome06
    Author

    Hey @huong8373tt-beep, I am actually working on this issue already.

  3. huong8373tt-beep commented on Oct 26, 2026

    @huong8373tt-beep

    Thanks for letting me know. I will not proceed with a competing implementation. Good luck with the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions