Skip to content

perf: Optimize NULL handling in StringViewArrayBuilder#21538

Open
neilconway wants to merge 2 commits intoapache:mainfrom
neilconway:neilc/perf-string-view-builder-nulls
Open

perf: Optimize NULL handling in StringViewArrayBuilder#21538
neilconway wants to merge 2 commits intoapache:mainfrom
neilconway:neilc/perf-string-view-builder-nulls

Conversation

@neilconway
Copy link
Copy Markdown
Contributor

@neilconway neilconway commented Apr 10, 2026

Which issue does this PR close?

Rationale for this change

StringViewArrayBuilder is implemented on top of Arrow's StringViewBuilder; the latter tracks NULLs incrementally. However, the StringViewArrayBuilder requires callers to pass a NULL buffer to finish() anyway, so the NULL bitmap that has been computed by StringViewBuilder is discarded. It would be more efficient to stop using StringViewBuilder so that we don't do this redundant work; in theory there might be room for inconsistency between the two NULL bitmaps as well.

Right now, StringViewArrayBuilder is only used by the concat and concat_ws UDFs, but I'd like to generalize the API and use it more broadly in place of StringViewBuilder (#21539). For the time being, here are the results of this PR on the concat benchmarks (Arm64):

  - 1024 rows: 29.6 µs → 28.0 µs, -5.3%
  - 4096 rows: 134.3 µs → 125.6 µs, -6.5%
  - 8192 rows: 289.7 µs → 273.5 µs, -5.6%

What changes are included in this PR?

  • Stop using StringViewBuilder and build the views ourselves
  • Improve some comments

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@github-actions github-actions bot added the functions Changes to functions implementation label Apr 10, 2026
@neilconway
Copy link
Copy Markdown
Contributor Author

FYI @Omega359

Copy link
Copy Markdown
Contributor

@Omega359 Omega359 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked this over and it seems reasonable, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize NULL handling in StringViewArrayBuilder

2 participants