perf(arrow/array): compare binary values by valid runs

[fallintoplace]

Rationale for this change

Binary and string equality currently compares every non-null value separately. This repeats offset lookups and small byte or string comparisons even when a long valid run is stored contiguously.

What changes are included in this PR?

• Walk contiguous valid runs for Binary, String, LargeBinary, and LargeString.
• Compare each run payload in one operation.
• Verify every value length so equal concatenated bytes with different boundaries still compare unequal.
• Keep null payloads ignored.
• Sample validity fragmentation and retain the scalar path for many tiny runs.
• Add benchmarks and tests for offsets, null patterns, mismatches, and value boundaries.

Apple M1 Pro results for 65,536 values of 32 bytes with -cpu=1:

Case	Before	After	Speedup
Binary equal	644 us	360 us	1.79x
Binary 10% null	582 us	439 us	1.33x
Binary mismatch last	648 us	350 us	1.85x
Binary different length	453 us	262 us	1.73x
String equal	633 us	390 us	1.62x
String 10% null	604 us	496 us	1.22x
String mismatch last	664 us	406 us	1.64x
String different length	426 us	235 us	1.81x

The alternating 50% null case selects the scalar path and remains neutral. All benchmark cases stay at zero allocations.

Are these changes tested?

Yes.

• go test ./arrow/...
• go test -race ./arrow/array
• go vet -composites=false ./arrow/array
• Cross-compiled array tests for linux/amd64 and linux/s390x.

Are there any user-facing changes?

No.

[zeroshade]

The run-based comparison is correct for valid arrays, and differential testing found no semantic divergence across Binary, String, LargeBinary, LargeString, random null patterns, and independent slices. The claimed large-array improvements also hold.

I am requesting changes for two points noted inline:

1. A malformed IPC/C Data array with a nonzero declared null count and an empty validity buffer now panics while selecting the comparison strategy.
2. The sampling heuristic materially regresses short arrays with fragmented validity, but the submitted benchmark only covers 65,536-element arrays.

The full Arrow suite, array race suite, vet, formatting, and diff checks otherwise pass.

---

This review was drafted by an AI-assisted tool and
confirmed by an Apache Arrow Go maintainer. After you've
addressed the points above and pushed an update, an Apache Arrow Go
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one is mis-applied, please reply on the
PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

[zeroshade]

Blocking: NullN() > 0 does not guarantee that a validity bitmap is present. Malformed IPC or C Data can supply a nonzero declared null count with an empty validity buffer.

This unconditionally constructs a run reader over that empty slice. I reproduced equality on two eight-element Binary arrays with nullCount=1 and an empty validity buffer:

• merge base: returns normally
• this PR: panics in baseSetBitRunReader.loadPartial

An independent end-to-end IPC probe reproduced the same panic after changing only the serialized null_count while leaving the validity buffer absent.

Please check len(values.NullBitmapBytes()) before constructing the reader and retain the scalar fallback when it is empty. Add a regression test confirming equality does not panic on this malformed-input case.

[zeroshade]

The sampling heuristic introduces a substantial regression for short arrays with fragmented validity because it constructs and advances a run reader before ultimately selecting the original scalar loop.

On Apple M4, alternating-null String arrays measured:

Length	Merge base	Current	Regression
4	~27 ns	~68 ns	2.5×
8	~42 ns	~100 ns	2.4×
16	~74 ns	~115 ns	55%
32	~147 ns	~187 ns	27%
64	~282 ns	~315 ns	11%

The submitted benchmark only uses 65,536-element arrays, so it cannot expose this fixed-cost regression. Please add a cheap length cutoff before sampling and include short-array cases in the benchmark matrix. A performance change should not make these common small-array comparisons materially slower.