fix(parquet/variant): validate compound value bounds

[fallintoplace]

Rationale for this change

NewWithMetadata accepts truncated arrays and objects because compound values are validated lazily. Accessing one of those values can then panic on a short offset table or an out-of-range child value.

What changes are included in this PR?

Validate compound headers, offset tables, monotonic offsets, and nested value boundaries before returning a value while keeping the existing compound accessors lazy.

Are these changes tested?

• go test ./parquet/variant

Are there any user-facing changes?

Malformed compound variant values now return an error during construction instead of panicking during access.

[zeroshade]

Since this is a draft, i'll hold off on further review until it's marked ready

[zeroshade]

Found two blocking regressions: indexed child values no longer round-trip because the strict trailing-byte check conflicts with untrimmed accessor results, and validation introduces substantial per-child allocation and memory amplification on the per-row read path.

---

This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the points above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think one of the findings is misapplied, please reply on the PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

[zeroshade]

Blocking: The exact-length check rejects byte slices returned by the public indexed accessors. ArrayValue.Value, ObjectValue.ValueByKey, and ObjectValue.FieldAt return an untrimmed slice extending through the parent’s remaining data, unlike the Values iterators. Reproduction: parse {"a":1,"b":2}, retrieve "a" with ValueByKey, then call NewWithMetadata(v.Metadata(), field.Value.Bytes()); this PR returns invalid variant value: trailing bytes. Please trim values returned by all indexed accessors to their encoded size and add round-trip tests for arrays and objects.

[zeroshade]

Blocking: Validation eagerly allocates a validationChild for every child; objects subsequently allocate another validationRange per field. This happens on every NewWithMetadata, including the Parquet row-read path. A direct 40-field benchmark changed from approximately 4.9 ns/op, 0 B/op, 0 allocs/op on main to 1.58 µs/op, 5576 B/op, 10 allocs/op. Wide or deeply nested untrusted values therefore create substantial memory amplification and potential OOM pressure. Please iterate child offsets incrementally or otherwise bound/reuse validator state, and add representative allocation benchmarks.

[zeroshade]

The indexed accessor round-trip issue is fixed, and the new 40-child benchmarks are allocation-free. One memory-amplification blocker remains for nesting beyond the inline stack capacity.

---

This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the point above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think the finding is misapplied, please reply on the PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

[zeroshade]

Blocking: The common 40-child case is now allocation-free, but attacker-controlled nesting still grows this 120-byte frame stack with repeated slice reallocations. A valid 1,000-level nested array occupies 5,873 encoded bytes yet NewWithMetadata allocates about 393 KB; at 10,000 levels validation allocates about 6.2 MB. This leaves the prior deep-input memory-amplification/OOM concern unresolved on the row-read path. Please reduce or bound depth-dependent state and add a benchmark beyond validationStackInlineCapacity.

[zeroshade]

The previous deep-validation memory-amplification blocker is resolved: 1,000-level array and object validation is allocation-free, and the repeated, race, boundary, vet, and fuzz checks passed locally. One correctness blocker remains for valid compounds whose complete encoding exceeds uint32 while their relative field offsets remain representable; details are inline.

---

This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the point above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think the finding is misapplied, please reply on the PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

[zeroshade]

Blocking: This treats dataStart + offset as though the complete encoded value were limited to uint32, but Variant's 4-byte field offsets are relative to the start of the fields region. A valid compound can therefore contain a math.MaxUint32-byte fields region plus its header and offset table. A sparse-mapped array with one max-sized binary child passes on the merge base but fails here with array offset 4294967295 is out of range. The equivalent object condition at line 907 has the same issue. Please retain total positions as uint64 and compare them against len(value) without imposing this undocumented total-value limit.

[zeroshade]

The prior large-compound blocker is resolved. Validation and accessors now preserve compound sizes beyond uint32, with regression coverage for both arrays and objects. I also verified the variant suite under repeated and race runs, fuzzed the parser, cross-compiled for Linux/386 and Windows/amd64, and confirmed the validation benchmarks remain allocation-free.