fix(arrow/array): validate union JSON type codes - #1157
Conversation
48269c3 to
af8b157
Compare
zeroshade
left a comment
There was a problem hiding this comment.
Found two blocking validation/test issues: default-decoder entry points can accept invalid type-code lexemes after float64 rounding, and the regression test converts the panic it is meant to prevent into a passing error assertion. Local package and stdlib-JSON union tests otherwise pass.
This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the points above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think one of the findings is misapplied, please reply on the PR and a maintainer will weigh in.
More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.
| if err != nil { | ||
| return 0, err | ||
| } | ||
| case float64: |
There was a problem hiding this comment.
Blocking: Default-decoder entry points validate the already-rounded float64, so invalid numeric type codes can become valid before this check. For both dense and sparse builders, AppendValueFromString("[-1e-400, 1]") accepts the mathematically negative value as code 0, and [127.00000000000000001, 1] is accepted as code 127. UnmarshalJSON rejects the same inputs because it enables UseNumber, so the public entry points are inconsistent. Please preserve the numeric lexeme before validation across AppendValueFromString/UnmarshalOne as well, and add regression coverage for underflow and rounded fractional values.
| builder := tc.new() | ||
| err := func() (err error) { | ||
| defer func() { | ||
| if r := recover(); r != nil { |
There was a problem hiding this comment.
Blocking: This recovery wrapper converts the panic being guarded against into an error, after which assert.Error passes. I verified that deleting both new bounds checks still leaves this test green: the 127 cases panic during indexing, are converted here, and satisfy the assertion. Please let unexpected panics fail naturally—or assert no panic separately—then assert that decoding returned an error.
af8b157 to
938f961
Compare
zeroshade
left a comment
There was a problem hiding this comment.
The two original blockers are fixed: all decoder entry points now reject rounded invalid type-code lexemes, and unexpected panics are no longer converted into passing error assertions. One new decoder-state blocker remains.
This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the point above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think the finding is misapplied, please reply on the PR and a maintainer will weigh in.
More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.
| } | ||
|
|
||
| func (b *SparseUnionBuilder) UnmarshalOne(dec *json.Decoder) error { | ||
| dec.UseNumber() |
There was a problem hiding this comment.
Blocking: UnmarshalOne receives a caller-owned decoder, and UseNumber permanently changes how every later value from that decoder is interpreted. This makes enclosing objects key-order-dependent: for a struct with union field u and int32 field i, {"i":1.5,"u":[0,1]} succeeds with a default decoder, while the equivalent {"u":[0,1],"i":1.5} fails because decoding u switches the shared decoder before i is read. The dense implementation has the same issue. Please preserve the type-code lexeme without reconfiguring the supplied decoder (for example, decode that token directly into json.Number or RawMessage); only internally created decoders should call UseNumber.
zeroshade
left a comment
There was a problem hiding this comment.
Reviewed at f0d1a61. The union decoder now preserves caller-owned decoder configuration while validating the original type-code lexeme. Dense/sparse behavior, maximum type code handling, and dynamic-child rollback passed repeated tests under both JSON backends, the full default and race suites, formatting, vet, and diff hygiene.
What
Validate union type IDs before narrowing them to int8 in both sparse and dense JSON decoders. Out-of-range, negative, and nonnumeric type IDs are rejected.
Test