Skip to content

fix(arrow/array): validate union JSON type codes - #1157

Merged
zeroshade merged 6 commits into
apache:mainfrom
fallintoplace:fix/array-union-json-type-codes
Aug 26, 2026
Merged

fix(arrow/array): validate union JSON type codes#1157
zeroshade merged 6 commits into
apache:mainfrom
fallintoplace:fix/array-union-json-type-codes

Conversation

@fallintoplace

Copy link
Copy Markdown
Contributor

What

Validate union type IDs before narrowing them to int8 in both sparse and dense JSON decoders. Out-of-range, negative, and nonnumeric type IDs are rejected.

Test

  • go test ./arrow/array -run ^TestUnion -count=1

@fallintoplace
fallintoplace force-pushed the fix/array-union-json-type-codes branch from 48269c3 to af8b157 Compare August 23, 2026 22:00
@fallintoplace
fallintoplace marked this pull request as ready for review August 23, 2026 22:00

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found two blocking validation/test issues: default-decoder entry points can accept invalid type-code lexemes after float64 rounding, and the regression test converts the panic it is meant to prevent into a passing error assertion. Local package and stdlib-JSON union tests otherwise pass.


This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the points above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think one of the findings is misapplied, please reply on the PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

Comment thread arrow/array/union.go Outdated
if err != nil {
return 0, err
}
case float64:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: Default-decoder entry points validate the already-rounded float64, so invalid numeric type codes can become valid before this check. For both dense and sparse builders, AppendValueFromString("[-1e-400, 1]") accepts the mathematically negative value as code 0, and [127.00000000000000001, 1] is accepted as code 127. UnmarshalJSON rejects the same inputs because it enables UseNumber, so the public entry points are inconsistent. Please preserve the numeric lexeme before validation across AppendValueFromString/UnmarshalOne as well, and add regression coverage for underflow and rounded fractional values.

Comment thread arrow/array/union_test.go Outdated
builder := tc.new()
err := func() (err error) {
defer func() {
if r := recover(); r != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: This recovery wrapper converts the panic being guarded against into an error, after which assert.Error passes. I verified that deleting both new bounds checks still leaves this test green: the 127 cases panic during indexing, are converted here, and satisfy the assertion. Please let unexpected panics fail naturally—or assert no panic separately—then assert that decoding returned an error.

@fallintoplace
fallintoplace force-pushed the fix/array-union-json-type-codes branch from af8b157 to 938f961 Compare August 25, 2026 20:09

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two original blockers are fixed: all decoder entry points now reject rounded invalid type-code lexemes, and unexpected panics are no longer converted into passing error assertions. One new decoder-state blocker remains.


This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. After you've addressed the point above and pushed an update, an Apache Arrow Go maintainer — a real person — will take the next look at the PR. If you think the finding is misapplied, please reply on the PR and a maintainer will weigh in.

More on how Apache Arrow Go handles maintainer review:
CONTRIBUTING.md.

Comment thread arrow/array/union.go Outdated
}

func (b *SparseUnionBuilder) UnmarshalOne(dec *json.Decoder) error {
dec.UseNumber()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: UnmarshalOne receives a caller-owned decoder, and UseNumber permanently changes how every later value from that decoder is interpreted. This makes enclosing objects key-order-dependent: for a struct with union field u and int32 field i, {"i":1.5,"u":[0,1]} succeeds with a default decoder, while the equivalent {"u":[0,1],"i":1.5} fails because decoding u switches the shared decoder before i is read. The dense implementation has the same issue. Please preserve the type-code lexeme without reconfiguring the supplied decoder (for example, decode that token directly into json.Number or RawMessage); only internally created decoders should call UseNumber.

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at f0d1a61. The union decoder now preserves caller-owned decoder configuration while validating the original type-code lexeme. Dense/sparse behavior, maximum type code handling, and dynamic-child rollback passed repeated tests under both JSON backends, the full default and race suites, formatting, vet, and diff hygiene.

@zeroshade
zeroshade merged commit 227f6e3 into apache:main Aug 26, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants