Skip to content

fix(deletion-vector): reject a bitmap count larger than the payload - #3992

Open
ghoshp83 wants to merge 1 commit into
apache:mainfrom
ghoshp83:dv-bitmap-count-bound
Open

ghoshp83 wants to merge 1 commit into
apache:mainfrom
ghoshp83:dv-bitmap-count-bound

Conversation

@ghoshp83

Copy link
Copy Markdown

Closes #3979

Rationale for this change

_deserialize_bitmap read an 8-byte count from the start of the payload and used it directly as a loop bound, without relating it to how much payload actually follows. A small blob could therefore declare a very large number of bitmaps.

The count cannot be trusted on its own, but it can be checked against the payload: every bitmap contributes at least a 4-byte key, so a count larger than len(payload) // 4 cannot be satisfied by the bytes present. This rejects such a payload up front with a ValueError instead of letting the loop run off the end of the buffer, which surfaced as IndexError: Out of bounds on buffer access (axis 0) from the native bitmap decoder.

Per SECURITY-THREAT-MODEL.md this is parser hardening (§7) rather than a vulnerability, so it is filed as an ordinary bug fix.

Are these changes tested?

Yes. tests/table/test_deletion_vector.py gains a case for a payload that declares more bitmaps than its bytes can hold. It fails on main with the IndexError above and passes with this change; the existing deletion-vector tests are unaffected.

Are there any user-facing changes?

A deletion vector whose declared bitmap count exceeds its payload now raises ValueError instead of IndexError. Valid deletion vectors are unaffected.

One related case is deliberately left out of this PR to keep it to a single concern: a single large key still drives the empty-bitmap fill loop up to MAX_JAVA_SIGNED appends, which is the same shape of problem reached through a different field. Happy to follow up on that separately if you would like it addressed.

_deserialize_bitmap read an 8-byte count and used it directly as a loop
bound, so a small blob could declare a large number of bitmaps. Reading
past the payload surfaced as an IndexError from the native bitmap
decoder. Validate the count against the remaining bytes, since every
bitmap contributes at least a 4-byte key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deletion vector bitmap count is read from the blob and used as a loop bound without validation

1 participant