check pledged src size at end of multithreaded frame - #4773
Open
avionicharshit-byte wants to merge 1 commit into
Open
check pledged src size at end of multithreaded frame#4773avionicharshit-byte wants to merge 1 commit into
avionicharshit-byte wants to merge 1 commit into
Conversation
avionicharshit-byte
force-pushed
the
mt-check-pledged-srcsize
branch
from
September 8, 2026 18:30
9ff2c12 to
233c7e3
Compare
avionicharshit-byte
marked this pull request as ready for review
September 8, 2026 18:32
Author
|
@Cyan4973 small one: the MT path never validates pledgedSrcSize at end of frame , so |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
with
nbWorkers >= 1and an input spanning more than one job , a wrong pledged source size is never checked. the frame header carries the pledged size , compression returns success , and decompression fails later withData corruption detected. both directions , too large and too small. the single-thread path has always caught this inZSTD_compressEnd_public.repro on current
dev:#1095 set the rule that the pledge is validated whenever it is written into the frame header , and #1106 enforced it "whenever known" , but only in
ZSTD_compressContinue_internalandZSTD_compressEnd. the MT branch ofZSTD_compressStream2was not part of it. this applies the same check there: at end of frame , comparepledgedSrcSizePlusOnewithconsumedSrcSize + 1and returnsrcSize_wrong. both counters already exist , no new state.tests:
tests/cli-tests/compression/mt-stream-size.sh(over , under , exact , none with-T2 -B512K) and three cases intests/zstreamtest.cwithnbWorkers=2and a 1 MB job size , streamed withZSTD_e_continueso the first-callZSTD_e_endoverride does not mask it. both fail on unfixeddev.make -C tests testpasses.one caveat: the error fires after the frame bytes have been handed to the caller , same as the single-thread path today.