Repository navigation
Conversation
…member ZipExtFile._read1() treated the end of the first zstd frame as the end of the member. When the decompressor reaches the end of a frame and compressed input remains, continue with a new ZstdDecompressor. Bytes that are not a valid frame raise ZstdError, and a truncated last frame still fails the CRC check. Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016uX6RoHqnajMDyVi3LrJjs
|
The following commit authors need to sign the Contributor License Agreement: |
|
Thank you for your interest in contributing to Python. This is a community-driven project, and we want it to be a welcoming place to learn and collaborate. Contributors are responsible for reviewing and understanding what they submit, checking its claims, and being able to explain changes in their own words. This helps reviewers focus on the contribution and leaves more time to support other contributors. We believe this contribution does not meet our guidelines for using AI tools, so we're closing it. If you think we've misunderstood your contribution, please let us know. You're welcome to explain your approach or ask questions about the guidelines. Please read our guidelines for using AI tools. |
A zstd stream may consist of several concatenated frames (RFC 8878, section 3.1), which multi-threaded and streaming compressors produce.
ZipExtFile._read1()used the decompressor'seofas the end of the member, so it returned only the first frame and then failed the CRC check.When the zstd decompressor reaches the end of a frame and compressed input remains (
unused_data, or_compress_left > 0),_read1()now continues with a newZstdDecompressor. The member ends when the compressed input is exhausted on a frame boundary, or when the declaredfile_sizehas been produced, as before.Failure behaviour:
ZstdError;BadZipFile.The other methods are unchanged. The repack scan in
_scan_data_descriptor_no_sig_by_decompression()still stops at the first frame. For a multi-frame member written with a data descriptor and no signature, the size check there rejects the candidate, so the result is the same as before this PR.Tests:
ZstdMultipleFramesTestsintest_zipfile/test_core.pycovers two frames read in several chunk sizes, 49 small frames, garbage between frames, and a truncated last frame. Without the fix, three of the four fail../python.exe -m test test_zipfilepasses on a macOS debug build of main. That build had no zlib, so the zlib-dependent tests were skipped.The issue asks one question: should trailing bytes after the last frame, inside
compress_size, be an error? This PR leaves that as it is for every method.Disclosure: this PR was authored by Claude (Anthropic's AI assistant, via Claude Code) on behalf of @HipHoff, who takes responsibility for the change. It is a draft until he has reviewed it and signed the CLA.
https://claude.ai/code/session_016uX6RoHqnajMDyVi3LrJjs