Bug report

Bug description:

zipfile decodes a Zstandard (method 93) member with a single compression.zstd.ZstdDecompressor, and ZipExtFile._read1() treats the decompressor's eof as the end of the member. A zstd stream may consist of several concatenated frames (RFC 8878, section 3.1), and multi-threaded or streaming compressors (zstd -T0, pzstd, any writer that flushes frames) produce exactly that. For such a member zipfile returns only the first frame's output, and the CRC check then raises BadZipFile. zstd.decompress() and ZstdFile both handle the same bytes correctly.

import io
import struct
import zipfile
import zlib
from compression import zstd

part1 = b"first frame " * 1000
part2 = b"second frame " * 1000
payload = part1 + part2
member_data = zstd.compress(part1) + zstd.compress(part2)  # two frames
assert zstd.decompress(member_data) == payload

# Store the two-frame stream, then relabel the member as method 93 with the
# CRC and size of the decompressed payload.
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
    zf.writestr(zipfile.ZipInfo("data.bin"), member_data)  # ZIP_STORED
raw = bytearray(buf.getvalue())
crc = zlib.crc32(payload)
cd = raw.rindex(b"PK\x01\x02")
for method, crc_off, size_off in ((8, 14, 22), (cd + 10, cd + 16, cd + 24)):
    struct.pack_into("<H", raw, method, zipfile.ZIP_ZSTANDARD)
    struct.pack_into("<L", raw, crc_off, crc)
    struct.pack_into("<L", raw, size_off, len(payload))

with zipfile.ZipFile(io.BytesIO(bytes(raw))) as zf:
    print(zf.testzip())   # 'data.bin'
    zf.read("data.bin")   # zipfile.BadZipFile: Bad CRC-32 for file 'data.bin'

Output on 3.14.4, 3.15.0a8 and main (19270d3):

data.bin
Traceback (most recent call last):
  ...
zipfile.BadZipFile: Bad CRC-32 for file 'data.bin'

We hit this with real archives whose members were written by a multi-threaded zstd encoder, and currently decode method-93 members with our own frame loop instead of zipfile.

Proposed fix: in ZipExtFile._read1(), when the zstd decompressor reaches the end of a frame and the member still has compressed input (unused_data, or _compress_left > 0), continue with a new ZstdDecompressor. Bytes that do not parse as a frame then raise ZstdError, and a truncated last frame still fails the CRC check. I have a PR with tests ready.

One question for maintainers: once the declared file_size has been produced, _read1() stops reading, so bytes after the last frame inside the member's compress_size are never examined. That matches how the other methods behave today (a deflate stream followed by junk is also accepted), so the PR keeps it. Should trailing data inside the member be an error for zstd, or for all methods? Happy to follow up either way.

Disclosure: this issue was drafted by Claude (Anthropic's AI assistant, via Claude Code) on behalf of @HipHoff, who approved filing it and takes responsibility for it. The repro above was run on CPython 3.14.4, 3.15.0a8 and a local debug build of main.

CPython versions tested on:

3.14, 3.15, CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-160842: Read every frame of a multi-frame Zstandard zipfile member #160843