Skip to content

gh-149945: Fix potential OOM for gzip with large header#149979

Merged
serhiy-storchaka merged 1 commit into
python:mainfrom
serhiy-storchaka:gzip-header-crc-large
May 19, 2026
Merged

gh-149945: Fix potential OOM for gzip with large header#149979
serhiy-storchaka merged 1 commit into
python:mainfrom
serhiy-storchaka:gzip-header-crc-large

Conversation

@serhiy-storchaka
Copy link
Copy Markdown
Member

@serhiy-storchaka serhiy-storchaka commented May 18, 2026

Do not read the whole filename and comment to memory for calculating the CRC.

cc @rhpvorderman, @KowalskiThomas

Do not read the whole filename and comment to memory for calculating the CRC.
@rhpvorderman
Copy link
Copy Markdown
Contributor

I did not foresee this can of worms...

Alternative:

if flag & FHCRC:
    warnings.warn("Header CRC is set but will not be checked by python")

The code looks good to me. Very elegant solution. A long header will take a long time to calculate due to the overhead of reading it byte by byte, but that byte by byte reading is what always has been done so probably not too problematic.

@serhiy-storchaka
Copy link
Copy Markdown
Member Author

Yes, I also missed it.

@serhiy-storchaka serhiy-storchaka merged commit 51a5715 into python:main May 19, 2026
55 checks passed
@miss-islington-app
Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

@serhiy-storchaka serhiy-storchaka deleted the gzip-header-crc-large branch May 19, 2026 16:20
@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented May 19, 2026

GH-150093 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 19, 2026
serhiy-storchaka added a commit that referenced this pull request May 19, 2026
…9979) (GH-150093)

Do not read the whole filename and comment to memory for calculating the CRC.
(cherry picked from commit 51a5715)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants