Skip to content

Release compression reader references on destruction and read errors - #382

Open
FFspiky wants to merge 1 commit into
indygreg:mainfrom
FFspiky:fix-compression-reader-lifetimes
Open

FFspiky wants to merge 1 commit into
indygreg:mainfrom
FFspiky:fix-compression-reader-lifetimes

Conversation

@FFspiky

@FFspiky FFspiky commented Nov 16, 2026

Copy link
Copy Markdown

Clear the compression reader's retained readResult during destruction and decref the allocated output bytes when reading new source input fails. Preserve the original exception.

Partially addresses #297: only compressionreader lifetime and read() output cleanup; read_to_iter and heap-type cleanup are outside this patch.

Regression coverage: Weak-reference checks for invalid source results and destruction with partially consumed input; eight 1 MiB read failures retain about 8 MiB on the C baseline. The post-fix test bounds retained allocations below one output buffer.

Validation on macOS 26.2 arm64 / CPython 3.12.14:

  • ZSTD_WARNINGS_AS_ERRORS=1 python setup.py build_ext --inplace builds the C and CFFI extensions.
  • ZSTD_SLOW_TESTS=1 python -m pytest -q -n 4: 295 passed.
  • PYTHON_ZSTANDARD_IMPORT_POLICY=cffi ZSTD_SLOW_TESTS=1 python -m pytest -q -n 4: 269 passed, 26 skipped.
  • Affected stream test modules: 22 passed on each backend. The new failure cases fail on the C baseline and pass after the fix; unchanged CFFI passes the same regressions.
  • ruff check ., mypy tests/*.py zstandard/*.py, changed-file ruff format --check, and git diff --check pass.
  • Full ruff format --check . reports only an existing setup.py indentation issue, reproduced on the untouched base.

Both runs enabled ZSTD_SLOW_TESTS=1 with the default Hypothesis profile (100 examples), rather than the CI profile (1000 examples).

The tracemalloc regression is restricted to CPython, so importing the test module does not introduce a tracemalloc dependency on PyPy.

Linux/Windows, other Python versions, PyPy, and the experimental Rust backend were not run locally.

AI disclosure: Code, tests, and this description were generated with OpenAI Codex and validated with automated local checks. This does not claim human review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant