Do not yield stale frames from blocks() when out= is longer than the file - #495
Open
dylanpulver wants to merge 1 commit into
Open
Do not yield stale frames from blocks() when out= is longer than the file#495dylanpulver wants to merge 1 commit into
dylanpulver wants to merge 1 commit into
Conversation
… file The last block was sliced as out[:frames + overlap], where frames is the number of frames still unread. That is only the valid length when the iteration already carries overlap frames at the front of the buffer. On the first pass output_offset is 0, so the slice reaches overlap frames past the data actually read, and with out= those frames are whatever the caller left in the array. The valid length is output_offset + toread on every pass. Slicing to it also removes the need for the blocksize > frames + overlap guard, which did not fire when blocksize == frames + overlap and yielded the whole buffer instead. bastibe#446 fixed the same arithmetic for the out is None path by allocating min(blocksize, frames); a caller-supplied array cannot be shrunk, so that path kept the defect.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
blocks(out=..., overlap=n)yields frames that were never read from the file when the file is shorter thanout. Withoutsupplied those frames are whatever the caller last left in the array.soundfile.py:1208slices the block asout[:frames + overlap], butframesis the number of frames still unread, soframes + overlapis the valid length only once the buffer already carriesoverlapframes at the front. On the first passoutput_offsetis 0 and the slice runsoverlapframes long. Theblocksize > frames + overlapguard also does not fire when the two are equal, and then the whole buffer is yielded. The valid length isoutput_offset + toreadon every pass, which is what this uses.#446 fixed the same arithmetic for the
out is Nonepath by allocatingmin(blocksize, frames)instead. A caller-supplied array cannot be shrunk, so that path kept the defect, and the regression test it added (test_block_longer_than_file_with_overlap_mono) usesblocksize=.test_blocks_with_outandtest_blocks_inplace_modificationuseout=but on a file longer thanout, so the intersection was untested.Measurement, same venv and libsndfile 1.2.2 both legs. Oracle: blocks are contiguous slices of the file, consecutive blocks share
overlapframes, iteration stops once every frame has been delivered — written from the docstring, never fromsoundfile.py. Grid of 196 cells (7 file lengths x blocksize 1..7 x every legal overlap):out=path 70 failures before, 0 after;blocksize=path 0 failures in both legs, which is what validates the oracle. A separate 105-cellfill_value=0.0grid asserting every block is exactlyblocksizelong and free of the poison value: 0 failures in both legs.python -m pytest331 -> 339.Mutants, each verified in the file before running: (A) restore the shipped slice — the 8 new nodes fail, the other 331 pass; (B) the naive
out[:frames]— 8 failures includingtest_blocks_inplace_modification, and the oracle reports 70 failures on theblocksize=path it previously passed, so dropping the overlap term is not the fix; (C) drop thefill_value is Noneguard — 12 failures, all the fill_value tests.python -m pyright soundfile.pyreports the same 2 pre-existing errors atsoundfile.py:188before and after; nothing new.Not tested: only WAV/DOUBLE and the mono int16 fixtures; no non-seekable file, no
frames/start/stopcombination beyond what the existing suite covers, and I did not touchoverlap_memory = np.copy(out[-overlap:]), which reads the same stale region — it is unreachable today because a short read always ends the loop.I see #213 proposes removing
outfromblocks()entirely. If that is still the direction, close this.Written with AI assistance (Claude).