Skip to content

Do not yield stale frames from blocks() when out= is longer than the file - #495

Open
dylanpulver wants to merge 1 commit into
bastibe:masterfrom
dylanpulver:fix-blocks-out-overlap
Open

Do not yield stale frames from blocks() when out= is longer than the file#495
dylanpulver wants to merge 1 commit into
bastibe:masterfrom
dylanpulver:fix-blocks-out-overlap

Conversation

@dylanpulver

Copy link
Copy Markdown

blocks(out=..., overlap=n) yields frames that were never read from the file when the file is shorter than out. With out supplied those frames are whatever the caller last left in the array.

out = np.full((3, 2), -999.0)          # caller's buffer
list(sf.blocks(one_frame_stereo_file, overlap=2, out=out))
# [[   1.    2.]
#  [-999. -999.]
#  [-999. -999.]]

soundfile.py:1208 slices the block as out[:frames + overlap], but frames is the number of frames still unread, so frames + overlap is the valid length only once the buffer already carries overlap frames at the front. On the first pass output_offset is 0 and the slice runs overlap frames long. The blocksize > frames + overlap guard also does not fire when the two are equal, and then the whole buffer is yielded. The valid length is output_offset + toread on every pass, which is what this uses.

#446 fixed the same arithmetic for the out is None path by allocating min(blocksize, frames) instead. A caller-supplied array cannot be shrunk, so that path kept the defect, and the regression test it added (test_block_longer_than_file_with_overlap_mono) uses blocksize=. test_blocks_with_out and test_blocks_inplace_modification use out= but on a file longer than out, so the intersection was untested.

Measurement, same venv and libsndfile 1.2.2 both legs. Oracle: blocks are contiguous slices of the file, consecutive blocks share overlap frames, iteration stops once every frame has been delivered — written from the docstring, never from soundfile.py. Grid of 196 cells (7 file lengths x blocksize 1..7 x every legal overlap): out= path 70 failures before, 0 after; blocksize= path 0 failures in both legs, which is what validates the oracle. A separate 105-cell fill_value=0.0 grid asserting every block is exactly blocksize long and free of the poison value: 0 failures in both legs. python -m pytest 331 -> 339.

Mutants, each verified in the file before running: (A) restore the shipped slice — the 8 new nodes fail, the other 331 pass; (B) the naive out[:frames] — 8 failures including test_blocks_inplace_modification, and the oracle reports 70 failures on the blocksize= path it previously passed, so dropping the overlap term is not the fix; (C) drop the fill_value is None guard — 12 failures, all the fill_value tests.

python -m pyright soundfile.py reports the same 2 pre-existing errors at soundfile.py:188 before and after; nothing new.

Not tested: only WAV/DOUBLE and the mono int16 fixtures; no non-seekable file, no frames/start/stop combination beyond what the existing suite covers, and I did not touch overlap_memory = np.copy(out[-overlap:]), which reads the same stale region — it is unreachable today because a short read always ends the loop.

I see #213 proposes removing out from blocks() entirely. If that is still the direction, close this.

Written with AI assistance (Claude).

… file

The last block was sliced as out[:frames + overlap], where frames is the
number of frames still unread.  That is only the valid length when the
iteration already carries overlap frames at the front of the buffer.  On
the first pass output_offset is 0, so the slice reaches overlap frames
past the data actually read, and with out= those frames are whatever the
caller left in the array.

The valid length is output_offset + toread on every pass.  Slicing to it
also removes the need for the blocksize > frames + overlap guard, which
did not fire when blocksize == frames + overlap and yielded the whole
buffer instead.

bastibe#446 fixed the same arithmetic for the out is None path by allocating
min(blocksize, frames); a caller-supplied array cannot be shrunk, so
that path kept the defect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant