Fix deadlock, silent data loss and crashes in the sort pipeline - #13
Merged
Merged
Conversation
- Run the build, sort and save stages in one errgroup, so a save error (toBytes failure, full disk) no longer deadlocks Sort() and a sort error no longer leaks the save goroutine and its temp file. - Report a read error on a chunk's first record instead of silently dropping the chunk, and treat a length header without its payload as io.ErrUnexpectedEOF instead of a clean end of chunk. - Return the recovered error from the legacy FromBytes wrapper (named results) instead of (nil, nil). - Convert panics in compareFunc, fromBytes and toBytes during the save and merge stages into ComparisonError, DeserializationError and SerializationError instead of crashing the process. - Close the temp reader before closing the result channels, fixing the "send on closed channel" panic when Close fails. - Create the temp file lazily, once a second chunk exists, and close it on every path. Creation errors are reported on the error channel, so the constructors never return a nil sorter. - Release the .extsort_<pid> directory reference when the reader closes. - Temp dir selection: return an explicit TempFilesDir unchanged so New reports it when unusable, and skip default candidates that are missing or not writable (no /var/tmp, read-only root) instead of failing. Add regression tests for each bug, and make TestDeserializationError, TestNilInputs and TestComparisonFunctionPanic reach FromBytes and the merge and require an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add regression tests for each bug, and make TestDeserializationError, TestNilInputs and TestComparisonFunctionPanic reach FromBytes and the merge and require an error.