Clamp ring socket transfers so a payload of 2 GiB or more can be sent - #4281
Open
erwinzhang7 wants to merge 2 commits into
Open
Clamp ring socket transfers so a payload of 2 GiB or more can be sent#4281erwinzhang7 wants to merge 2 commits into
erwinzhang7 wants to merge 2 commits into
Conversation
send(2) and recv(2) reject a length above INT_MAX with EINVAL, and the ring backend passes the whole remaining size of a task straight to the syscall. Any single transfer of 2 GiB or more therefore fails immediately, ten times in a row, and the group aborts: [ring] Sending to socket 3 failed with errno 22 ... x10 [ring] Too many send/recv errors. Aborting... Measured on two M4 Pro minis over a direct Thunderbolt link, the boundary is exactly INT_MAX. 2,147,483,640 bytes goes through and 2,147,483,648 does not, so it is the syscall's limit rather than anything about the payload. The loop already advances the buffer and shrinks the remaining size after a short transfer, which is the same path a large payload takes on a busy socket, so clamping each individual call is all that is needed and the rest of the machinery is unchanged. 1 GiB leaves plenty of room under the limit. A caller could avoid this today only by knowing to split the payload, or by raising --connections-per-ip so the segments happen to land under 2 GiB, which is a strange thing to have to discover. Verified on the same pair: every size from INT_MAX - 4099 to INT_MAX + 1048577 now succeeds where the last three failed, a dtype sweep of float32, float16, bfloat16, uint32 and int8 at up to 2.24 GiB succeeds, and all_sum bandwidth at 16, 64 and 256 MB is unchanged at 4.16, 5.55 and 5.55 GB/s.
zcbenz
approved these changes
Aug 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4280.
send(2)andrecv(2)reject a length aboveINT_MAXwith EINVAL, and the ringbackend hands the whole remaining size of a task to the syscall, so a single transfer
of 2 GiB or more fails outright. Measured boundary is exactly
INT_MAX:2,147,483,640 bytes goes through, 2,147,483,648 does not.
The loop already advances the buffer and shrinks the remaining size after a short
transfer, which is the same path a large payload takes on a busy socket, so clamping
each individual call is all that's needed. 1 GiB leaves plenty of headroom.
Verified on two M4 Pro minis over a direct Thunderbolt link:
INT_MAX - 4099toINT_MAX + 1048577now succeeds, where thelast three aborted
all_sumbus bandwidth at 16, 64 and 256 MB is unchanged: 4.16, 5.55, 5.55 GB/s