refactor(storage): remove the unused tus replication path - #529
Open
rickyrombo wants to merge 1 commit into
Open
refactor(storage): remove the unused tus replication path#529rickyrombo wants to merge 1 commit into
rickyrombo wants to merge 1 commit into
Conversation
replicateToHost was never called -- the only reference was a commented line behind a TODO -- so no node has ever sent a tus replication upload, and the receiving branch it fed has never run in production. Leaving it in place was the liability. It is a third way to write blobs into the bucket, and structurally the weakest of the three: tusd notifies completion on a channel after the response is sent, so the receiver cannot report a rejection -- a full disk, a cid mismatch, a failed bucket write -- to the sender. A sender that trusted the stream's success would have recorded a mirror for a blob the peer never stored, and the sweep would not re-queue it because the mirror count already met target. Its checks also run after the payload is staged on disk, where the pull and multipart paths decide before any bytes move. Only the replication branch goes. Tus stays for user uploads, which is what it is for, and the tusgo client remains in the sdk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes the tus-based blob replication path. It was never reachable:
replicateToHost's only reference was a commented-out line behind a// TODO: Replicate with TUSD, so no node has ever sent a tus replication upload and the receiving branch has never run in production.Why remove rather than finish it
Tusd notifies completion on a channel after the response is sent, so the receiver cannot tell the sender that it rejected the blob — full disk, cid mismatch, failed bucket write all just log and return. A sender that treated the stream's success as replication success would record a mirror for a blob the peer never stored, and
findMissedReplicationswould not re-queue it, because the mirror count already meets target. Silent under-replication is worse than a visible failure.Its checks are also ordered the wrong way round. The pull path answers
already_presentor "no space" before any payload moves; the tus path stages the whole file to/tmp/tusd-uploadsfirst and decides afterward. There is no expiration or GC configured either, so an aborted transfer would leave a partial file behind indefinitely.Two existing paths already cover this —
/internal/blobs/pulland the multipart/internal/blobspush — and both report their outcome synchronously.Scope
replicateToHost,tusAuthTransport, and theisReplicationbranches invalidateTusUploadBeforeCreate,handleTusdUploadCreated, andhandleTusdUploadComplete.tusgoclient stays too;pkg/sdkuses it.Follow-ups
Part of a three-PR sequence from a review of replication:
BlobStorageStreaminggate inreplicateStoredFileToHost, leaving multipart push as the version-skew fallbackStill open after those: chunked ranged pull plus an async handoff, so a single large transfer cannot occupy a replication worker for tens of minutes. If large-blob replication ever wants resumability again, the right shape is
PreFinishResponseCallback— it runs before the response, so the sender gets a truthful status — not the post-finish channel this PR removes.Testing
go build ./...clean.go test ./pkg/mediorum/server/...shows the same three failures as unmodifiedmain(TestPollDelistStatuses,TestRepair,TestUploadFile) and no new ones.🤖 Generated with Claude Code