Skip to content

fix(multipart): resume interrupted uploads without file hashes - #3332

Open
thetapilla wants to merge 2 commits into
OpenListTeam:mainfrom
thetapilla:fix/multipart-file-identity
Open

fix(multipart): resume interrupted uploads without file hashes#3332
thetapilla wants to merge 2 commits into
OpenListTeam:mainfrom
thetapilla:fix/multipart-file-identity

Conversation

@thetapilla

@thetapilla thetapilla commented Oct 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary / 摘要

让分块上传在不勾"尝试秒传"时也能断点续传，并修复原地写入的驱动上续传和"重试"被这次上传自己的半成品挡住的问题。

用户可感知的变化：

  • 不勾"尝试秒传"时，上传中刷新页面后重新选择同一文件也会自动续传，不再从头重传。此前只有带了文件哈希才能续传，而网页端只在勾选秒传时计算哈希
  • Local 这类原地写入的驱动上，不勾"覆盖已存在的文件"时，续传和"重试"不再报 file exists

实现要点：

  • 文件指纹：POST /fs/multipart/init 新增可选请求头 X-File-Identity，是客户端生成的文件指纹。服务端不解析、只比对，最长 256 字节（超过返回 400），随会话保存。任一方带了指纹时，双方指纹相同、且两次 init 都带的同种哈希不冲突，才续传；都不带指纹时沿用原有的哈希规则
  • 哈希仍然有效：指纹可能只覆盖部分内容（网页端取首尾各 1 MiB），所以双方都带了同种哈希时，哈希也必须一致。否则一个只改了中间部分、大小和修改时间都没变的文件，会续传进旧会话，把两份内容拼在一起
  • 续传不被自己的半成品挡住：再次 init 能续上已有会话、并且指纹或哈希证明是同一文件时，跳过 file exists 检查。目标位置上的文件这时属于这次上传：要么是原地写入的驱动写下的半成品，要么是它开始时就允许替换的文件。只续上一个失败的尝试不算证明，同路径、同大小的任何 init 都会续上它，所以换了文件照常拒绝

兼容性说明：新增一个可选请求头，不发这个头的客户端，续传规则不变。会话仍按用户隔离、按路径和大小定位，指纹只决定是否续传用户自己的会话，不是安全边界。

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。
  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。
  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

Related Issues / 关联 Issue

Relates to #3178

Testing / 测试

测试平台：macOS arm64，Go 1.27.3。

  • go test ./...
  • Manual test / 手动测试:

自动化测试：

  • go test -race -count=1 ./internal/multipart/ ./server/handles/
  • 本地 e2e 脚本：指纹相同的再次 init 续传并返回可跳过的分片；指纹不同、或指纹相同但哈希不同时新建会话；257 字节的指纹返回 400，256 字节的被接受；不勾覆盖时续传越过自己的半成品，换了文件仍报 file exists；勾覆盖开始的上传，刷新后不勾覆盖也能续传同一文件。其中哈希冲突、长度上限、越过半成品这三项，都先在修复前的服务端上确认会失败
  • 续传在 Local、S3（MinIO）、SMB（Samba 4.23）上均验证

浏览器实测：

  • 上传到一半刷新页面，重新选择同一文件（Local，不勾覆盖）：init 返回 resumed，只补传剩余分片，内容一致
  • 上传中途换成同名、同大小、内容不同的文件：新建会话，最终是新文件
  • 勾选秒传，两个文件首尾 1 MiB、大小、修改时间都相同，只有中间不同：不续传，结果正好是后一个文件
  • 注入一次分片错误后点"重试"：续传到同一会话，只补发失败的分片

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING。
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmt、go fmt 或 prettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他（请注明）:

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

@thetapilla @claude
fix(multipart): resume uploads by file identity when no hash is sent
e82f855 
- A receiving session could only be resumed when the client sent file
  hashes, and the web client only computes them when rapid upload is
  requested. By default a reload therefore superseded the session and
  restarted the upload from scratch.
- Accept an optional X-File-Identity header on init: an opaque client
  fingerprint (at most 256 bytes) stored with the session. When either side
  has one, a receiving session resumes only if both are equal and no hash
  sent on both inits differs; otherwise the existing hash rule applies
  unchanged.
- Add TestResumeReceivingByIdentity covering eight identity/hash
  combinations.

Co-Authored-By: Claude <noreply@anthropic.com>
@thetapilla thetapilla mentioned this pull request Oct 30, 2026
Open
14 of 25 tasks
@thetapilla @claude
fix(multipart): let an interrupted upload resume over its own partial…
1211d4e 
… file

- With Overwrite off, init answered "file exists" before deciding whether
  to resume. In-place drivers such as Local write the destination from the
  first chunk on, so after a reload the upload's own partial output blocked
  its resume, and the Retry button failed the same way.
- Skip the check when the re-init would resume a session and its identity
  or hashes prove it is the same file: what sits at the destination then
  belongs to that upload.
- Picking up a failed attempt is no proof on its own, since any init for the
  same path and size does that, so a different file is still refused.
- Add TestResumesSameFile.

Co-Authored-By: Claude <noreply@anthropic.com>
@thetapilla
thetapilla force-pushed the fix/multipart-file-identity branch from 2950ba1 to 1211d4e Compare October 31, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@thetapilla