Skip to content

fix(s3): enforce conditional writes with valid object ETags - #3068

Draft
jyxjjj wants to merge 1 commit into
mainfrom
fix-3067
Draft

fix(s3): enforce conditional writes with valid object ETags#3068
jyxjjj wants to merge 1 commit into
mainfrom
fix-3067

Conversation

@jyxjjj

@jyxjjj jyxjjj commented Sep 10, 2026

Copy link
Copy Markdown
Member

Summary / 摘要

  • 更新 gofakes3,支持 PUT、CompleteMultipartUpload 和 CopyObject 目标对象的条件写入。
  • 条件不满足时返回 412,避免错误覆盖或创建对象。
  • 为 HEAD/GET 提供基于内容的 ETag,并在内容未改变时保留 multipart ETag。
  • 阻止条件请求通过下载或上传重定向绕过检查。
  • 修正 Last-Modified 的 UTC 格式,传播存储错误并关闭内容读取器。

驱动没有提供 MD5 时,需要完整读取对象计算 ETag,包括 HEAD 和 Range 请求,会增加读取开销。Multipart ETag 保存在内存元数据中,重启后改用内容 MD5。

并发保护限于同一个 GoFakeS3 实例中的相同 bucket/key,不涵盖外部写入或多个实例。

  • This PR has breaking changes.
  • This PR changes public API, config, storage format, or migration behavior.
  • This PR requires corresponding changes in related repositories.

Related repository PRs / 关联仓库 PR:

  • OpenList-Frontend: 不适用。
  • OpenList-Docs: 不适用。

Related Issues / 关联 Issue

Fixes #3067

Testing / 测试

  • go test ./...
  • Manual test / 手动测试:未执行。
  • gofakes3 条件请求回归测试通过,包含并发条件写入和竞态检查。
  • gofakes3 内部测试通过,启用竞态检查。
  • 已运行 gofmt 和 git diff --check。

OpenList 集成测试和完整测试套件因所需依赖不可用而未完成。
已添加 OpenList 对象哈希、multipart 元数据和条件重定向回归测试,但尚未执行成功。

Checklist / 检查清单

  • I have read CONTRIBUTING.
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
  • I have requested review from relevant maintainers or code owners where applicable.

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.

Tools used / 使用工具:

  • Codex

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Tests / 测试

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.

  • I can reproduce all AI-assisted content included in this PR without any AI tools.

- Pin gofakes3 with conditional PUT, multipart completion, and copy support
- Derive object ETags from content and preserve matching multipart validators
- Keep conditional requests in the S3 handler instead of redirecting them
- Format modification times in UTC and propagate storage errors
- Close content readers and discard metadata after successful deletion
- Add regression tests for object hashes, metadata, and conditional redirects

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
@jyxjjj
jyxjjj marked this pull request as draft September 10, 2026 15:39
@jyxjjj

jyxjjj commented Sep 10, 2026

Copy link
Copy Markdown
Member Author

需要等待依赖合并后更新提交哈希

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] S3 服务器不兼容RFC 9110 HTTP Semantics(Preconditions)

1 participant