Skip to content

fix(aliyundrive): limit callback concurrency - #3071

Open
nostalume wants to merge 1 commit into
OpenListTeam:mainfrom
nostalume:feat/aliyun-callback-admission
Open

fix(aliyundrive): limit callback concurrency#3071
nostalume wants to merge 1 commit into
OpenListTeam:mainfrom
nostalume:feat/aliyun-callback-admission

Conversation

@nostalume

@nostalume nostalume commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary / 摘要

  • Add a per-Aliyun-user admission limit for proxied callback downloads. The new Callback Concurrency storage setting defaults to 1; nonpositive legacy values normalize to 1, and mounts sharing a user use the smallest live limit.

  • Hold admission for the complete response-body lifetime and release it exactly once on EOF, read failure, cancellation, or Close. Direct 302 links remain URL-only and proxy readers retain the server download limiter.

  • Retry only HTTP 403 callback responses containing both RequestDeniedByCallback and ExceedMaxConcurrency, with three total attempts and bounded exponential jitter.

  • Return a typed temporary-capacity error after bounded admission/retry exhaustion and translate only that error to standard S3 SlowDown / HTTP 503.

  • Advance the official github.com/OpenListTeam/gofakes3 dependency from v0.8.1 to merged commit cd3c030a83b4, which provides the required SlowDown error.

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。

  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。

  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

Related Issues / 关联 Issue

Relates to #2969

Testing / 测试

Validated after rebasing onto OpenList main at f18b4ac with Go 1.27.1:

  • go test ./drivers/aliyundrive_open -count=5
  • Relevant S3 error, range, and redirect tests with -count=5
  • go test ./server/s3 -run '^$'
  • go test -vet=off ./... -run '^$'
  • go vet ./drivers/aliyundrive_open ./server/s3
  • go mod verify
  • go test ./server/s3 -count=1
    • The new and related tests pass. The Windows package run reaches four upstream multipart tests that reject manually interpolated temporary paths as invalid JSON escapes; the independent test-only repair is fix(s3): encode multipart fixture paths #3074. The complete package passes under Linux race instrumentation below.
  • go test -race ./drivers/aliyundrive_open -count=1 (Go 1.27.1, Tumbleweed WSL, CGO enabled, GCC 16.2.0)
  • go test -race ./server/s3 -count=1 (same environment; complete package including multipart tests)
  • Manual test / 手动测试:
    • No live Aliyun account canary was run. Deterministic transports cover peak active bodies, same-user sharing, independent users, reconfiguration drain, cancellation, all body release paths, exact provider-error classification, URL redaction, and S3 503/XML output.

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmtgo fmtprettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他(请注明):

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

This draft intentionally leaves the human-review and reproducibility confirmations unchecked for the author to complete after review.

- Share proxy callback admission by Aliyun user identity and hold permits for complete response-body lifetimes.
- Retry only verified callback-capacity rejections while preserving direct redirects and server download limiting.
- Map exhausted temporary capacity to S3 SlowDown through the merged OpenListTeam gofakes3 module.
- Cover shared limits, lifecycle release, cancellation, retry classification, and the S3 HTTP response.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

# Conflicts:
#	go.mod
#	go.sum
#	server/s3/pager.go
@nostalume
nostalume force-pushed the feat/aliyun-callback-admission branch from 91fcfab to 902a7d6 Compare September 11, 2026 15:44
@nostalume
nostalume marked this pull request as ready for review September 12, 2026 01:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant