Skip to content

feat(drivers/s3): support direct multipart upload - #609

Open
Arielfoever wants to merge 1 commit into
OpenListTeam:mainfrom
Arielfoever:feat/s3
Open

feat(drivers/s3): support direct multipart upload#609
Arielfoever wants to merge 1 commit into
OpenListTeam:mainfrom
Arielfoever:feat/s3

Conversation

@Arielfoever

Copy link
Copy Markdown

Summary / 摘要

There is no user-visible behavior changes for uploading.

HTTP Direct for S3 is now upload by multipart unless DirectUploadMaxParts=1 while the default value is 10000. If S3 is standard, there will be nothing change.

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。
  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。
  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

Related Issues / 关联 Issue

Testing / 测试

  • go test ./...
    no go test available
  • Manual test / 手动测试:
    test on Aliyun OSS and it uploads in 50 parts as planned. No larger file tested.

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmtgo fmtprettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。
    No right to request review.

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他(请注明):

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

@Arielfoever
Arielfoever force-pushed the feat/s3 branch 2 times, most recently from c0ab82b to afb0937 Compare July 26, 2026 15:30
@xrgzs xrgzs changed the title feat(s3): support direct multipart upload feat(drivers/s3): support direct multipart upload Jul 26, 2026
- Upload S3 direct multipart parts using presigned part URLs.

- Complete multipart uploads with collected ETags and abort on failures.

- Treat embedded CompleteMultipartUpload XML errors as upload failures.

- Build completion XML with DOM APIs instead of manual escaping.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant