From 683313ed2cf521023a049b2aa94a7c81ab6a7fd1 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 21 Mar 2026 19:32:27 +0000 Subject: [PATCH 1/2] Initial plan From 9f4b58d58acccb03d4233131e77646b3b915fe51 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 21 Mar 2026 19:34:14 +0000 Subject: [PATCH 2/2] docs: specify ETag computation format for single-part and multipart uploads Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/e7aa7963-d972-4405-b25c-afdba2f7b535 --- docs/s3_compatible_adapter_design.md | 30 +++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/s3_compatible_adapter_design.md b/docs/s3_compatible_adapter_design.md index b090e31f..47f42ba8 100644 --- a/docs/s3_compatible_adapter_design.md +++ b/docs/s3_compatible_adapter_design.md @@ -188,6 +188,34 @@ Each manifest part entry should include: The chunk keys remain immutable. The manifest is the only key that makes an upload visible to readers. +### 5.4 ETag computation + +ETags must match the format used by Amazon S3 so that clients and SDKs can use them for integrity checks and conditional requests. + +**Single-part uploads (`PutObject`):** + +The ETag is the lowercase hex-encoded MD5 hash of the complete object body: + +``` +ETag = hex(md5(object_body)) +``` + +Example: `"d41d8cd98f00b204e9800998ecf8427e"` (MD5 of empty body). + +**Multipart uploads (`CompleteMultipartUpload`):** + +The ETag is the lowercase hex-encoded MD5 hash of the binary concatenation of the MD5 hash of each part (in part-number order), followed by a hyphen and the total number of parts: + +``` +ETag = hex(md5(md5_bytes(part_1) || md5_bytes(part_2) || ... || md5_bytes(part_N))) + "-N" +``` + +where `md5_bytes(part_i)` is the raw 16-byte MD5 digest of part `i`'s body and `N` is the number of parts. + +Example with 3 parts: `"abc123...-3"`. + +Per-part ETags stored in each part descriptor must use the single-part format (MD5 of the part body). `CompleteMultipartUpload` validates that the caller-supplied per-part ETags match the stored part descriptors before committing the manifest. + ## 6. Routing Model ### 6.1 Logical route key @@ -250,7 +278,7 @@ Detailed steps: 2. Resolve leader for the logical object key. 3. If local node is not the verified leader, reverse-proxy the HTTP request. 4. Stream the body into chunk writes under a fresh hidden `upload_id`. -5. Compute `ETag`, length, and header-derived metadata while streaming. +5. Compute `ETag` (hex-encoded MD5 of the full object body), length, and header-derived metadata while streaming. 6. Finalize with a small OCC transaction that writes the new manifest and optional GC marker for the previous `upload_id`. 7. Return success only after the manifest commit is durable.