Block blob client stage block from url - #2442
Conversation
|
@microsoft-github-policy-service agree company="Klarrio GmbH" |
|
/azp run |
|
Azure Pipelines could not run because the pipeline triggers exclude this branch/path. |
|
Thanks for the quick fix! I might still need more time to test and review this PR. |
| context: Context | ||
| ): Promise<Models.BlockBlobStageBlockFromURLResponse> { | ||
| throw new NotImplementedError(context.contextId); | ||
|
|
There was a problem hiding this comment.
The implementation doesn't respect header "x-ms-source-range". (rest API doc)
The header is important, since normally this API will be used to copy blob block blob (For small blob less then 256MB, can use single CopyFromUrl() API.) And for big blob, normally the copy should happen chunk by chunk, so customer can use "x-ms-source-range" to handle which chunk of source blob will be copy.
| throw StorageErrorFactory.getConditionNotMet(context.contextId!); | ||
| } | ||
|
|
||
| const stageBlockRes = await this.stageBlock(blockId, |
There was a problem hiding this comment.
When copy a 300000KB block blob, will get error here:
2024-08-23T09:40:50.503Z 8f01df46-0ea6-4ade-9594-7a46dec001a7 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The size of the request body 307167232 mismatches the content-length 307200000. ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=The size of the request body 307167232 mismatches the content-length 307200000. ErrorHTTPHeaders={"x-ms-error-code":"InvalidOperation","x-ms-request-id":"8f01df46-0ea6-4ade-9594-7a46dec001a7"} ErrorHTTPBody="\n\n
InvalidOperation\n The size of the request body 307167232 mismatches the content-length 307200000.\nRequestId:8f01df46-0ea6-4ade-9594-7a46dec001a7\nTime:2024-08-23T09:40:17.319Z\n" ErrorStack="StorageError: The size of the request body 307167232 mismatches the content-length 307200000.\n at Function.getInvalidOperation (D:\code\azurite\src\blob\errors\StorageErrorFactory.ts:151:12)\n at BlockBlobHandler.stageBlock (D:\code\azurite\src\blob\handlers\BlockBlobHandler.ts:203:33)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async BlockBlobHandler.stageBlockFromURL (D:\code\azurite\src\blob\handlers\BlockBlobHandler.ts:324:27)"
There was a problem hiding this comment.
Thanks for the contribution!
I tried to test the change by Powershell cmdlet Copy-AzStorageBlob with a 300000KB block blob, it failed.
See details in my comments.
Besides fix the issue, please also rebase the code on latest main branch to get latest test fix, so the CI can pass.
| await containerClient.delete(); | ||
| }); | ||
|
|
||
| it("Blob should be staged from URL and committed @loki", async () => { |
There was a problem hiding this comment.
Please also cover :
- "x-ms-source-range" when only copy part of source blob as block
- Big source blob like > 300MB
- Cross account copy (both account in same Azurite instance)
| - Abort Copy Blob (Only supports copy within same Azurite instance) | ||
| - Copy Blob From URL (Only supports copy within same Azurite instance, only on Loki) | ||
| - Access control based on conditional headers | ||
| - Stage Block From URL (Only supports copy within same Azurite instance, only on Loki) |
2 participants
This PR implements blockBlob stageBlockFromURL and relates to #2440.
Add Change Log