Describe the feature
When a CheckpointDurableExecution response arrives without a CheckpointToken, the SDK should treat it as a signal that the service will accept no further checkpoints from this invocation. The SDK should stop issuing checkpoints and end the invocation cleanly with Status = InvocationStatus.Pending.
Use Case
A missing token does not mean the execution is finished. It means this invocation cannot make further progress and the execution continues in a later invocation.
Failed is wrong: it claims the execution finished.- Letting an exception escape to the host is wrong: Lambda retries the invocation, but the retry has no valid token and can do nothing useful. It also puts an error in customer logs for a condition the SDK understood.
Pending is what the SDK already returns for every suspend (wait, callback, scheduled retry). This is a suspend.
Current behavior (master at 8ed93ce):
LambdaDurableServiceClient.CheckpointAsync returns response.CheckpointToken unchanged, so a missing token becomes null:
|
return response.CheckpointToken; |
CheckpointBatcher stores it:
|
var newToken = await _flushAsync(_checkpointToken, updates, cancellationToken).ConfigureAwait(false); |
|
Volatile.Write(ref _checkpointToken, newToken); |
The next flush sends CheckpointToken = checkpointToken ?? "". The service rejects the empty token with a 400. IsTerminalCheckpointError classifies that 400 as terminal (see the linked bug for the stale-token carve-out; an empty token may also be rejected as a validation error with a different message, which the carve-out would not cover either way). The invocation returns Failed. So a missing token fails the execution today.
Proposed Solution
- In
CheckpointBatcher (or the flush delegate), when the returned token is null, stop flushing and signal the TerminationManager with a new suspend reason. DurableExecutionHandler.RunAsync already maps a termination without an exception to InvocationStatus.Pending, so no change is needed there.
- Do not issue further checkpoint calls; there is no token to send. Queued updates are abandoned and replay on the next invocation. AT_MOST_ONCE steps whose START landed in the last accepted checkpoint will not run again, which is the defined semantics of AT_MOST_ONCE.
- Unit test: a checkpoint response with a null token produces
Pending, no further checkpoint calls, and no exception.
Amazon.Lambda.DurableExecution.Testing: allow the in-memory service to omit the token on a chosen checkpoint so customers can test their workflows against this path.
Other Information
Prerequisite: the stale-token classification bug (linked in a comment below). That fix is the fallback path whenever the missing token is not detected, so it should land first.
The same feature is being tracked in the JS, Python, Java, Rust, and Go durable execution SDKs; links are in a comment below.
Acknowledgements
AWS .NET SDK and/or Package version used
Amazon.Lambda.DurableExecution 2.0.0 (master at 8ed93ce)
Targeted .NET Platform
Any
Operating System and version
Any
Describe the feature
When a
CheckpointDurableExecutionresponse arrives without aCheckpointToken, the SDK should treat it as a signal that the service will accept no further checkpoints from this invocation. The SDK should stop issuing checkpoints and end the invocation cleanly withStatus = InvocationStatus.Pending.Use Case
A missing token does not mean the execution is finished. It means this invocation cannot make further progress and the execution continues in a later invocation.
Failedis wrong: it claims the execution finished.Pendingis what the SDK already returns for every suspend (wait, callback, scheduled retry). This is a suspend.Current behavior (master at 8ed93ce):
LambdaDurableServiceClient.CheckpointAsyncreturnsresponse.CheckpointTokenunchanged, so a missing token becomesnull:aws-lambda-dotnet/Libraries/src/Amazon.Lambda.DurableExecution/Services/LambdaDurableServiceClient.cs
Line 88 in 8ed93ce
CheckpointBatcherstores it:aws-lambda-dotnet/Libraries/src/Amazon.Lambda.DurableExecution/Internal/CheckpointBatcher.cs
Lines 242 to 243 in 8ed93ce
The next flush sends
CheckpointToken = checkpointToken ?? "". The service rejects the empty token with a 400.IsTerminalCheckpointErrorclassifies that 400 as terminal (see the linked bug for the stale-token carve-out; an empty token may also be rejected as a validation error with a different message, which the carve-out would not cover either way). The invocation returnsFailed. So a missing token fails the execution today.Proposed Solution
CheckpointBatcher(or the flush delegate), when the returned token isnull, stop flushing and signal theTerminationManagerwith a new suspend reason.DurableExecutionHandler.RunAsyncalready maps a termination without an exception toInvocationStatus.Pending, so no change is needed there.Pending, no further checkpoint calls, and no exception.Amazon.Lambda.DurableExecution.Testing: allow the in-memory service to omit the token on a chosen checkpoint so customers can test their workflows against this path.Other Information
Prerequisite: the stale-token classification bug (linked in a comment below). That fix is the fallback path whenever the missing token is not detected, so it should land first.
The same feature is being tracked in the JS, Python, Java, Rust, and Go durable execution SDKs; links are in a comment below.
Acknowledgements
AWS .NET SDK and/or Package version used
Amazon.Lambda.DurableExecution 2.0.0 (master at 8ed93ce)
Targeted .NET Platform
Any
Operating System and version
Any