Merge testing repo into this monorepo#423
Open
zhongkechen wants to merge 146 commits into
Open
Conversation
Initial test framework to run AWS Durable Functions locally in a unit test environment. Includes validation for: - step - wait - run_in_child_context - create_callback - wait_for_callback - wait_for_condition - parallel - map
Renamed the external module references from aws_durable_functions_sdk_python
to aws_durable_execution_sdk_python and updated the testing module name from
aws_durable_functions_sdk_python_testing to aws_durable_execution_sdk_python_testing
throughout the entire codebase.
Also update Lambda Service API to include new durable_execution_arn arg.
1. Updated all import statements across 52 files including:
- All Python source files in src/aws_durable_execution_sdk_python_testing/
- All test files in tests/
- Documentation files (README.md, CONTRIBUTING.md)
2. Fixed API compatibility issues that arose from the module rename:
- Updated InMemoryServiceClient.checkpoint() method to include the new
durable_execution_arn parameter
- Updated InMemoryServiceClient.get_execution_state() method to include
the new durable_execution_arn parameter
- Updated corresponding test cases to use the new method signatures
- Added appropriate # noqa: ARG002 comments for unused parameters in
the in-memory implementation
3. Maintained code quality standards:
- All 406 tests pass ✅
- Type checking passes ✅
- Code formatting passes ✅
- Test coverage remains above 99% (99.15%) ✅
Add ci to sync package with Gitfarm --------- Co-authored-by: hsilan <hsilan@amazon.com>
…n permissions (#5) Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
- Fix checkpoint processor to preserve attempt count and timestamp - Update InvokeOptions with latest svc signature - Add DurableChildContextTestRunner
Example test was failing because return type now deserialized.
next_attempt_timestamp is a datetime now, not a str.
Implement complete HTTP service that mimics AWS APIs for local durable function testing. Add cli to start web-runner and invoke methods on it. It has a multi-threaded server, strongly-typed routing, and AWS CLI compatibility. Note this PR does not fully implement all features. This is a stub to get the end-to-end flow hooked up. Specifically the various list and history apis are not implement yet. Temporarily lower cov minimum to 96% while placeholder functionality still in place. - Add ThreadingHTTPServer-based WebServer with graceful shutdown - Implement strongly-typed Route system with Router class for efficient path matching and parameter extraction - Create comprehensive HTTP models with AWS serialization integration - Add boto3-compatible serialization using rest-json protocol - Implement Router class with pattern matching for all AWS API endpoints - Add EndpointHandler base class with common HTTP utilities and error handling - Create complete handler implementations for all durable execution operations - Integrate shared router and handler registry for thread-safe request processing - Add comprehensive serialization dataclasses with from_dict/to_dict methods - Implement AWS rest-json serialization using botocore components - Support all AWS Lambda durable execution endpoints (/2025-12-01/*) - Add automatic fallback from AWS to JSON serialization - Create dex-local-runner CLI with start-server, invoke, get-durable-execution, and get-durable-execution-history commands - Add comprehensive configuration via CLI args and AWS_DEX_* env variables - Integrate LambdaInvoker for better Lambda service compatibility - Support boto3 client integration with lambdainternal-local service - Add comprehensive AWS-compatible exception mapping (400/404/409/500) - Implement consistent error response formatting across all endpoints - Add request validation with proper field checking and type safety - Support operation-specific error codes and messages
* feat: add python SDK examples and CLI tool for managing them * fix: update deployment script to package python files at root level * fix: skip wait for callback in integration test
…g durable config (#23)
fix: update branch in default pr template
Signed-off-by: Astraea Sinclair <quinsclr@amazon.com>
- Update test case after we add early exit for concurrency - #242
- Sometimes the function is still being deleted when we exausted the default 5 retries. Increase max_retry for resource_conflict to 8.
- Remove customized json encoder - Use to_json_dict() and from_json_dict() for Operation and ExecutionInput serdes
SQLite storage was failing with "Object of type datetime is not JSON serializable" when attempting to save execution state after Lambda invocation. The root cause was InvocationCompletedDetails.to_dict() returning raw datetime objects instead of JSON-serializable integers. This fix adds to_json_dict() and from_json_dict() methods to InvocationCompletedDetails that convert datetime objects to/from Unix milliseconds using TimestampConverter, matching the pattern already used by the SDK's Operation class. Changes: - Add InvocationCompletedDetails.to_json_dict() for serialization - Add InvocationCompletedDetails.from_json_dict() for deserialization - Update Execution.to_json_dict() to call completion.to_json_dict() - Update Execution.from_json_dict() to call from_json_dict() The to_dict() method is preserved for internal use where datetime objects are needed, while to_json_dict() is used for storage and JSON serialization paths. Fixes execution persistence failures in SQLite and filesystem stores. closes #193
Co-authored-by: Alex Wang <wangyb@amazon.com>
Co-authored-by: Alex Wang <wangyb@amazon.com>
Co-authored-by: Alex Wang <wangyb@amazon.com>
Create emulator image from the python testing library. The image tag will be the aws-durable-sdk-python-testing version with v prefixing it, e.g. v1.1.1. --------- Co-authored-by: hsilan <hsilan@amazon.com>
- Introduce a shared Lambda client config with a 960s read_timeout and retries disabled - Route all Lambda client construction through a single helper
* remove requests library * update timeout and handling of timeout errors
- notify-issues.yml: issue notifications - notify-pr.yml: pull request notifications - notify-release.yml: callable workflow for release notifications - pypi-publish.yml: call notify-release after successful publish
After PR #216, durable-execution ARNs minted by Execution.new() contain a literal '/' of the form "<uuid>/<invocation-id>". boto's rest-json serializer percent-encodes '/' as %2F in the non-greedy {DurableExecutionArn} URI label, so paths arriving at the local WebServer look like: /2025-12-01/durable-executions/<uuid>%2F<invocation-id> The same shape applies to ListDurableExecutionsByFunction with function names like "MyFunction:$LATEST" (':' -> %3A, '$' -> %24). Without decoding, store lookups never match the key and every Get/State/History/Checkpoint/Stop returns 404. List queries silently return an empty result set. - Decode each segment once in Route.from_string. raw_path is kept as the original wire string for logging. Splitting on '/' happens before decoding so a captured value containing %2F stays inside its segment instead of acting as a path separator. - Remove the now-redundant per-route unquote() calls from the three callback routes (added in #117 for the same bug shape). - Add a real-boto regression test under tests/web/e2e/ that drives a live WebServer for every affected operation with values containing the characters boto percent-encodes. Closes the test-coverage gap that let the bug ship. - Strengthen test_route_with_special_characters to assert both segments[N] and the named field are decoded while raw_path keeps the wire form. Affects users running WebRunner / dex-local-runner against their durable function in RIE; pre-fix, the function 404s on its first checkpoint after upgrading to 1.2.0. Closes #222
zhongkechen
force-pushed
the
merge_testing
branch
from
508f391 to
3399283
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Fix aws/aws-durable-execution-sdk-python-testing#226
Description of changes:
Review only the last 3 commits in this PR
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.