[zhongkechen]

A callback can complete while a durable function is suspended, and replay may never visit it. This change preserves telemetry for visible terminal operations using the existing plugin snapshots and hooks.

Changes

• Keep completion delivery inside OTel. Each invocation's plugin retains terminal input/checkpoint updates, exports them through natural operation hooks or parent registration, and drains unvisited completions before the invocation span ends. An OTel export ledger suppresses duplicate observations while allowing each newly opened replay/virtual context segment to complete. Core no longer adds a completion scheduler, ancestor scans, or extra operation-map copies.
• Preserve replay parents and recover missing parentage. Pending children use the actual replay segment before it closes. Completed context references remain available for later updates. At invocation End, an unavailable parent uses a logged fallback: the invocation span in Invocation view, or a derived logical parent context in Execution view. This does not manufacture missing operation records or reconstruct an unavailable virtual parent's original span.
• Retry failed completion exports. In-progress claims are separate from the successful-export ledger. Failed starts/ends remain pending and can retry through a later notification or invocation End. Retries preserve the span ID, parent, original timestamps and exception events; a failed context segment is completed before another segment opens. During invocation End, failures are isolated per operation with at most one ordinary retry. Other completions and Invocation/Workflow finalization continue, and errors are reported only after cleanup; control exceptions are not retried.
• Isolate operation errors. OperationInfo.from_operation constructs the requested info subclass and copies mutable error stack traces. Terminal hooks use this conversion, so plugin mutations cannot alter checkpointed errors.
• Report terminal outcomes consistently. FAILED, TIMED_OUT and STOPPED map to ERROR even without diagnostic details; exception events use the normalized span-end timestamp.
• Bound flush waiting per provider. One outstanding attempt per provider, with a 30-second caller budget. A blocked provider does not prevent another provider from flushing. Later callers on the same provider perform a fresh flush after admission. Lifecycle hooks and context cleanup remain on their owner.

Stack and scope

main → #758 → #795 → #796

The testing-library implementation is unchanged here, and installed-wheel checks use the published testing library. #795 carries replay visibility and successful-input acknowledgment fixes, local testing-wheel installation, and the CI filters needed by the remaining stack. All five workflow PR filters in #758 target only main. #796 adds the conformance handlers and qualification coverage.

Validation and limits

Validation on Python 3.13.17 passes:

• 875 installed OTel tests against published testing-library 1.3.0, 12 lifecycle compatibility cases, and four released-OTel registration rejection cases for fix(otel): preserve replay completion telemetry #758.
• 906 installed OTel tests, 16 lifecycle/callback cases and four legacy rejection cases for the complete stack with its built testing library.
• 112 workflow/conformance-package checks, plus type, lint, formatting, artifact hash/source verification and package builds.

The stack retains linear history. The additional #758 fix is carried into #795/#796 by rebasing their unchanged layer contents; independent tree checks verify those boundaries. Commit-message validation passes, with the accepted documentation suggestion and its coauthor attribution preserved. GitHub CI runs separately on the new pushed heads.

A processor can deliver a span and then raise, so retrying partial delivery can export it again. Deduplication is invocation-scoped; failed delivery can still be exported again after redelivery because durable state has no exporter acknowledgment. Shared conformance #131 remains a merge prerequisite. The separate virtual-FLAT replay persistence issue JS #997 remains outside this change.

[yaythomas]

Thanks for moving the testing changes to #795 and the conformance changes to #796

[zhongkechen]

Fixed in a5114c4. Both views now isolate completion failures during invocation End, allow one bounded ordinary retry, and finish other completions and root spans before reporting errors. Control exceptions are not retried or swallowed. Public-handler, nested-parent and permanent-failure regressions pass.

[github-actions]

Codex AI review · Finding arf_v1_opkwyjxdk23t3ijca65noht5av

[P2] Isolate deferred export failures from parent scope setup

This drain runs inside on_user_function_start, before _attach_context. If a callback completed during suspension and its processor raises when replay registers the parent, _render_operation_end re-raises because _finalization is still None. Core contains that exception and executes the parent's body without attaching its OTel context, so user spans and correlated logs use the outer/invocation context. Later export retries cannot repair those spans. Contain ordinary failures per pending completion while retaining retry state and allowing parent setup to finish; add a replay regression with a processor that fails once.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review

Two P2 findings: packages/aws-durable-execution-sdk-python-otel/src/aws_durable_execution_sdk_python_otel/execution_plugin.py:619 and packages/aws-durable-execution-sdk-python-otel/src/aws_durable_execution_sdk_python_otel/invocation_plugin.py:593. Static review only; tests were not executed.

Suppressed repeats

• arf_v1_sbn32537tiw6bn2pupqpzjw64o — deferred (maintainer verdict)

Reviewed commit a5114c4a3e7c3b57f7edd8d1170f59b6db3a4469. Workflow run