Skip to content

Log slow workflow task completion diagnostics#2946

Open
Debashismitra01 wants to merge 1 commit into
temporalio:mainfrom
Debashismitra01:fix/slow-wft-completion-diagnostics
Open

Log slow workflow task completion diagnostics#2946
Debashismitra01 wants to merge 1 commit into
temporalio:mainfrom
Debashismitra01:fix/slow-wft-completion-diagnostics

Conversation

@Debashismitra01

Copy link
Copy Markdown

What was changed

Added diagnostics logging for slow Workflow Task completion.

The worker now emits TMPRL1104 logs when Workflow Task processing takes longer than the configured thresholds:

  • INFO when duration is greater than 5 seconds
  • WARN when duration is greater than 10 seconds

The log includes external payload storage statistics:

  • total external storage download time
  • number of downloaded payloads
  • total downloaded payload size
  • total external storage upload time
  • number of uploaded payloads
  • total uploaded payload size

Implementation changes:

  • Added WorkflowTaskPayloadStats for collecting payload operation metrics
  • Added WorkflowTaskPayloadStatsContext for tracking stats during Workflow Task execution
  • Added payload encode/decode instrumentation in CodecDataConverter
  • Added slow Workflow Task completion logging in WorkflowWorker

Why?

External payload storage operations can increase Workflow Task completion time because payloads may need to be uploaded to or downloaded from external storage.

When this latency causes Workflow Tasks to approach their timeout, users need visibility into whether payload storage operations are contributing to the delay.

These diagnostics make slow Workflow Task completion easier to investigate.

Checklist

  1. Closes SDK should log when WFT completion takes a long time - Java #2883

  2. How was this tested:

Added and updated unit tests:

  • WorkflowWorkerTest
    • verifies slow Workflow Task completion emits TMPRL1104
  • CodecDataConverterTest
    • verifies payload encode/decode operations record statistics
  • WorkflowTaskPayloadStatsTest
    • verifies upload/download timing, payload count, and size tracking

Ran:

./gradlew :temporal-sdk:test --tests io.temporal.internal.worker.WorkflowWorkerTest
./gradlew :temporal-sdk:test --tests io.temporal.common.converter.CodecDataConverterTest
./gradlew :temporal-sdk:test --tests io.temporal.internal.worker.WorkflowTaskPayloadStatsTest
./gradlew spotlessCheck

3.Any docs updates needed?

No docs updates needed. This only adds internal diagnostics logging.

@Debashismitra01 Debashismitra01 requested a review from a team as a code owner July 10, 2026 12:27
@CLAassistant

CLAassistant commented Jul 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SDK should log when WFT completion takes a long time - Java

2 participants