Skip to content

fix(metrics): deduct downstream utilization on task transforms#24731

Merged
gwenaskell merged 11 commits intomasterfrom
yoenn.burban/fix-task-transform-utilization
Mar 27, 2026
Merged

fix(metrics): deduct downstream utilization on task transforms#24731
gwenaskell merged 11 commits intomasterfrom
yoenn.burban/fix-task-transform-utilization

Conversation

@gwenaskell
Copy link
Copy Markdown
Contributor

@gwenaskell gwenaskell commented Feb 25, 2026

Summary

This PR fixes task transforms utilization reporting by sending a utilization start_wait when the component downstream finishes polling the transform, and a stop_wait when it starts polling again. This ensures that the time elapsed while the transform is idle because the component downstream is busy is reported as idle and not busy, which is already the case for sync/function transforms.

Note: as of now, there is no task transform that spawns a task internally, i.e. they are not run concurrently (https://vector.dev/docs/architecture/concurrency-model/, #11857).
If that becomes a thing in the future, there might be situations where this fix makes the utilization report 0 while the component is doing CPU work, because the component downstream is also busy; but this is already what happens for sync and task transforms.

Vector configuration

How did you test this PR?

Added a unit test that replicates a task transform and measures the utilization in different situations.

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

  • Closes: 24641
  • Closes: OPA-4673

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@gwenaskell gwenaskell requested a review from a team as a code owner February 25, 2026 13:06
@github-actions github-actions Bot added the domain: topology Anything related to Vector's topology code label Feb 25, 2026
@gwenaskell gwenaskell changed the title add output utilization on task transforms fix(utilization): add output utilization on task transforms Feb 25, 2026
@gwenaskell gwenaskell changed the title fix(utilization): add output utilization on task transforms fix(utilization): deduct downstream utilization on task transforms Feb 25, 2026
@gwenaskell gwenaskell changed the title fix(utilization): deduct downstream utilization on task transforms fix(metrics): deduct downstream utilization on task transforms Feb 25, 2026
Copy link
Copy Markdown
Member

@bruceg bruceg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me a couple of reads, because the logic is subtle, but I agree with this approach. A couple small nits below.

Comment thread src/utilization.rs Outdated
Comment thread src/utilization.rs Outdated
Comment thread src/utilization.rs Outdated
@pront
Copy link
Copy Markdown
Member

pront commented Mar 18, 2026

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@gwenaskell gwenaskell requested a review from pront March 23, 2026 08:52
@pront
Copy link
Copy Markdown
Member

pront commented Mar 23, 2026

Hye @gwenaskell, I am going to review this today. Need some time to evaluate future implications of this PR.

Copy link
Copy Markdown
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good. Left some suggestions for improving the changelog and rustdocs.

Comment thread src/utilization.rs Outdated
Comment thread src/utilization.rs
Comment thread changelog.d/24641-task-transform-utilization.fix.md Outdated
gwenaskell and others added 4 commits March 27, 2026 12:22
@gwenaskell gwenaskell requested a review from pront March 27, 2026 11:29
@gwenaskell gwenaskell enabled auto-merge March 27, 2026 11:29
@gwenaskell gwenaskell removed the request for review from pront March 27, 2026 11:29
@gwenaskell gwenaskell added this pull request to the merge queue Mar 27, 2026
Merged via the queue into master with commit 4e6b49d Mar 27, 2026
61 checks passed
@gwenaskell gwenaskell deleted the yoenn.burban/fix-task-transform-utilization branch March 27, 2026 12:27
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: topology Anything related to Vector's topology code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Utilization metric reports 1 when Task transform waits on downstream channel

3 participants