Skip to content

refactor(file source): Use Tokio JoinMap - #26432

Open
pront wants to merge 2 commits into
masterfrom
pront-extract-file-source-task-set
Open

pront wants to merge 2 commits into
masterfrom
pront-extract-file-source-task-set

Conversation

@pront

@pront pront commented Sep 18, 2026

Copy link
Copy Markdown
Member

Motivation

File cleanup must retain the path when a task fails or panics. Use Tokio's existing JoinMap to handle that bookkeeping, preparing the same approach for the notification-driven file source.

Summary

Replace the cleanup loop's JoinSet and task-ID map with tokio_util::task::JoinMap. Enable join-map on the existing dependency; preserve deletion and error reporting behavior.

References

Preliminary extraction from #26429.

Vector configuration

No configuration changes.

How did you test this PR?

  • All 26 file-source and file-source-common tests passed.
  • Applied JoinMap to both ifile call sites locally: all 14 E2E tests passed, including truncate/regrow and restart coverage. Both call sites use unique paths.
  • Affected-crate Clippy, formatting, and license generation passed. The ifile workspace Clippy check also passed with ifile-tests,sources-file enabled.
  • The preliminary branch's workspace Clippy check still encounters master's unchanged unused_async error at src/test_util/http.rs:51.

Does this PR include user facing changes?

  • No.

Contributor Guidelines

Followed the repository contribution guidance.

@pront pront added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label Sep 18, 2026
@pront
pront marked this pull request as ready for review September 18, 2026 14:39
@pront
pront requested a review from a team as a code owner September 18, 2026 14:39
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 18, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-18T15:18:29.960300Z e7ab510 New commits
🔒 Security Review Completed 2026-09-18T15:18:54.778398Z e7ab510 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb5fc5f4cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/file-source-common/src/task_set.rs Outdated
@pront pront changed the title refactor(file source): Share keyed task handling refactor(file source): Use Tokio JoinMap Sep 18, 2026
@pront
pront enabled auto-merge September 18, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog Changes in this PR do not need user-facing explanations in the release changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant