in_tail: fix DB.compare_filename mismatch when monitoring symlinks#11544
in_tail: fix DB.compare_filename mismatch when monitoring symlinks#11544zdyj3170101136 wants to merge 1 commit intofluent:masterfrom
Conversation
When DB.compare_filename is enabled and the monitored path is a
symbolic link (e.g. glog-style logging), the filename comparison
in flb_tail_target_file_name_cmp() always fails on restart.
The database stores the symlink path (file->name), but the comparison
checks against file->real_name which is resolved via the file descriptor
to the actual target path. For symlinks such as:
app.INFO -> app.INFO.20250303-120000.12345
the basenames differ ("app.INFO" vs "app.INFO.20250303-120000.12345"),
so db_file_exists() returns FLB_FALSE. The existing entry is then
deleted as stale and re-inserted with offset 0, causing a full
re-read of the file and duplicate log ingestion.
Fix this by adding a fallback comparison against file->name (the
original glob-matched path) when the real_name comparison fails.
This correctly handles symlinks while still protecting against
inode reuse for regular files.
Signed-off-by: jie.yang <zdyj3170101136@gmail.com>
📝 WalkthroughWalkthroughAdds a fallback mechanism to Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related issues
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 39ad31904a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@plugins/in_tail/tail_file.h`:
- Around line 103-109: The code compares base names using flb_strdup(file->name)
but when that allocation fails it leaves ret set to the previous compare result,
causing OOM to be treated as a mismatch; change the logic around flb_strdup to
detect NULL and propagate an allocation error (set ret = -1 or return -1 per the
function contract) instead of continuing the comparison. Specifically, in the
block that calls flb_strdup(file->name) (variables: name_b, base_b, flb_strdup,
_stricmp, ret, file->name) ensure that if flb_strdup returns NULL you set ret =
-1 (or return -1 immediately) and do not call basename/_stricmp; mirror the same
change for the second occurrence (lines 126-132) so allocation failures are
propagated consistently.
- Around line 98-110: The fallback comparison that uses file->name should be
limited to symlink-backed files only; modify the blocks in db_file_exists() and
flb_tail_file_is_rotated() where you currently check "if (ret != 0 &&
file->name)" to also require "file->is_link == FLB_TRUE" (i.e. change to "if
(ret != 0 && file->is_link == FLB_TRUE && file->name)"), and apply the same
guard in the analogous fallback at the other location (the block around lines
121-133) so the basename comparison only runs for symlinks.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2a9e0858-0a8a-456d-a5f3-c5660fcd1115
📒 Files selected for processing (1)
plugins/in_tail/tail_file.h
1 participant
When DB.compare_filename is enabled and the monitored path is a symbolic link (e.g. glog-style logging), the filename comparison in flb_tail_target_file_name_cmp() always fails on restart.
The database stores the symlink path (file->name), but the comparison checks against file->real_name which is resolved via the file descriptor to the actual target path. For symlinks such as:
app.INFO -> app.INFO.20250303-120000.12345
the basenames differ ("app.INFO" vs "app.INFO.20250303-120000.12345"), so db_file_exists() returns FLB_FALSE. The existing entry is then deleted as stale and re-inserted with offset 0, causing a full re-read of the file and duplicate log ingestion.
Fix this by adding a fallback comparison against file->name (the original glob-matched path) when the real_name comparison fails. This correctly handles symlinks while still protecting against inode reuse for regular files.
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
Summary by CodeRabbit