fix: perform drain when file is deleted in tail package#5139
Merged
Conversation
thampiotr
reviewed
Dec 18, 2025
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses a potential data loss issue during file rotation in the tail package. The main change introduces a drain() function that reads all remaining lines from an old file handle before it's closed during rotation, storing them in a buffer to be returned before reading from the new file.
Key changes:
- Added buffering mechanism (
bufferedLines) to preserve lines from rotated files - Implemented
drain()function to read remaining lines when file deletion is detected - Enhanced logging consistency by adding path to logger context and standardizing error log keys
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| internal/component/loki/source/file/internal/tail/file.go | Core implementation of drain functionality, bufferedLines field, and improved debug logging for file events |
| internal/component/loki/source/file/internal/tail/file_test.go | Added test case to verify drain behavior during file rotation and helper function for rotation simulation |
| internal/component/loki/source/file/tailer.go | Logging improvements: added path to logger context and standardized error log key from "err" to "error" |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
ac85f44 to
8c40953
Compare
thampiotr
approved these changes
Dec 18, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description
After the refactor of tail package, and maybe even before that, we have the potential to miss log lines when files are rotated.
If we got to EOF and we detected a delete event we would ignore whats left for that file. It could be that a partial line read before was fully flushed or even more lines.
So to handle this I added
bufferedLines. If we detect a delete event we use the open fd we have and drain all remaning lines into that and then perform the reopen.On reads we check if we have any lines buffered and reads from that first.
Which issue(s) this PR fixes
Notes to the Reviewer
PR Checklist
BEGIN_COMMIT_OVERRIDE
fix: Perform drain when file is deleted in tail package (#5139)
END_COMMIT_OVERRIDE