Allow users to configure initial buffer size#37786
Merged
bogdandrutu merged 2 commits intoopen-telemetry:mainfrom Feb 10, 2025
Merged
Allow users to configure initial buffer size#37786bogdandrutu merged 2 commits intoopen-telemetry:mainfrom
bogdandrutu merged 2 commits intoopen-telemetry:mainfrom
Conversation
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
70fe2aa to
acaa6d4
Compare
bogdandrutu
commented
Feb 7, 2025
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
acaa6d4 to
0ddbb6f
Compare
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
0ddbb6f to
ad85a6a
Compare
djaglowski
suggested changes
Feb 10, 2025
Member
djaglowski
left a comment
djaglowski
left a comment
There was a problem hiding this comment.
I don't see any reason why initial buffer size should be fixed for header, or why header buffer size is applicable to the body of the file.
This comment was marked as resolved.
This comment was marked as resolved.
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
ad85a6a to
39a098b
Compare
This comment was marked as resolved.
This comment was marked as resolved.
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
39a098b to
74d3f72
Compare
Member
Author
|
@djaglowski PTAL |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
bogdandrutu
force-pushed
the
add-initial-buffer
branch
from
74d3f72 to
3c40e88
Compare
Member
Author
|
@djaglowski thanks for the pointers. All updates are in place. |
djaglowski
approved these changes
Feb 10, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Using the filelogreceiver with a system that does not allow for in-progress file reads (holds a lock on the file while writing) and we want to read the entire file (can think of a big proto binary) then decode and construct logs.
Because of this use-case and because of the files being ~5MB we do a lot of re-size of the scanner buffer until we are able to read the whole file/token.
Proposal is to allow customizing the initial buffer size for use-cases like this.