Support IOBuf Profiler#2497
Merged
Merged
Conversation
9466369 to
b5317ea
Compare
Contributor
|
借问个问题 这种被socket read创建的io block被 其他类似压缩/解压通过tls共享的io block有什么办法强制回收; 如果业务中 收包很大,挂到tls共享出来了,解压/解压通过也可能用这块io buf的block。 可否主动调用这个代码,强制socket归还 block |
Contributor
Author
|
socket 不会一直持有申请的block吧。每个请求解析的时候,从_read_buf cut出一个包长度的数据。直到socket在解析完_read_buf的数据,会调用 为了后续更快地分配block,每个线程的tls缓存block数软上限是8。应该不需要强制回收吧? |
Contributor
|
是否可以给IOBuf开发一个不共享模式,然后通过memory profiler来定位问题? |
Contributor
Author
应该是可以的。 |
c3d6ce4 to
b6775e1
Compare
wwbmmm
reviewed
Feb 26, 2024
b6775e1 to
04efc6f
Compare
Contributor
|
LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What problem does this PR solve?
Issue Number:
Problem Summary: 此前遇到过IOBuf泄漏的情况,从heap profiler看到泄漏的位置是Socket读包的位置,实际上是因为IOBuf的tls缓存复用机制,掩盖了业务代码中存在IOBuf泄漏的问题。
What is changed and the side effects?
Changed:
受Contention Profiler启发,开发一个IOBuf Profiler,将未释放的IOBuf::Block的引用计数操作的调用栈及其操作数量(整数表示持有引用计数,负数表示释放引用计数)展示位调用图。通过可视化的方式,可快速定位到一些可能存在泄漏的模块,缩小排查的范围。
因为数据上报有一定的顺序要求,所以没有使用Contention Profiler的方案(Collector)来上报数据,而是使用 #2492 封装的无锁队列来上报。
该工具可以扩展到其他引用计数相关的数据结构。
Side effects:
Performance effects(性能影响):
Breaking backward compatibility(向后兼容性):
Check List: