Skip to content

[pulsar-proxy] Fix memory leak with debug log-level - #7963

Merged
merlimat merged 1 commit into
apache:masterfrom
rdhabalia:proxy_leak2
Oct 29, 2020
Merged

[pulsar-proxy] Fix memory leak with debug log-level#7963
merlimat merged 1 commit into
apache:masterfrom
rdhabalia:proxy_leak2

Conversation

@rdhabalia

Copy link
Copy Markdown
Contributor

Motivation

Right now, Proxy has memory leak debug log-level=2 because proxy parses RawMessage and doesn't release the payload. It can be easily and quickly reproducible by enabling proxyLogLevel=2 and publish messages across 10 topics.

#2:
	io.netty.buffer.AdvancedLeakAwareByteBuf.readByte(AdvancedLeakAwareByteBuf.java:400)
	org.apache.pulsar.common.util.protobuf.ByteBufCodedInputStream.readRawVarint64(ByteBufCodedInputStream.java:251)
	org.apache.pulsar.common.util.protobuf.ByteBufCodedInputStream.readUInt64(ByteBufCodedInputStream.java:218)
	org.apache.pulsar.common.api.proto.PulsarApi$MessageMetadata$Builder.mergeFrom(PulsarApi.java:4067)
	org.apache.pulsar.common.protocol.Commands.parseMessageMetadata(Commands.java:378)
	org.apache.pulsar.common.api.raw.MessageParser.parseMessage(MessageParser.java:65)
	org.apache.pulsar.proxy.server.ParserProxyHandler.channelRead(ParserProxyHandler.java:126)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:648)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:583)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:500)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
	io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	java.lang.Thread.run(Thread.java:745)

Modification

Release RawMessage payload to avoud memory leak.

@rdhabalia rdhabalia added this to the 2.7.0 milestone Sep 3, 2020
@rdhabalia rdhabalia self-assigned this Sep 3, 2020
@rdhabalia

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants