Skip to content

[Question] chuckedMessageRate vs chunkedMessageRate - Is chucked a typo? #10046

Description

@michaeljmarshall

Observation of Typo

Based on looking through the results of a topic stats call, I noticed that there appears to be a typo in one of the keys. Producers use the key chunkedMessageRate while consumers use chuckedMessageRate. Here is an example result from topic stats. Notice that the first uses chunk and the second uses chuck.

It looks like the first possible typo was introduced in June, 2020 by #4400, and it also looks like the PIP has a single typo where it refers to maxPendingChuckedMessage: https://github.com/apache/pulsar/wiki/PIP-37:-Large-message-size-handling-in-Pulsar#client-changes. Otherwise, the PIP only uses chunk.

After digging into this, I am pretty sure that chucked is a typo, but I think I might also be wrong about that. Please let me know if chucked was intentionally used.

Examples

Here are two examples where chuck and chunk are used in close proximity, as if they are interchangeable.

@Parameter(names = { "-mc", "--max_chunked_msg" }, description = "Max pending chunk messages")
private int maxPendingChuckedMessage = 0;

/**
* Consumer buffers chunk messages into memory until it receives all the chunks of the original message. While
* consuming chunk-messages, chunks from same message might not be contiguous in the stream and they might be mixed
* with other messages' chunks. so, consumer has to maintain multiple buffers to manage chunks coming from different
* messages. This mainly happens when multiple publishers are publishing messages on the topic concurrently or
* publisher failed to publish all chunks of the messages.
*
* <pre>
* eg: M1-C1, M2-C1, M1-C2, M2-C2
* Here, Messages M1-C1 and M1-C2 belong to original message M1, M2-C1 and M2-C2 messages belong to M2 message.
* </pre>
* Buffering large number of outstanding uncompleted chunked messages can create memory pressure and it can be
* guarded by providing this @maxPendingChuckedMessage threshold. Once, consumer reaches this threshold, it drops
* the outstanding unchunked-messages by silently acking or asking broker to redeliver later by marking it unacked.
* This behavior can be controlled by configuration: @autoAckOldestChunkedMessageOnQueueFull
*
* @default 100
*
* @param maxPendingChuckedMessage
* @return
*/
ConsumerBuilder<T> maxPendingChuckedMessage(int maxPendingChuckedMessage);
/**
* Buffering large number of outstanding uncompleted chunked messages can create memory pressure and it can be
* guarded by providing this @maxPendingChuckedMessage threshold. Once, consumer reaches this threshold, it drops
* the outstanding unchunked-messages by silently acking if autoAckOldestChunkedMessageOnQueueFull is true else it
* marks them for redelivery.
*
* @default false
*
* @param autoAckOldestChunkedMessageOnQueueFull
* @return
*/
ConsumerBuilder<T> autoAckOldestChunkedMessageOnQueueFull(boolean autoAckOldestChunkedMessageOnQueueFull);

Proposal

If chucked is confirmed to be a typo, which I think it is, I am wondering what the right path forward is for fixing a typo that is in the public api. I believe a simple update would count as a breaking change, so I am thinking we should add a @deprecated flag where possible as well as add the correctly spelled method/variable to whatever classes. In the JSON, is it worth supplying it under both the correct and the typo'ed name? I am definitely open to suggestions.

I'm happy to help work this issue, if we confirm that it is in fact a typo.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/enhancementThe enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions