Skip to content

[feat][broker] PIP-264: Add OpenTelemetry producer metrics - #22882

Merged
heesung-sohn merged 24 commits into
apache:masterfrom
dragosvictor:dmisca-pip-264-producer-metrics
Jun 15, 2024
Merged

[feat][broker] PIP-264: Add OpenTelemetry producer metrics#22882
heesung-sohn merged 24 commits into
apache:masterfrom
dragosvictor:dmisca-pip-264-producer-metrics

Conversation

@dragosvictor

Copy link
Copy Markdown
Contributor

PIP-264

Motivation

Adds producer related metrics to the OpenTelemetry pipeline.

Modifications

  • Adds the metrics described in [feat][doc] PIP-264: Add OpenTelemetry producer metrics reference pulsar-site#911 to the collector runs.
  • Relocates the metric backend data-sources (Producer#msgIn, Producer#chunkedMessageRate and Producer#msgDrop) from the Producer class to PublisherStatsImpl and NonPersistentPublisherStatsImpl, for a cleaner design.
  • Add field Rate.totalValueAdder keeping track of cumulative values added to the Rate, further simplifying the design.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

  • Added basic messaging test in OpenTelemetryProducerStatsTest
  • Updated existing test `` to cover the message drop counter functionality

Does this pull request potentially affect one of the following parts:

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: dragosvictor#25

@dragosvictor
dragosvictor marked this pull request as ready for review June 7, 2024 23:52
@github-actions github-actions Bot added the doc-required Your PR changes impact docs and you will update later. label Jun 7, 2024
@lhotari lhotari added this to the 3.4.0 milestone Jun 9, 2024
@heesung-sohn
heesung-sohn merged commit f83dbe9 into apache:master Jun 15, 2024
@dragosvictor
dragosvictor deleted the dmisca-pip-264-producer-metrics branch June 15, 2024 00:49
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
Demogorgon314 pushed a commit to Demogorgon314/kop that referenced this pull request Apr 15, 2026
…osition implementation for compaction (streamnative#606)

### Motivation

apache/pulsar#22891 introduces some changes for
the PositionImpl. Now we need to use the Position interface instead of
the PositionImpl.

### Modifications

- Refactor to use Position instead of PositionImpl
- apache/pulsar#22572 uses
`getLastDispatchablePosition` instead of `getMaxPosition` to return the
last message id, this patch implements the new method for
`KopPersistentTopic`.
- apache/pulsar#22838 changed the result for
`checkTopicExists`. This PR also adapts to changes.
- apache/pulsar#22882 removed the method
`org.apache.pulsar.broker.service.Producer.updateRates(int
numOfMessages, long msgSizeInBytes)`. We need to use
`producer.getStats().recordMsgIn` instead.

---------

Co-authored-by: Yunze Xu <xyzinfernity@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/metrics doc-required Your PR changes impact docs and you will update later. ready-to-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants