[improve][monitor] Upgrade OpenTelemetry libraries - #25662
Merged
Conversation
The opentelemetry-runtime-telemetry-java17 module and its RuntimeMetrics/RuntimeMetricsBuilder API were deprecated in opentelemetry-java-instrumentation 2.27.0 in favor of the unified opentelemetry-runtime-telemetry module's RuntimeTelemetry API. Switch the dependency and update OpenTelemetryService to call Internal.setDisableAllJfrFeatures + Experimental.setEmitExperimentalMetrics on the new builder, preserving the previous JMX-only, experimental-enabled behavior.
3 tasks
The OpenTelemetry 1.61.0 upgrade transitively bumps the bundled prometheus-metrics-* jars from 1.3.10 to 1.5.1.
Member
Author
|
Similar change in BookKeeper: apache/bookkeeper#4773 |
In opentelemetry-java 1.61 the BatchSpanProcessor's "queueSize" self-monitoring gauge is registered lazily on the first addSpan(...) call (via LegacySpanProcessorInstrumentation.buildQueueMetricsOnce); in 1.56 it was registered eagerly in the BatchSpanProcessor constructor. Pulsar emits no spans, so under 1.61 the gauge never appears at the OTel collector and the test times out waiting for queueSize_ratio. Assert on jvm_cpu_count instead. It is unconditionally emitted by RuntimeTelemetry, which OpenTelemetryService always configures, and the sibling testOpenTelemetryMetricsPrometheusExport already uses the same metric.
merlimat
approved these changes
May 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Keep Pulsar's OpenTelemetry stack on supported, non-deprecated APIs by upgrading the three OTel artifacts together:
opentelemetry-java: 1.56.0 → 1.61.0opentelemetry-java-instrumentation: 2.21.0 → 2.27.0opentelemetry-semantic-conventions: 1.37.0 → 1.41.0The instrumentation 2.27.0 release deprecates the
opentelemetry-runtime-telemetry-java17module (and itsRuntimeMetrics/RuntimeMetricsBuilderAPI) in favor of the unifiedopentelemetry-runtime-telemetrymodule'sRuntimeTelemetryAPI, so this PR also migratesOpenTelemetryServiceoff that deprecated API.Modifications
gradle/libs.versions.toml: bump the three OTel version refs and switch the runtime-telemetry library coordinate fromopentelemetry-runtime-telemetry-java17to the unifiedopentelemetry-runtime-telemetry.pulsar-opentelemetry/build.gradle.kts: use the renamed library alias.pulsar-opentelemetry/.../OpenTelemetryService.java: replaceRuntimeMetrics.builder(...).disableAllFeatures().emitExperimentalTelemetry().build()withRuntimeTelemetry.builder(...)plusInternal.setDisableAllJfrFeatures(builder, true)andExperimental.setEmitExperimentalMetrics(builder, true)(the replacements named in the deprecation javadoc), preserving the existing JMX-only, experimental-enabled behavior.distribution/server/src/assemble/LICENSE.bin.txt: update the bundled-jar list (the tworuntime-telemetry-java17/-java8entries collapse into a singleruntime-telemetryjar — verified via:dependenciesagainstpulsar-broker,pulsar-proxy,pulsar-functions-worker,pulsar-testclient, andpulsar-opentelemetryruntime classpaths).A clean recompile of all OTel-touching modules (
pulsar-opentelemetry,pulsar-broker,pulsar-client-original,pulsar-broker-common,pulsar-metadata,pulsar-proxy,pulsar-functions-worker,pulsar-transaction-coordinator,pulsar-testclient) emits zero OTel-related deprecation warnings after this change.Verifying this change
This change is already covered by existing tests, specifically
pulsar-opentelemetry'sOpenTelemetryServiceTest— includingtestJvmRuntimeMetrics, which asserts that JMX-sourced runtime metrics (jvm.cpu.time,jvm.memory.used,jvm.gc.duration,jvm.thread.count,jvm.class.count, etc.) are still emitted after the runtime-telemetry API switch.Does this pull request potentially affect one of the following parts:
The OpenTelemetry Java SDK, instrumentation, and semantic-conventions artifacts are upgraded; the runtime-telemetry dependency coordinate changes from
opentelemetry-runtime-telemetry-java17to the unifiedopentelemetry-runtime-telemetry. No public Pulsar API changes.