[exporter/kafka] Kafka exporter metrics do not contain kafka broker hostname information

[anubhav21sharma]

Component(s)

exporter/kafka

Is your feature request related to a problem? Please describe.

The kafka exporter provides a great set of metrics to monitor the performance of the kafka producer. The current set of metrics contain the label node_id to differentiate between kafka nodes, but that becomes ambiguous if your otel pipeline is exporting data to multiple kafka clusters and the clusters share the same broker id convention (which I believe is the case in many organizations).

Describe the solution you'd like

I propose to include a new label node_name to each of these metrics that also capture the hostname of the broker.

So, a metric set like this -

otelcol_kafka_exporter_write_latency_seconds_count{node_id="1",otel_scope_name="github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter",otel_scope_schema_url="",otel_scope_version="",outcome="success"} 2636

otelcol_kafka_exporter_write_latency_seconds_count{node_id="2",otel_scope_name="github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter",otel_scope_schema_url="",otel_scope_version="",outcome="success"} 2631

otelcol_kafka_exporter_write_latency_seconds_count{node_id="3",otel_scope_name="github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter",otel_scope_schema_url="",otel_scope_version="",outcome="success"} 2419

becomes

otelcol_kafka_exporter_write_latency_seconds_count{node_id="1",node_name="broker1",otel_scope_name="github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter",otel_scope_schema_url="",otel_scope_version="",outcome="success"} 2636

otelcol_kafka_exporter_write_latency_seconds_count{node_id="2",node_name="broker2",otel_scope_name="github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter",otel_scope_schema_url="",otel_scope_version="",outcome="success"} 2631

otelcol_kafka_exporter_write_latency_seconds_count{node_id="3",node_name="broker3",otel_scope_name="github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter",otel_scope_schema_url="",otel_scope_version="",outcome="success"} 2419

Describe alternatives you've considered

No response

Additional context

No response

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• exporter/kafka: @pavolloffay @MovieStoreGuy @axw @paulojmdias

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

Capturing the broker hostname sounds sensible to me. I think we should use server.address as the attribute, according to https://opentelemetry.io/docs/specs/semconv/messaging/messaging-metrics/

[axw]

@anubhav21sharma would you like to work on this change?

[anubhav21sharma]

@axw Yes, I will take this up. Thank you!