Skip to content

Make opentelemetry_metrics_exporter entrypoint support pull exporters#3428

Merged
lzchen merged 6 commits intoopen-telemetry:mainfrom
aabmass:reader-or-exporter-3411
Sep 15, 2023
Merged

Make opentelemetry_metrics_exporter entrypoint support pull exporters#3428
lzchen merged 6 commits intoopen-telemetry:mainfrom
aabmass:reader-or-exporter-3411

Conversation

@aabmass
Copy link
Copy Markdown
Member

@aabmass aabmass commented Sep 7, 2023 •
edited
Loading

Description

Update SDK configuration to handle when opentelemetry_metrics_exporter entrypoint points to a MetricReader, which is how we represent pull exporters e.g. Prometheus.

Fixes #3411

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Added new tests cases for configuration components. Will add Prometheus support to fix #2864 and test this out concretely.

Does This PR Require a Contrib Repo Change?

  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated – N/A

@aabmass aabmass force-pushed the reader-or-exporter-3411 branch 2 times, most recently from 06228e6 to 7cced02 Compare September 7, 2023 19:01
@aabmass aabmass marked this pull request as ready for review September 7, 2023 19:01
@aabmass aabmass requested a review from a team September 7, 2023 19:01
@aabmass aabmass force-pushed the reader-or-exporter-3411 branch from 7cced02 to b5f3f50 Compare September 7, 2023 19:03
@ocelotl
Copy link
Copy Markdown
Contributor

ocelotl commented Sep 8, 2023

This is smart, saves us from breaking changes and/or a much more complicated solution 👍

In practical terms, this change is aimed at opentelemetry-instrument users, so I think it would make sense to also add some documentation in the corresponding contrib component so that when --help is used with that command there is some information that lets the user know they can use --metrics-exporter with a metrics reader entry point as well.

@ocelotl
Copy link
Copy Markdown
Contributor

ocelotl commented Sep 8, 2023

BTW, I noticed that the OTEL_*_EXPORTER environment variables are defined in the API. Shouldn't they be in the SDK instead?

@aabmass
Copy link
Copy Markdown
Member Author

aabmass commented Sep 11, 2023

BTW, I noticed that the OTEL_*_EXPORTER environment variables are defined in the API. Shouldn't they be in the SDK instead?

Possibly but it would be a breaking change

@aabmass
Copy link
Copy Markdown
Member Author

aabmass commented Sep 11, 2023 •
edited
Loading

In practical terms, this change is aimed at opentelemetry-instrument users, so I think it would make sense to also add some documentation in the corresponding contrib component so that when --help is used with that command there is some information that lets the user know they can use --metrics-exporter with a metrics reader entry point as well.

@ocelotl I took a look at the output of opentelemetry-instrument --help:

--help output 

usage: opentelemetry-instrument [-h] [--logs_exporter LOGS_EXPORTER]
                                [--metrics_exporter METRICS_EXPORTER]
                                [--propagators PROPAGATORS]
                                [--context CONTEXT]
                                [--id_generator ID_GENERATOR]
                                [--meter_provider METER_PROVIDER]
                                [--tracer_provider TRACER_PROVIDER]
                                [--traces_exporter TRACES_EXPORTER]
                                [--configurator CONFIGURATOR]
                                [--disabled_instrumentations DISABLED_INSTRUMENTATIONS]
                                [--distro DISTRO]
                                [--attribute_count_limit ATTRIBUTE_COUNT_LIMIT]
                                [--attribute_value_length_limit ATTRIBUTE_VALUE_LENGTH_LIMIT]
                                [--blrp_export_timeout BLRP_EXPORT_TIMEOUT]
                                [--blrp_max_export_batch_size BLRP_MAX_EXPORT_BATCH_SIZE]
                                [--blrp_max_queue_size BLRP_MAX_QUEUE_SIZE]
                                [--blrp_schedule_delay BLRP_SCHEDULE_DELAY]
                                [--bsp_export_timeout BSP_EXPORT_TIMEOUT]
                                [--bsp_max_export_batch_size BSP_MAX_EXPORT_BATCH_SIZE]
                                [--bsp_max_queue_size BSP_MAX_QUEUE_SIZE]
                                [--bsp_schedule_delay BSP_SCHEDULE_DELAY]
                                [--event_attribute_count_limit EVENT_ATTRIBUTE_COUNT_LIMIT]
                                [--experimental_resource_detectors EXPERIMENTAL_RESOURCE_DETECTORS]
                                [--exporter_jaeger_agent_host EXPORTER_JAEGER_AGENT_HOST]
                                [--exporter_jaeger_agent_port EXPORTER_JAEGER_AGENT_PORT]
                                [--exporter_jaeger_agent_split_oversized_batches EXPORTER_JAEGER_AGENT_SPLIT_OVERSIZED_BATCHES]
                                [--exporter_jaeger_certificate EXPORTER_JAEGER_CERTIFICATE]
                                [--exporter_jaeger_endpoint EXPORTER_JAEGER_ENDPOINT]
                                [--exporter_jaeger_grpc_insecure EXPORTER_JAEGER_GRPC_INSECURE]
                                [--exporter_jaeger_password EXPORTER_JAEGER_PASSWORD]
                                [--exporter_jaeger_timeout EXPORTER_JAEGER_TIMEOUT]
                                [--exporter_jaeger_user EXPORTER_JAEGER_USER]
                                [--exporter_otlp_certificate EXPORTER_OTLP_CERTIFICATE]
                                [--exporter_otlp_compression EXPORTER_OTLP_COMPRESSION]
                                [--exporter_otlp_endpoint EXPORTER_OTLP_ENDPOINT]
                                [--exporter_otlp_headers EXPORTER_OTLP_HEADERS]
                                [--exporter_otlp_insecure EXPORTER_OTLP_INSECURE]
                                [--exporter_otlp_logs_certificate EXPORTER_OTLP_LOGS_CERTIFICATE]
                                [--exporter_otlp_logs_compression EXPORTER_OTLP_LOGS_COMPRESSION]
                                [--exporter_otlp_logs_endpoint EXPORTER_OTLP_LOGS_ENDPOINT]
                                [--exporter_otlp_logs_headers EXPORTER_OTLP_LOGS_HEADERS]
                                [--exporter_otlp_logs_insecure EXPORTER_OTLP_LOGS_INSECURE]
                                [--exporter_otlp_logs_protocol EXPORTER_OTLP_LOGS_PROTOCOL]
                                [--exporter_otlp_logs_timeout EXPORTER_OTLP_LOGS_TIMEOUT]
                                [--exporter_otlp_metrics_certificate EXPORTER_OTLP_METRICS_CERTIFICATE]
                                [--exporter_otlp_metrics_client_certificate EXPORTER_OTLP_METRICS_CLIENT_CERTIFICATE]
                                [--exporter_otlp_metrics_client_key EXPORTER_OTLP_METRICS_CLIENT_KEY]
                                [--exporter_otlp_metrics_compression EXPORTER_OTLP_METRICS_COMPRESSION]
                                [--exporter_otlp_metrics_default_histogram_aggregation EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION]
                                [--exporter_otlp_metrics_endpoint EXPORTER_OTLP_METRICS_ENDPOINT]
                                [--exporter_otlp_metrics_headers EXPORTER_OTLP_METRICS_HEADERS]
                                [--exporter_otlp_metrics_insecure EXPORTER_OTLP_METRICS_INSECURE]
                                [--exporter_otlp_metrics_protocol EXPORTER_OTLP_METRICS_PROTOCOL]
                                [--exporter_otlp_metrics_temporality_preference EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE]
                                [--exporter_otlp_metrics_timeout EXPORTER_OTLP_METRICS_TIMEOUT]
                                [--exporter_otlp_protocol EXPORTER_OTLP_PROTOCOL]
                                [--exporter_otlp_timeout EXPORTER_OTLP_TIMEOUT]
                                [--exporter_otlp_traces_certificate EXPORTER_OTLP_TRACES_CERTIFICATE]
                                [--exporter_otlp_traces_compression EXPORTER_OTLP_TRACES_COMPRESSION]
                                [--exporter_otlp_traces_endpoint EXPORTER_OTLP_TRACES_ENDPOINT]
                                [--exporter_otlp_traces_headers EXPORTER_OTLP_TRACES_HEADERS]
                                [--exporter_otlp_traces_insecure EXPORTER_OTLP_TRACES_INSECURE]
                                [--exporter_otlp_traces_protocol EXPORTER_OTLP_TRACES_PROTOCOL]
                                [--exporter_otlp_traces_timeout EXPORTER_OTLP_TRACES_TIMEOUT]
                                [--exporter_prometheus_host EXPORTER_PROMETHEUS_HOST]
                                [--exporter_prometheus_port EXPORTER_PROMETHEUS_PORT]
                                [--exporter_zipkin_endpoint EXPORTER_ZIPKIN_ENDPOINT]
                                [--exporter_zipkin_timeout EXPORTER_ZIPKIN_TIMEOUT]
                                [--link_attribute_count_limit LINK_ATTRIBUTE_COUNT_LIMIT]
                                [--log_level LOG_LEVEL]
                                [--metrics_exemplar_filter METRICS_EXEMPLAR_FILTER]
                                [--metric_export_interval METRIC_EXPORT_INTERVAL]
                                [--metric_export_timeout METRIC_EXPORT_TIMEOUT]
                                [--resource_attributes RESOURCE_ATTRIBUTES]
                                [--service_name SERVICE_NAME]
                                [--span_attribute_count_limit SPAN_ATTRIBUTE_COUNT_LIMIT]
                                [--span_attribute_value_length_limit SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT]
                                [--span_event_count_limit SPAN_EVENT_COUNT_LIMIT]
                                [--span_link_count_limit SPAN_LINK_COUNT_LIMIT]
                                [--traces_sampler TRACES_SAMPLER]
                                [--traces_sampler_arg TRACES_SAMPLER_ARG]
                                [--version]
                                command ...

opentelemetry-instrument automatically instruments a Python program and its

dependencies and then runs the program.


positional arguments:

command               Your Python application.

command_args          Arguments for your application.


options:

-h, --help            show this help message and exit

--logs_exporter LOGS_EXPORTER

--metrics_exporter METRICS_EXPORTER

--propagators PROPAGATORS

--context CONTEXT

--id_generator ID_GENERATOR

--meter_provider METER_PROVIDER

--tracer_provider TRACER_PROVIDER

--traces_exporter TRACES_EXPORTER

--configurator CONFIGURATOR

--disabled_instrumentations DISABLED_INSTRUMENTATIONS

--distro DISTRO

--attribute_count_limit ATTRIBUTE_COUNT_LIMIT

--attribute_value_length_limit ATTRIBUTE_VALUE_LENGTH_LIMIT

--blrp_export_timeout BLRP_EXPORT_TIMEOUT

--blrp_max_export_batch_size BLRP_MAX_EXPORT_BATCH_SIZE

--blrp_max_queue_size BLRP_MAX_QUEUE_SIZE

--blrp_schedule_delay BLRP_SCHEDULE_DELAY

--bsp_export_timeout BSP_EXPORT_TIMEOUT

--bsp_max_export_batch_size BSP_MAX_EXPORT_BATCH_SIZE

--bsp_max_queue_size BSP_MAX_QUEUE_SIZE

--bsp_schedule_delay BSP_SCHEDULE_DELAY

--event_attribute_count_limit EVENT_ATTRIBUTE_COUNT_LIMIT

--experimental_resource_detectors EXPERIMENTAL_RESOURCE_DETECTORS

--exporter_jaeger_agent_host EXPORTER_JAEGER_AGENT_HOST

--exporter_jaeger_agent_port EXPORTER_JAEGER_AGENT_PORT

--exporter_jaeger_agent_split_oversized_batches EXPORTER_JAEGER_AGENT_SPLIT_OVERSIZED_BATCHES

--exporter_jaeger_certificate EXPORTER_JAEGER_CERTIFICATE

--exporter_jaeger_endpoint EXPORTER_JAEGER_ENDPOINT

--exporter_jaeger_grpc_insecure EXPORTER_JAEGER_GRPC_INSECURE

--exporter_jaeger_password EXPORTER_JAEGER_PASSWORD

--exporter_jaeger_timeout EXPORTER_JAEGER_TIMEOUT

--exporter_jaeger_user EXPORTER_JAEGER_USER

--exporter_otlp_certificate EXPORTER_OTLP_CERTIFICATE

--exporter_otlp_compression EXPORTER_OTLP_COMPRESSION

--exporter_otlp_endpoint EXPORTER_OTLP_ENDPOINT

--exporter_otlp_headers EXPORTER_OTLP_HEADERS

--exporter_otlp_insecure EXPORTER_OTLP_INSECURE

--exporter_otlp_logs_certificate EXPORTER_OTLP_LOGS_CERTIFICATE

--exporter_otlp_logs_compression EXPORTER_OTLP_LOGS_COMPRESSION

--exporter_otlp_logs_endpoint EXPORTER_OTLP_LOGS_ENDPOINT

--exporter_otlp_logs_headers EXPORTER_OTLP_LOGS_HEADERS

--exporter_otlp_logs_insecure EXPORTER_OTLP_LOGS_INSECURE

--exporter_otlp_logs_protocol EXPORTER_OTLP_LOGS_PROTOCOL

--exporter_otlp_logs_timeout EXPORTER_OTLP_LOGS_TIMEOUT

--exporter_otlp_metrics_certificate EXPORTER_OTLP_METRICS_CERTIFICATE

--exporter_otlp_metrics_client_certificate EXPORTER_OTLP_METRICS_CLIENT_CERTIFICATE

--exporter_otlp_metrics_client_key EXPORTER_OTLP_METRICS_CLIENT_KEY

--exporter_otlp_metrics_compression EXPORTER_OTLP_METRICS_COMPRESSION

--exporter_otlp_metrics_default_histogram_aggregation EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION

--exporter_otlp_metrics_endpoint EXPORTER_OTLP_METRICS_ENDPOINT

--exporter_otlp_metrics_headers EXPORTER_OTLP_METRICS_HEADERS

--exporter_otlp_metrics_insecure EXPORTER_OTLP_METRICS_INSECURE

--exporter_otlp_metrics_protocol EXPORTER_OTLP_METRICS_PROTOCOL

--exporter_otlp_metrics_temporality_preference EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE

--exporter_otlp_metrics_timeout EXPORTER_OTLP_METRICS_TIMEOUT

--exporter_otlp_protocol EXPORTER_OTLP_PROTOCOL

--exporter_otlp_timeout EXPORTER_OTLP_TIMEOUT

--exporter_otlp_traces_certificate EXPORTER_OTLP_TRACES_CERTIFICATE

--exporter_otlp_traces_compression EXPORTER_OTLP_TRACES_COMPRESSION

--exporter_otlp_traces_endpoint EXPORTER_OTLP_TRACES_ENDPOINT

--exporter_otlp_traces_headers EXPORTER_OTLP_TRACES_HEADERS

--exporter_otlp_traces_insecure EXPORTER_OTLP_TRACES_INSECURE

--exporter_otlp_traces_protocol EXPORTER_OTLP_TRACES_PROTOCOL

--exporter_otlp_traces_timeout EXPORTER_OTLP_TRACES_TIMEOUT

--exporter_prometheus_host EXPORTER_PROMETHEUS_HOST

--exporter_prometheus_port EXPORTER_PROMETHEUS_PORT

--exporter_zipkin_endpoint EXPORTER_ZIPKIN_ENDPOINT

--exporter_zipkin_timeout EXPORTER_ZIPKIN_TIMEOUT

--link_attribute_count_limit LINK_ATTRIBUTE_COUNT_LIMIT

--log_level LOG_LEVEL

--metrics_exemplar_filter METRICS_EXEMPLAR_FILTER

--metric_export_interval METRIC_EXPORT_INTERVAL

--metric_export_timeout METRIC_EXPORT_TIMEOUT

--resource_attributes RESOURCE_ATTRIBUTES

--service_name SERVICE_NAME

--span_attribute_count_limit SPAN_ATTRIBUTE_COUNT_LIMIT

--span_attribute_value_length_limit SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT

--span_event_count_limit SPAN_EVENT_COUNT_LIMIT

--span_link_count_limit SPAN_LINK_COUNT_LIMIT

--traces_sampler TRACES_SAMPLER

--traces_sampler_arg TRACES_SAMPLER_ARG

--version             print version information


Optional arguments (except for --help and --version) for opentelemetry-

instrument directly correspond with OpenTelemetry environment variables. The

corresponding optional argument is formed by removing the OTEL_ or

OTEL_PYTHON_ prefix from the environment variable and lower casing the rest.

For example, the optional argument --attribute_value_length_limit corresponds

with the environment variable OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT. These

optional arguments will override the current value of the corresponding

environment variable during the execution of the command.

It doesn't have any detailed info as the flags are just coming from environment variables. I guess it would make the most sense to just document it along with the environment variable then?

@aabmass
Copy link
Copy Markdown
Member Author

aabmass commented Sep 12, 2023

I updated the sphinx docs for OTEL_METRICS_EXPORTER explaining what's supported and that it's considered experimental

@aabmass
Copy link
Copy Markdown
Member Author

aabmass commented Sep 14, 2023 •
edited
Loading

Test failures look like census-instrumentation/opencensus-python#1219 @lzchen did we do a recent OpenCensus release?

@aabmass
Copy link
Copy Markdown
Member Author

aabmass commented Sep 14, 2023

Fix in this repo for the problem #3433

@lzchen lzchen merged commit 6973de2 into open-telemetry:main Sep 15, 2023
@aabmass aabmass deleted the reader-or-exporter-3411 branch September 16, 2023 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lzchen lzchen lzchen approved these changes

+1 more reviewer

@ocelotl ocelotl ocelotl approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Make opentelemetry_metrics_exporter entrypoint support pull exporters Cannot run opentelemetry-instrument with prometheus as metric exporter

3 participants

@aabmass @ocelotl @lzchen