Define telemetry stability guarantees

[tedsuo]

It is critical that OpenTelemetry produces telemetry which remains stable. Changes to telemetry produced by OpenTelemetry instrumentation should avoid breaking analysis tools, such as dashboards and alerts. However, it is not clear at this time what type of instrumentation changes (for example, adding additional spans and labels) would actually cause a breaking change.

Related questions:

• What types of changes are known to definitely break the analysis tools which currently target OpenTelemetry?
• Better is always different. If we must make a breaking change in order to offer better instrumentation, what should the procedure be?
• How should we convey changes in telemetry to operators and application owners?
• How should auto-installers handle cases where application owners do not wish to install the latest version of an instrumentation library, in order to manage breaking changes?

Until this issue is resolved, instrumentation packages must not be marked as stable. API and SDK packages may still be marked as stable. The lack of telemetry stability should be clearly communicated in the documentation for every OpenTelemetry client, to avoid confusion.

[tigrannajaryan]

What constitutes a breaking change

For metrics here is an incomplete list of changes that may be breaking for readers of the telemetry (e.g. for dashboards in the backends):

• Renaming of the metric or of a metric label.
• Elimination of a label (dimension) of a metric.
• Introduction of a label to a metric.
• Splitting of a metric into multiple metric across a label dimension.
• Combining of multiple metrics into one metric with an additional label dimension.
• Change of value data type, aggregation or temporality.
• Change of the unit.
• Change of the scale of the values.

For spans:

• Change of the span name.
• Renaming of an attribute.
• Elimination of an attribute.
• Change of the data type of the attribute value.
• Possibly, addition of an attribute.
• Change of an event name or change of an attribute as described above.

For logs:

• Change of the log record name.
• Renaming of an attribute.
• Elimination of an attribute.
• Change of the data type of the attribute value.
• Possibly, addition of an attribute.
• Change of the body (similar to attribute changes).

[EDIT] We will also need to define relevant changes for Resources.

For future discussions I suggest we call the shape and structure of all of the data that is referenced above the schema of telemetry.

How we handle the changes

Unlike API, I believe changes described above are more likely to occur during the lifelime of an instrumentation library. I do not think we should aim to lock the telemetry schema and disallow changes listed above. Such locking would place a huge limitation on how instrumentation can evolve and would make it nearly impossible to fix mistakes in the semantic conventions, in the schema or in the implementation of the instrumentation (which will inevitably happen sooner or later).

[EDIT] I extracted full proposal here #1324 to avoid derailing this issue.

In the context of this issue here is a shorter summary:

• For 1.0 GA we do not provide stability guarantees for telemetry schema emitted by instrumentation.
• We introduce telemetry schema concept which allows instrumentation to change emitted data structure and allows recipients to treat in a backwards compatible manner.
• Telemetry schema will become available as a concept in one of 1.x releases. Until then OpenTelemetry users should be advised that breaking changes to instrumentation may happen (however, we will try to minimize such changes until the telemetry schemas are introduced).

[tedsuo]

Thanks @tigrannajaryan. After more conversations I agree. We should issue a v1.0 without these data guarantees. We can't even begin to address this until everything else in tracing and metrics is complete. I've removed the required-for-ga label.