Skip to content

Commit 8e0e6f4

Browse files
committed
Add a otelcol.processor.metric_start_time component
1 parent 37656f8 commit 8e0e6f4

File tree

16 files changed

+461
-0
lines changed

16 files changed

+461
-0
lines changed

.github/ISSUE_TEMPLATE/blank.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ body:
116116
- otelcol.processor.interval
117117
- otelcol.processor.k8sattributes
118118
- otelcol.processor.memory_limiter
119+
- otelcol.processor.metric_start_time
119120
- otelcol.processor.probabilistic_sampler
120121
- otelcol.processor.resourcedetection
121122
- otelcol.processor.span

.github/ISSUE_TEMPLATE/bug_report.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ body:
116116
- otelcol.processor.interval
117117
- otelcol.processor.k8sattributes
118118
- otelcol.processor.memory_limiter
119+
- otelcol.processor.metric_start_time
119120
- otelcol.processor.probabilistic_sampler
120121
- otelcol.processor.resourcedetection
121122
- otelcol.processor.span

.github/ISSUE_TEMPLATE/docs.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ body:
119119
- otelcol.processor.interval
120120
- otelcol.processor.k8sattributes
121121
- otelcol.processor.memory_limiter
122+
- otelcol.processor.metric_start_time
122123
- otelcol.processor.probabilistic_sampler
123124
- otelcol.processor.resourcedetection
124125
- otelcol.processor.span

.github/ISSUE_TEMPLATE/feature_request.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ body:
116116
- otelcol.processor.interval
117117
- otelcol.processor.k8sattributes
118118
- otelcol.processor.memory_limiter
119+
- otelcol.processor.metric_start_time
119120
- otelcol.processor.probabilistic_sampler
120121
- otelcol.processor.resourcedetection
121122
- otelcol.processor.span

.github/ISSUE_TEMPLATE/proposal.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ body:
116116
- otelcol.processor.interval
117117
- otelcol.processor.k8sattributes
118118
- otelcol.processor.memory_limiter
119+
- otelcol.processor.metric_start_time
119120
- otelcol.processor.probabilistic_sampler
120121
- otelcol.processor.resourcedetection
121122
- otelcol.processor.span

collector/go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -674,6 +674,7 @@ require (
674674
github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor v0.142.0 // indirect
675675
github.com/open-telemetry/opentelemetry-collector-contrib/processor/intervalprocessor v0.142.0 // indirect
676676
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.142.0 // indirect
677+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstarttimeprocessor v0.142.0 // indirect
677678
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.142.0 // indirect
678679
github.com/open-telemetry/opentelemetry-collector-contrib/processor/tailsamplingprocessor v0.142.0 // indirect
679680
github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor v0.142.0 // indirect

collector/go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1870,6 +1870,8 @@ github.com/open-telemetry/opentelemetry-collector-contrib/processor/intervalproc
18701870
github.com/open-telemetry/opentelemetry-collector-contrib/processor/intervalprocessor v0.142.0/go.mod h1:Ggho2ol6+k+WvicHJqGlD26ARkJt5NEPoz7bdOUYzdE=
18711871
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.142.0 h1:oq+Qai4+JgqvDkbVy9ObqD4p527ObwyDRvtThtT5FZ0=
18721872
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.142.0/go.mod h1:/rXd9kVmBw277+LEDcJGJ/RRb2rVL89hNL95FUaMN9k=
1873+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstarttimeprocessor v0.142.0 h1:RMdnEzbfqBmNI3UT8K5//1Z1HpD8Adq1RIzCItmdgaw=
1874+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstarttimeprocessor v0.142.0/go.mod h1:DbnNU+vFFh3sYuJRfCDRV80fPr1GJT0YSSvqSF/2Q8c=
18731875
github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.142.0 h1:HZMvNV/HhFqUipmTGj1vZj+V6MSG/wQs4p0fAMbDF4o=
18741876
github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.142.0/go.mod h1:Ss0gY7Lj+RZAKKCUp3wzc0JFloI+f0L6OFTX+I003Qs=
18751877
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.142.0 h1:JHsjepWLopcut4j2+NH545OP6V6A7mfFiEKfddOM4Jg=

docs/sources/reference/compatibility/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -341,6 +341,7 @@ The following components, grouped by namespace, _export_ OpenTelemetry `otelcol.
341341
- [otelcol.processor.interval](../components/otelcol/otelcol.processor.interval)
342342
- [otelcol.processor.k8sattributes](../components/otelcol/otelcol.processor.k8sattributes)
343343
- [otelcol.processor.memory_limiter](../components/otelcol/otelcol.processor.memory_limiter)
344+
- [otelcol.processor.metric_start_time](../components/otelcol/otelcol.processor.metric_start_time)
344345
- [otelcol.processor.probabilistic_sampler](../components/otelcol/otelcol.processor.probabilistic_sampler)
345346
- [otelcol.processor.resourcedetection](../components/otelcol/otelcol.processor.resourcedetection)
346347
- [otelcol.processor.span](../components/otelcol/otelcol.processor.span)
@@ -382,6 +383,7 @@ The following components, grouped by namespace, _consume_ OpenTelemetry `otelcol
382383
- [otelcol.processor.interval](../components/otelcol/otelcol.processor.interval)
383384
- [otelcol.processor.k8sattributes](../components/otelcol/otelcol.processor.k8sattributes)
384385
- [otelcol.processor.memory_limiter](../components/otelcol/otelcol.processor.memory_limiter)
386+
- [otelcol.processor.metric_start_time](../components/otelcol/otelcol.processor.metric_start_time)
385387
- [otelcol.processor.probabilistic_sampler](../components/otelcol/otelcol.processor.probabilistic_sampler)
386388
- [otelcol.processor.resourcedetection](../components/otelcol/otelcol.processor.resourcedetection)
387389
- [otelcol.processor.span](../components/otelcol/otelcol.processor.span)
Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
---
2+
canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.processor.metric_start_time/
3+
aliases:
4+
- ../otelcol.processor.metric_start_time/ # /docs/alloy/latest/reference/otelcol.processor.metric_start_time/
5+
description: Learn about otelcol.processor.metric_start_time
6+
labels:
7+
stage: general-availability
8+
products:
9+
- oss
10+
title: otelcol.processor.metric_start_time
11+
---
12+
13+
# `otelcol.processor.metric_start_time`
14+
15+
`otelcol.processor.metric_start_time` accepts metrics from other `otelcol` components and sets the start time for cumulative metric points when the start time is unknown.
16+
This processor is commonly used with `otelcol.receiver.prometheus`, which produces metric points without a [start time][otlp-start-time].
17+
18+
`otelcol.processor.metric_start_time` does not modify metric datapoints which already have a start time,
19+
20+
{{< admonition type="note" >}}
21+
`otelcol.processor.metric_start_time` is a wrapper over the upstream OpenTelemetry Collector [`metricstarttime`][] processor.
22+
Bug reports or feature requests will be redirected to the upstream repository, if necessary.
23+
24+
[`metricstarttime`]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/{{< param "OTEL_VERSION" >}}/processor/metricstarttimeprocessor
25+
{{< /admonition >}}
26+
27+
You can specify multiple `otelcol.processor.metric_start_time` components by giving them different labels.
28+
29+
[otlp-start-time]: https://github.com/open-telemetry/opentelemetry-proto/blob/v1.9.0/opentelemetry/proto/metrics/v1/metrics.proto#L181-L187
30+
31+
## Usage
32+
33+
```alloy
34+
otelcol.processor.metric_start_time "<LABEL>" {
35+
output {
36+
metrics = [...]
37+
}
38+
}
39+
```
40+
41+
## Arguments
42+
43+
You can use the following arguments with `otelcol.processor.metric_start_time`:
44+
45+
| Name | Type | Description | Default | Required |
46+
|---------------------------|------------|-------------------------------------------------------------------------------------------------------|----------------------|----------|
47+
| `strategy` | `string` | Strategy to use for setting start time. | `"true_reset_point"` | no |
48+
| `gc_interval` | `duration` | How long to wait before removing a metric from the cache. | `"10m"` | no |
49+
| `start_time_metric_regex` | `string` | Regex for a metric name containing the start time. Only applies when strategy is `start_time_metric`. | `""` | no |
50+
51+
### Strategies
52+
53+
The `strategy` argument determines how the processor handles missing start times for cumulative metrics. Valid values are:
54+
55+
#### `true_reset_point` (default)
56+
57+
Produces a stream of points that starts with a [True Reset Point][true-reset-point].
58+
The true reset point has its start time set to its end timestamp, indicating the absolute value of the cumulative point when the collector first observed it.
59+
Subsequent points reuse the start timestamp of the initial true reset point.
60+
61+
[true-reset-point]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md#cumulative-streams-inserting-true-reset-points
62+
63+
**Pros:**
64+
* The absolute value of the cumulative metric is preserved.
65+
* It is possible to calculate the correct rate between any two points since the timestamps and values are not modified.
66+
67+
**Cons:**
68+
* This strategy is **stateful** because the initial True Reset point is necessary to properly calculate rates on subsequent points.
69+
* The True Reset point doesn't make sense semantically. It has a zero duration, but non-zero values.
70+
* Many backends reject points with equal start and end timestamps.
71+
* If the True Reset point is rejected, the next point will appear to have a very large rate.
72+
73+
**Example transformation:**
74+
75+
| Input Timestamp | Input Value | Output Start Time | Output End Time | Output Value |
76+
|-----------------|-------------|-------------------|-----------------|--------------|
77+
| T1 | 10 | T1 | T1 | 10 |
78+
| T2 | 15 | T1 | T2 | 15 |
79+
| T3 | 25 | T1 | T3 | 25 |
80+
81+
#### `subtract_initial_point`
82+
83+
Drops the first point in a cumulative series, subtracting that point's value from subsequent points and using the initial point's timestamp as the start timestamp for subsequent points.
84+
85+
**Pros:**
86+
* Cumulative semantics are preserved. This means that for a point with a given `[start, end]` interval, the cumulative value occurred in that interval.
87+
* Rates over resulting timeseries are correct, even if points are lost. This strategy is not stateful.
88+
89+
**Cons:**
90+
* The absolute value of counters is modified. This is generally not an issue, since counters are usually used to compute rates.
91+
* The initial point is dropped, which loses information.
92+
93+
**Example transformation:**
94+
95+
| Input Timestamp | Input Value | Output Start Time | Output End Time | Output Value |
96+
|-----------------|-------------|-------------------|-----------------|--------------|
97+
| T1 | 10 | _(dropped)_ | _(dropped)_ | _(dropped)_ |
98+
| T2 | 15 | T1 | T2 | 5 |
99+
| T3 | 25 | T1 | T3 | 15 |
100+
101+
#### `start_time_metric`
102+
103+
Looks for the `process_start_time` metric (or a metric matching `start_time_metric_regex`) and uses its value as the start time for all other cumulative points in the batch of metrics.
104+
If the start time metric is not found, it falls back to the time at which the collector started.
105+
106+
This strategy should only be used in limited circumstances:
107+
* When your application has a metric with the start time in Unix seconds, such as `process_start_time_seconds`.
108+
* The processor is used _before_ any batching, so that the batch of metrics all originate from a single application.
109+
* This strategy can be used when the collector is run as a sidecar to the application, where the collector's start time is a good approximation of the application's start time.
110+
111+
**Cons:**
112+
* If the collector's start time is used as a fallback and the collector restarts, it can produce rates that are incorrect and higher than expected.
113+
* The process' start time isn't the time at which individual instruments or timeseries are initialized. It may result in lower rates if the first observation is significantly later than the process' start time.
114+
115+
**Example transformation:**
116+
117+
Given a `process_start_time_seconds` metric with value `T0`:
118+
119+
| Input Timestamp | Input Value | Output Start Time | Output End Time | Output Value |
120+
|-----------------|-------------|-------------------|-----------------|--------------|
121+
| T1 | 10 | T0 | T1 | 10 |
122+
| T2 | 15 | T0 | T2 | 15 |
123+
| T3 | 25 | T0 | T3 | 25 |
124+
125+
### Garbage collection
126+
127+
The `gc_interval` argument defines how often to check if any resources have not emitted data since the last check.
128+
If a resource hasn't emitted any data, it's removed from the cache to free up memory.
129+
Any additional data from resources removed from the cache will be given a new start time.
130+
131+
## Blocks
132+
133+
You can use the following blocks with `otelcol.processor.metric_start_time`:
134+
135+
| Block | Description | Required |
136+
|----------------------------------|----------------------------------------------------------------------------|----------|
137+
| [`output`][output] | Configures where to send received telemetry data. | yes |
138+
| [`debug_metrics`][debug_metrics] | Configures the metrics that this component generates to monitor its state. | no |
139+
140+
[output]: #output
141+
[debug_metrics]: #debug_metrics
142+
143+
### `output`
144+
145+
{{< badge text="Required" >}}
146+
147+
{{< docs/shared lookup="reference/components/output-block.md" source="alloy" version="<ALLOY_VERSION>" >}}
148+
149+
### `debug_metrics`
150+
151+
{{< docs/shared lookup="reference/components/otelcol-debug-metrics-block.md" source="alloy" version="<ALLOY_VERSION>" >}}
152+
153+
## Exported fields
154+
155+
The following fields are exported and can be referenced by other components:
156+
157+
| Name | Type | Description |
158+
|---------|--------------------|------------------------------------------------------------------|
159+
| `input` | `otelcol.Consumer` | A value that other components can use to send telemetry data to. |
160+
161+
`input` accepts `otelcol.Consumer` data for metrics.
162+
163+
## Component health
164+
165+
`otelcol.processor.metric_start_time` is only reported as unhealthy if given an invalid configuration.
166+
167+
## Debug information
168+
169+
`otelcol.processor.metric_start_time` doesn't expose any component-specific debug information.
170+
171+
## Examples
172+
173+
### Basic usage with default strategy
174+
175+
This example uses the default `true_reset_point` strategy to set start times for Prometheus metrics:
176+
177+
```alloy
178+
otelcol.receiver.prometheus "default" {
179+
output {
180+
metrics = [otelcol.processor.metric_start_time.default.input]
181+
}
182+
}
183+
184+
otelcol.processor.metric_start_time "default" {
185+
output {
186+
metrics = [otelcol.exporter.otlp.production.input]
187+
}
188+
}
189+
190+
otelcol.exporter.otlp "production" {
191+
client {
192+
endpoint = sys.env("OTLP_SERVER_ENDPOINT")
193+
}
194+
}
195+
```
196+
197+
### Using subtract_initial_point strategy
198+
199+
This example uses the `subtract_initial_point` strategy, which preserves cumulative semantics and produces correct rates:
200+
201+
```alloy
202+
otelcol.receiver.prometheus "default" {
203+
output {
204+
metrics = [otelcol.processor.metric_start_time.default.input]
205+
}
206+
}
207+
208+
otelcol.processor.metric_start_time "default" {
209+
strategy = "subtract_initial_point"
210+
211+
output {
212+
metrics = [otelcol.exporter.otlp.production.input]
213+
}
214+
}
215+
216+
otelcol.exporter.otlp "production" {
217+
client {
218+
endpoint = sys.env("OTLP_SERVER_ENDPOINT")
219+
}
220+
}
221+
```
222+
223+
### Using start_time_metric strategy with custom regex
224+
225+
This example uses the `start_time_metric` strategy with a custom regex to find the start time metric:
226+
227+
```alloy
228+
otelcol.receiver.prometheus "default" {
229+
output {
230+
metrics = [otelcol.processor.metric_start_time.default.input]
231+
}
232+
}
233+
234+
otelcol.processor.metric_start_time "default" {
235+
strategy = "start_time_metric"
236+
gc_interval = "1h"
237+
start_time_metric_regex = "^.+_start_time$"
238+
239+
output {
240+
metrics = [otelcol.exporter.otlp.production.input]
241+
}
242+
}
243+
244+
otelcol.exporter.otlp "production" {
245+
client {
246+
endpoint = sys.env("OTLP_SERVER_ENDPOINT")
247+
}
248+
}
249+
```
250+
251+
<!-- START GENERATED COMPATIBLE COMPONENTS -->
252+
253+
## Compatible components
254+
255+
`otelcol.processor.metric_start_time` can accept arguments from the following components:
256+
257+
- Components that export [OpenTelemetry `otelcol.Consumer`](../../../compatibility/#opentelemetry-otelcolconsumer-exporters)
258+
259+
`otelcol.processor.metric_start_time` has exports that can be consumed by the following components:
260+
261+
- Components that consume [OpenTelemetry `otelcol.Consumer`](../../../compatibility/#opentelemetry-otelcolconsumer-consumers)
262+
263+
{{< admonition type="note" >}}
264+
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
265+
Refer to the linked documentation for more details.
266+
{{< /admonition >}}
267+
268+
<!-- END GENERATED COMPATIBLE COMPONENTS -->

extension/alloyengine/go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -645,6 +645,7 @@ require (
645645
github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor v0.142.0 // indirect
646646
github.com/open-telemetry/opentelemetry-collector-contrib/processor/intervalprocessor v0.142.0 // indirect
647647
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.142.0 // indirect
648+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstarttimeprocessor v0.142.0 // indirect
648649
github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.142.0 // indirect
649650
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.142.0 // indirect
650651
github.com/open-telemetry/opentelemetry-collector-contrib/processor/spanprocessor v0.142.0 // indirect

0 commit comments

Comments
 (0)