Skip to content

Commit 8a81d02

Browse files
x1unixx1unix
committed
feat: add component docs
1 parent a297921 commit 8a81d02

File tree

1 file changed

+219
-0
lines changed

1 file changed

+219
-0
lines changed

‎docs/sources/reference/components/otelcol/otelcol.receiver.awss3.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
---
2+
canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.receiver.awss3/
3+
description: Learn about otelcol.receiver.awss3
4+
labels:
5+
stage: experimental
6+
products:
7+
- oss
8+
title: otelcol.receiver.awss3
9+
---
10+
11+
# `otelcol.receiver.awss3`
12+
13+
{{< docs/shared lookup="stability/experimental.md" source="alloy" version="<ALLOY_VERSION>" >}}
14+
15+
`otelcol.receiver.awss3` receives traces previously stored in S3 by the [AWS S3 Exporter](./otelcol.exporter.awss3.md).
16+
17+
{{< admonition type="warning" >}}
18+
`otelcol.receiver.awss3` is a wrapper over the upstream OpenTelemetry Collector [`awss3`][] receiver.
19+
Bug reports or feature requests will be redirected to the upstream repository, if necessary.
20+
21+
[`awss3`]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/{{< param "OTEL_VERSION" >}}/receiver/awss3receiver
22+
{{< /admonition >}}
23+
24+
The receiver has two modes of operation:
25+
26+
- **Time Range Mode** - Specify start and end to fetch data from a specific time range.
27+
- **SQS Message Mode** - Subscribe to SQS messages to process new objects as they arrive.
28+
29+
The receiver supports following encodings:
30+
31+
- otlp_json (OpenTelemetry Protocol format represented as json) with a suffix of `.json`
32+
- otlp_proto (OpenTelemetry Protocol format represented as Protocol Buffers) with a suffix of `.binpb`
33+
34+
{{< admonition type="note" >}}
35+
Currently, `otelcol.receiver.awss3` receiver doesn't support encoding extensions.
36+
{{< /admonition >}}
37+
38+
You can specify multiple `otelcol.receiver.awss3` components by giving them different labels.
39+
40+
## Usage
41+
42+
```alloy
43+
otelcol.receiver.awss3 "<LABEL>" {
44+
start_time = "..."
45+
end_time = "..."
46+
47+
s3downloader {
48+
s3_bucket = "..."
49+
s3_prefix = "..."
50+
}
51+
52+
output {
53+
logs = [...]
54+
metrics = [...]
55+
trace = [...]
56+
}
57+
}
58+
```
59+
60+
## Arguments
61+
62+
You can use the following arguments with `otelcol.receiver.awss3`:
63+
64+
| Name | Type | Description | Default | Required |
65+
| ------------ | -------- | ------------------------------------------- | ------- | ----------------------------- |
66+
| `start_time` | `string` | The time at which to start retrieving data. | | Required if fetching by time. |
67+
| `end_time` | `string` | The time at which to stop retrieving data. | | Required if fetching by time. |
68+
69+
The `start_time` and `end_time` fields time format is either RFC3339, `YYYY-MM-DD HH:MM` or simply `YYYY-MM-DD`, in which case the time is assumed to be `00:00`.
70+
71+
{{< admonition type="warning" >}}
72+
Time-based configuration (`start_time` and `end_time` arguments) cannot be combined together with [`sqs`][] block.
73+
74+
[`sqs`]: #sqs
75+
76+
{{< /admonition >}}
77+
78+
Refer to the upstream receiver [documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/{{< param "OTEL_VERSION" >}}/receiver/awss3receiver#configuration) for more details.
79+
80+
## Blocks
81+
82+
You can use the following blocks with `otelcol.receiver.cloudflare`:
83+
84+
| Block | Description | Required |
85+
| ------------------------------ | ---------------------------------------------------------------------------- | ----------------------------------------- |
86+
| [`s3downloader`][s3downloader] | Configures S3 downloader. | yes |
87+
| [`sqs`][output] | Configures SQS queue configuration for receiving object change notification. | Required if fetching by SQS notification. |
88+
| [`output`][output] | Configures where to send received telemetry data. | yes |
89+
90+
[s3downloader]: #s3downloader
91+
[sqs]: #sqs
92+
[output]: #output
93+
94+
### `s3downloader`
95+
96+
{{< badge text="Required" >}}
97+
98+
The `s3downloader` block contains AWS S3 downloader related config to control things like bucket, prefix, batching, connections, retries, etc.
99+
100+
The following arguments are supported:
101+
102+
| Name | Type | Description | Default | Required |
103+
| ----------------------- | -------- | ----------------------------------------------------------------------------------------------------- | ------------- | -------- |
104+
| `region` | `string` | AWS region. | `"us-east-1"` | no |
105+
| `s3_bucket` | `string` | S3 bucket. | | yes |
106+
| `s3_prefix` | `string` | Prefix for the S3 key (root directory inside bucket). | | yes |
107+
| `s3_partition` | `string` | Time granularity of S3 key: hour or minute. | `"minute"` | no |
108+
| `file_prefix` | `string` | File prefix defined by user. | | no |
109+
| `endpoint` | `string` | Overrides the endpoint used by the exporter instead of constructing it from `region` and `s3_bucket`. | | no |
110+
| `endpoint_partition_id` | `string` | Partition id to use if `endpoint` is specified. | `"aws"` | no |
111+
| `s3_force_path_style` | `bool` | Wnen enabled, forces the request to use [path-style addressing][s3-force-path-style-ref]. | `false` | no |
112+
113+
[s3-force-path-style-ref]: http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html
114+
115+
### `sqs`
116+
117+
The `sqs` block holds SQS queue configuration for receiving object change notifications.
118+
119+
The following arguments are supported:
120+
121+
| Name | Type | Description | Default | Required |
122+
| ------------------------ | -------- | --------------------------------------------------------------- | ------- | -------- |
123+
| `queue_url` | `string` | The URL of the SQS queue that receives S3 bucket notifications. | | yes |
124+
| `region` | `string` | AWS region of the SQS queue. | | yes |
125+
| `endpoint` | `string` | Custom endpoint for the SQS service. | | no |
126+
| `max_number_of_messages` | `int` | Maximum number of messages to retrieve in a single SQS request. | `10` | no |
127+
| `wait_time_seconds` | `int` | Wait time in seconds for long polling SQS requests. | `20` | no |
128+
129+
{{< admonition type="note" >}}
130+
You must configure your S3 bucket to send event notifications to the SQS queue. Time-based configuration (`start_time`/`end_time`) and SQS configuration cannot be used together.
131+
{{< /admonition >}}
132+
133+
### `output`
134+
135+
{{< badge text="Required" >}}
136+
137+
{{< docs/shared lookup="reference/components/output-block.md" source="alloy" version="<ALLOY_VERSION>" >}}
138+
139+
## Exported fields
140+
141+
`otelcol.receiver.awss3` doesn't export any fields.
142+
143+
## Component health
144+
145+
`otelcol.receiver.awss3` is only reported as unhealthy if given an invalid configuration.
146+
147+
## Debug information
148+
149+
`otelcol.receiver.awss3` doesn't expose any component-specific debug information.
150+
151+
## Example
152+
153+
This example forwards received traces through a batch processor before finally sending it to an OTLP-capable endpoint:
154+
155+
```alloy
156+
// Time range mode:
157+
otelcol.receiver.awss3 "default" {
158+
start_time = "2024-01-01 01:00"
159+
end_time = "2024-01-02"
160+
161+
s3downloader {
162+
region = "us-west-1"
163+
s3_bucket = "mybucket"
164+
s3_prefix = "trace"
165+
s3_partition = "minute"
166+
}
167+
168+
output {
169+
traces = [otelcol.processor.batch.default.input]
170+
}
171+
}
172+
173+
// SQS message mode:
174+
otelcol.receiver.awss3 "sqs_traces" {
175+
s3downloader {
176+
region = "us-east-1"
177+
s3_bucket = "mybucket"
178+
s3_prefix = "mytrace"
179+
}
180+
181+
sqs {
182+
queue_url = "https://sqs.us-east-1.amazonaws.com/123456789012/test-queue"
183+
region = "us-east-1"
184+
}
185+
186+
output {
187+
traces = [otelcol.processor.batch.default.input]
188+
}
189+
}
190+
191+
otelcol.processor.batch "default" {
192+
output {
193+
metrics = [otelcol.exporter.otlp.default.input]
194+
logs = [otelcol.exporter.otlp.default.input]
195+
traces = [otelcol.exporter.otlp.default.input]
196+
}
197+
}
198+
199+
otelcol.exporter.otlp "default" {
200+
client {
201+
endpoint = sys.env("<OTLP_ENDPOINT>")
202+
}
203+
}
204+
```
205+
206+
<!-- START GENERATED COMPATIBLE COMPONENTS -->
207+
208+
## Compatible components
209+
210+
`otelcol.receiver.awss3` can accept arguments from the following components:
211+
212+
- Components that export [OpenTelemetry `otelcol.Consumer`](../../../compatibility/#opentelemetry-otelcolconsumer-exporters)
213+
214+
{{< admonition type="note" >}}
215+
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
216+
Refer to the linked documentation for more details.
217+
{{< /admonition >}}
218+
219+
<!-- END GENERATED COMPATIBLE COMPONENTS -->

0 commit comments

Comments
 (0)