[otel-engine] Documentation

[jharvey10]

Description

Tracking issue for the doc requirements for the OTel engine.

Related PRs

• feat: Expose otel subcommand and add user-facing documentation #5244
• docs: Update experimental text wording for otel cli topic #5265

Tip

_{React with 👍 if this issue is important to you.}

[blewis12]

Clayton and I had a chat offline about this, I'll summarize the points covered here just for visibility

I'll separate these out for now by what I think we should definitely include, vs what i'm more hesitant about and we can discuss here:

What to include

• We should add some descriptive text outside of the CLI topic (ie explaining the idea of two engines, what an otel distribution is, impact, why one might want to use it etc)
• A list of components, currently we link to the OCB manifest - i'm not sure if there's a way to generate this, but can have a look to see if there are any tools we can use
• Aligning the alloy engine readme - currently we link to that in the documentation. We can instead include the important user facing information in the CLI docs, whilst the README contains more in-detail developer-facing docs
• Include information on feature flags, OTel feature flags are supported - but Alloy feature flags (i'm thinking in particular stability level) are not, since these are attached to the run command and not otel
• Information about what is exposed on which HTTP endpoint (by default), as well as storage paths
• Include a point that remote management is not yet supported for the OTel Engine
• Mention that logs for both the OTel Engine and Default Engine will be sent to stdout - if the user enables OTel Engine they will see these logs in the output, this is the same as the upstream experience
• Mention the upstream validate feature for OTel, if YAML is invalid the user will see an error in the logs and the collector will not start
• Mention that the debug UI and support bundle functionality are not available with the otel engine yet
• Mention that /-/reload is not exposed for the OTel Engine - the OTel Engine does not support hot reload. Instead, the user must restart the collector, which is the same experience as upstream
• Mention that there are no pre-requisites before invoking otel, aside from having a valid YAML file

Unsure about including

1. Migration strategies - we do support converting OTel YAML to Alloy Config, but not vice versa. We don't have a set migration story there, so i'm not sure what to include
2. Clayton you mentioned a point about version compatibility for the OTel Collector - could you clarify what you meant there?
3. Mentioning what flags can be used in the otel command - these are the exact same as upstream, so i'd rather just link there than duplicate them. I feel like our documentation should avoid duplicating upstream docs as much as possible
4. Clayton you mentioned that "accepts standard OpenTelemetry Collector YAML" could be unclear, shall we link to upstream documentation here?

@clayton-cornell did I summarize this all well? Let me know if I missed anything - and any feedback on the above points

Today I will work on a PR including some of those points from the first section

[clayton-cornell]

In the list of unsure items:

1. Migration strategy is just something we might be asked for. It's noted. I'd say leave it for now.
2. For OTel version compatibility, just a question if there's any version limits on the OTel side... latest... latest-n etc.
3. Linking to upstream is 100% what we should do.
4. Yes, linking to upstream there as well.

I think that's everything. It's a really comprehensive starting point.

[blewis12]

Marking this as done - the only thing to do next would be to expose these documents in the next Alloy version