diff --git a/docs/configure-plugins/observability/openinference.mdx b/docs/configure-plugins/observability/openinference.mdx index 2aeff3b3d..e4108396b 100644 --- a/docs/configure-plugins/observability/openinference.mdx +++ b/docs/configure-plugins/observability/openinference.mdx @@ -31,7 +31,7 @@ kind = "observability" enabled = true [components.config] -version = 1 +version = 2 [components.config.openinference] enabled = true @@ -63,6 +63,7 @@ OpenInference uses the same OTLP section shape as | `enabled` | `false` | Must be `true` to construct and register the subscriber. | | `mark_projection` | `inherit` | `inherit` uses exporter-native handling; `event` forces span events; `tool` emits zero-duration OpenInference `TOOL` spans, parented as children when context is available. | | `mark_exclude_names` | `["llm.chunk"]` | Mark names excluded from `tool` projection; excluded marks retain exporter-native handling. Metadata `hook_event_name` aliases are also matched. | +| `attribute_mappings` | `[]` | Copies each fully qualified projected `key` to its `alias` without changing the OTLP type. Set both values to nonblank strings, and use each alias only once. | | `transport` | `http_binary` | `http_binary` or `grpc`. | | `endpoint` | Exporter default | OTLP endpoint. | | `headers` | `{}` | String-to-string exporter headers. | @@ -122,6 +123,38 @@ Cost appears as USD-denominated `llm.cost.total`. Refer to [Token and Cost Field Semantics](/integrate-into-frameworks/provider-response-codecs#token-and-cost-field-semantics) for the full mapping. +NeMo Relay projects top-level lifecycle payload fields to typed OTLP attributes +with dotted names. Non-LLM start metadata and all end metadata use the +`openinference.metadata` prefix, so `metadata = { tenant = "acme" }` becomes +`openinference.metadata.tenant = "acme"`. Non-LLM start events use +`nemo_relay.start.data`, `nemo_relay.start.input`, and +`nemo_relay.handle_attributes`. End events use `nemo_relay.end.data` and +`nemo_relay.end.output`. LLM start events use OpenInference semantic input +attributes instead of those generic start projections, and their final metadata +comes from the end event. Mark data, metadata, attributes, and category-profile +fields use the corresponding `nemo_relay.mark.*` prefixes. + +Scalar strings, booleans, and numbers that fit an OTLP numeric type keep their +types. NeMo Relay emits larger unsigned integers as strings. When a top-level +field contains an object or array, NeMo Relay emits its value as a JSON string +at that field's dotted name. Nested `null` values remain in that string, but a +top-level field whose value is `null` is omitted. NeMo Relay no longer emits the +old aggregate `*_json` payload attributes. The exporter still keeps the +OpenInference `metadata` JSON-string attribute for backend compatibility. + +Configure an alias when a backend expects a different attribute name: + +```toml +[components.config.openinference] +attribute_mappings = [ + { key = "openinference.metadata.tenant", alias = "tenant.id" }, +] +``` + +The source attribute remains in the span. If the span already contains +`tenant.id`, NeMo Relay keeps the existing value instead of replacing it with +the alias. + Redact sensitive event payloads with sanitize guardrails before production export. @@ -191,7 +224,7 @@ void (async () => { version: 1, components: [ observability.ComponentSpec({ - version: 1, + version: 2, openinference: observability.otlpConfig({ enabled: true, transport: "http_binary", diff --git a/docs/configure-plugins/observability/opentelemetry.mdx b/docs/configure-plugins/observability/opentelemetry.mdx index bc31c68c2..6c434c9cd 100644 --- a/docs/configure-plugins/observability/opentelemetry.mdx +++ b/docs/configure-plugins/observability/opentelemetry.mdx @@ -26,7 +26,7 @@ kind = "observability" enabled = true [components.config] -version = 1 +version = 2 [components.config.opentelemetry] enabled = true @@ -57,6 +57,7 @@ The following table describes OpenTelemetry exporter settings: | `enabled` | `false` | Must be `true` to construct and register the subscriber. | | `mark_projection` | `inherit` | `inherit` uses exporter-native handling; `event` forces span events; `tool` emits zero-duration spans, parented as children when context is available, for trace-tree visibility. | | `mark_exclude_names` | `["llm.chunk"]` | Mark names excluded from `tool` projection; excluded marks retain exporter-native handling. Metadata `hook_event_name` aliases are also matched. | +| `attribute_mappings` | `[]` | Copies each fully qualified projected `key` to its `alias` without changing the OTLP type. Set both values to nonblank strings, and use each alias only once. | | `transport` | `http_binary` | `http_binary` or `grpc`. | | `endpoint` | Exporter default | OTLP endpoint. | | `headers` | `{}` | String-to-string exporter headers. | @@ -94,6 +95,38 @@ attributes. These values match ATIF `step.extra.ancestry.function_id` and the trajectory-root span's `nemo_relay.uuid` also matches the ATIF `session_id`. Backend-native `trace_id` and `span_id` values are not written into ATIF. +NeMo Relay projects top-level lifecycle payload fields to typed OTLP attributes +with dotted names. For example, `metadata = { tenant = "acme" }` becomes +`nemo_relay.start.metadata.tenant = "acme"`. +- Start events use the +`nemo_relay.start.data`, `nemo_relay.start.metadata`, and +`nemo_relay.start.input` prefixes. +- End events use `nemo_relay.end.data`, +`nemo_relay.end.metadata`, and `nemo_relay.end.output`. +- Handle attributes use +`nemo_relay.handle_attributes`. Mark data, metadata, attributes, and +category-profile fields use the corresponding `nemo_relay.mark.*` prefixes. + +Scalar strings, booleans, and numbers that fit an OTLP numeric type keep their +types. NeMo Relay emits larger unsigned integers as strings. When a top-level +field contains an object or array, NeMo Relay emits its value as a JSON string +at that field's dotted name. Nested `null` values remain in that string, but a +top-level field whose value is `null` is omitted. NeMo Relay no longer emits the +old aggregate `*_json` payload attributes. + +Configure an alias when a backend expects a different attribute name: + +```toml +[components.config.opentelemetry] +attribute_mappings = [ + { key = "nemo_relay.start.metadata.tenant", alias = "tenant.id" }, +] +``` + +The source attribute remains in the span. If the span already contains +`tenant.id`, NeMo Relay keeps the existing value instead of replacing it with +the alias. + Coding-agent trace roots also carry these correlation attributes: | Attribute | Meaning | @@ -185,7 +218,7 @@ void (async () => { version: 1, components: [ observability.ComponentSpec({ - version: 1, + version: 2, opentelemetry: observability.otlpConfig({ enabled: true, transport: "http_binary", diff --git a/skills/nemo-relay-plugin-observability/references/openinference.md b/skills/nemo-relay-plugin-observability/references/openinference.md index 6acd97e66..55606a26e 100644 --- a/skills/nemo-relay-plugin-observability/references/openinference.md +++ b/skills/nemo-relay-plugin-observability/references/openinference.md @@ -20,14 +20,22 @@ for example Arize Phoenix or another OpenInference-aware OTLP backend. - OpenInference export is for OTLP backends that understand model-centric OpenInference semantic conventions. -- Configure the exporter with the following settings: - - `transport`, `endpoint`, `service_name`, optional namespace and version, - instrumentation scope, headers, resource attributes, and timeout. - - `http_binary` transport and an OTLP/HTTP traces endpoint. Use `grpc` only - when a Tokio runtime is active. -- Exported events have the following mappings: - - Scope, tool, and LLM start inputs become `input.value`. - - Scope, tool, and LLM end outputs become `output.value`. +- Set `transport`, `endpoint`, and `service_name`, then add a namespace, version, + instrumentation scope, headers, resource attributes, timeout, or + `attribute_mappings` when needed. +- NeMo Relay projects lifecycle payload fields to typed OTLP attributes with + dotted names. Non-LLM start metadata and all end metadata use + `openinference.metadata`, while mark metadata uses + `nemo_relay.mark.metadata`. +- NeMo Relay emits a top-level object or array field as a JSON string, omits a + top-level `null` field, and no longer emits the old aggregate `*_json` payload + attributes. +- Use `attribute_mappings` to copy a fully qualified projected attribute to a + backend-specific alias without changing its OTLP type. +- Start with `http_binary` transport and an OTLP/HTTP traces endpoint. Use + `grpc` only when a Tokio runtime is active. +- Scope, tool, and LLM start inputs become `input.value`. +- Scope, tool, and LLM end outputs become `output.value`. - LLM annotations follow the freshness rules: - Each owning agent scope starts fresh, and a `compaction` mark refreshes it. - The annotated input for the first subsequent LLM start retains complete diff --git a/skills/nemo-relay-plugin-observability/references/opentelemetry.md b/skills/nemo-relay-plugin-observability/references/opentelemetry.md index f5265d69d..dffc7cead 100644 --- a/skills/nemo-relay-plugin-observability/references/opentelemetry.md +++ b/skills/nemo-relay-plugin-observability/references/opentelemetry.md @@ -21,8 +21,17 @@ OpenTelemetry Collector, Jaeger, Tempo, or Honeycomb. - OpenTelemetry export maps NeMo Relay runtime events into OTLP traces for tracing backends and collectors. -- Configure `transport`, `endpoint`, `service_name`, optional namespace and - version, instrumentation scope, headers, resource attributes, and timeout. +- Set `transport`, `endpoint`, and `service_name`, then add a namespace, version, + instrumentation scope, headers, resource attributes, timeout, or + `attribute_mappings` when needed. +- NeMo Relay projects start, end, handle, and mark payload fields to typed OTLP + attributes with dotted names. Start and end metadata use + `nemo_relay.start.metadata` and `nemo_relay.end.metadata`. +- NeMo Relay emits a top-level object or array field as a JSON string, omits a + top-level `null` field, and no longer emits the old aggregate `*_json` payload + attributes. +- Use `attribute_mappings` to copy a fully qualified projected attribute to a + backend-specific alias without changing its OTLP type. - Start with `http_binary` transport and an OTLP traces endpoint such as a local collector on port `4318` unless deployment requirements differ. - `grpc` transport is available when a Tokio runtime is active.