From 97d8c7b93f44745a171daad88034542ba738a76a Mon Sep 17 00:00:00 2001 From: Will Killian Date: Tue, 7 Jul 2026 11:01:22 -0400 Subject: [PATCH 1/9] docs: document mark and scope event sanitizers Signed-off-by: Will Killian --- .agents/skills/add-middleware/SKILL.md | 8 +- crates/node/README.md | 4 +- docs/about-nemo-relay/concepts/events.mdx | 6 + docs/about-nemo-relay/concepts/middleware.mdx | 22 ++- docs/build-plugins/language-binding/about.mdx | 2 + .../advanced-configuration.mdx | 2 + .../language-binding/register-behavior.mdx | 8 + .../configure-plugins/pii-redaction/about.mdx | 15 +- .../pii-redaction/configuration.mdx | 37 +++- .../adding-scopes-and-marks.mdx | 4 +- docs/reference/event-sanitizers.mdx | 181 ++++++++++++++++++ .../tool-execution-intercept-outcomes.mdx | 13 +- go/nemo_relay/README.md | 3 +- python/nemo_relay/README.md | 3 +- .../SKILL.md | 4 +- 15 files changed, 288 insertions(+), 24 deletions(-) create mode 100644 docs/reference/event-sanitizers.mdx diff --git a/.agents/skills/add-middleware/SKILL.md b/.agents/skills/add-middleware/SKILL.md index c4bd69f0a..13dd674b6 100644 --- a/.agents/skills/add-middleware/SKILL.md +++ b/.agents/skills/add-middleware/SKILL.md @@ -24,7 +24,7 @@ middleware behavior to a new pipeline stage. Decide these before editing code: -- Is this for tools, LLMs, or both? +- Is this for tools, LLMs, marks, scope events, or a combination? - Is it a conditional guardrail, sanitize guardrail, request intercept, or execution intercept? - Does it run on request input, inner callable execution, stream chunks, or @@ -33,6 +33,8 @@ Decide these before editing code: - Does it need both global and scope-local registration? - What should subscribers observe in `event.input` and `event.output` after this middleware runs? +- If this is an event sanitizer, which of `data`, `category_profile`, and + `metadata` may change, and is the event used only as immutable context? ## Pipeline Order @@ -44,6 +46,9 @@ See `docs/about/concepts/middleware.md` for the full diagrams. - **LLM execute**: conditional guardrails -> request intercepts -> sanitize request (for events) | execution intercept chain(callable) -> sanitize response +- **Mark and scope events**: + specialized tool/LLM sanitizer (when applicable) -> mark or scope event + sanitizer -> subscriber dispatch ## Core Steps @@ -88,6 +93,7 @@ Follow the `add-binding-feature` skill for the cross-binding implementation chec - [ ] Callback error propagation - [ ] Scope-local registration, inheritance, and cleanup on pop - [ ] Event input/output semantics after middleware mutation +- [ ] Mark and scope event field semantics, including immutable identity fields - [ ] Parity coverage in every affected binding ## Key References diff --git a/crates/node/README.md b/crates/node/README.md index 8a131f83f..3ff1cca3d 100644 --- a/crates/node/README.md +++ b/crates/node/README.md @@ -41,7 +41,9 @@ should install it from npm rather than depend on the Rust crate directly. napi-rs native extension. - **Managed tool and LLM execution**: Helpers that emit lifecycle events and run middleware in a consistent order. -- **Middleware APIs**: Guardrails and intercepts for tool and LLM boundaries. +- **Middleware APIs**: Guardrails and intercepts for tool and LLM boundaries, + plus mark and scope event sanitizers for `data`, `categoryProfile`, and + `metadata`. - **Observability exporters**: Subscriber and exporter support for common runtime telemetry flows. - **Additional entry points**: `nemo-relay-node/typed`, diff --git a/docs/about-nemo-relay/concepts/events.mdx b/docs/about-nemo-relay/concepts/events.mdx index dddf27cd0..15d9706ac 100644 --- a/docs/about-nemo-relay/concepts/events.mdx +++ b/docs/about-nemo-relay/concepts/events.mdx @@ -103,6 +103,12 @@ exporters. Consumers should prefer typed event fields, binding helpers, and annotated request or response data before falling back to raw payload re-reading. +Before that handoff, event sanitizers can replace `data`, `category_profile`, +and `metadata`. The runtime keeps identity, lifecycle, category, timestamp, +and parent fields immutable. Refer to +[Event Sanitizers](/reference/event-sanitizers) for registration levels and +ordering. + Events preserve runtime facts. They do not decide which scope owns a call, how replay or cost policy is applied, how redaction policy is configured, how streams are blocked, or how exporter-specific semantic projection works. Those diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index 427782afe..08cc96d38 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -8,7 +8,8 @@ import { MermaidStyles } from "@/components/MermaidStyles"; {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} -This page explains the runtime behavior that runs around managed tool and LLM calls. +This page explains the runtime behavior that runs around managed tool and LLM +calls and sanitizes emitted mark and scope events. ## What Middleware Is @@ -106,6 +107,22 @@ Sanitize-response guardrails rewrite the payload recorded on emitted end events. Use them when the event stream should hide or reduce sensitive response data. +### Sanitize Mark and Scope Events + +Event sanitizers cover observability fields that are outside the specialized +tool and LLM payload APIs. Separate registries apply to marks, scope starts, +and scope ends. They can rewrite `data`, `category_profile`, and `metadata` +while receiving the complete event as immutable context. + +Scope event sanitizers run for every category. On tool and LLM scope events, +they run after the specialized request or response sanitizer. Mark sanitizers +cover explicit marks and marks materialized by middleware, plugins, and +streaming lifecycle helpers. + +Register event sanitizers globally, on an owning scope, or through a plugin +context. For the callback contract and binding APIs, refer to +[Event Sanitizers](/reference/event-sanitizers). + Sanitize guardrails are observability-oriented. They do not rewrite the real arguments passed to the callback or the real value returned to the caller. @@ -261,6 +278,9 @@ Use these comparisons to pick the middleware surface that matches the behavior y boundary. - Use a **sanitize guardrail** when only subscribers and exporters should see rewritten data. +- Use a **mark or scope event sanitizer** when the sensitive fields are in + `data`, `category_profile`, or `metadata` rather than the managed tool or LLM + request/response payload. - Use a **stream execution intercept** when you need streaming-specific behavior applied across the lifecycle of a long-lived or chunked response, such as per-chunk transformation, incremental authorization, logging or diff --git a/docs/build-plugins/language-binding/about.mdx b/docs/build-plugins/language-binding/about.mdx index 6e8e1a306..73359fd44 100644 --- a/docs/build-plugins/language-binding/about.mdx +++ b/docs/build-plugins/language-binding/about.mdx @@ -64,6 +64,8 @@ references during registration. A plugin can install one or more of these runtime surfaces: - **Subscribers:** Event subscribers. +- **Event sanitizers:** Mark, scope-start, and scope-end observability-field + sanitizers. - **Tool middleware:** Sanitize-request and sanitize-response guardrails, conditional-execution guardrails, request intercepts, and execution intercepts. diff --git a/docs/build-plugins/language-binding/advanced-configuration.mdx b/docs/build-plugins/language-binding/advanced-configuration.mdx index a1b75d68d..a9bec92cb 100644 --- a/docs/build-plugins/language-binding/advanced-configuration.mdx +++ b/docs/build-plugins/language-binding/advanced-configuration.mdx @@ -158,6 +158,8 @@ Prefer observe-only defaults for new policies and execution-affecting intercepts Use `PluginContext` to register: - Subscribers +- Mark event sanitizers +- Scope-start and scope-end event sanitizers - Tool guardrails - Tool request and execution intercepts - LLM guardrails diff --git a/docs/build-plugins/language-binding/register-behavior.mdx b/docs/build-plugins/language-binding/register-behavior.mdx index 1bf911723..f84967af8 100644 --- a/docs/build-plugins/language-binding/register-behavior.mdx +++ b/docs/build-plugins/language-binding/register-behavior.mdx @@ -34,6 +34,14 @@ Use the context only after validation succeeds. Keep validation deterministic an side-effect free. Inspect configuration and return diagnostics. Create runtime objects and attach them to the context during registration. +The context includes mark, scope-start, and scope-end event sanitizer +registrations in addition to the tool and LLM middleware surfaces. Event +sanitizer callbacks receive the immutable event plus `data`, +`category_profile`, and `metadata`, and return only those observability fields. +Use the context methods so component name qualification and rollback also apply +to these registries. Refer to [Event Sanitizers](/reference/event-sanitizers) +for the binding-specific method names. + ## Header Plugin Example The same model applies in every binding: validate component-local config, then install middleware through the component-scoped registration context. diff --git a/docs/configure-plugins/pii-redaction/about.mdx b/docs/configure-plugins/pii-redaction/about.mdx index 73711e4e8..6c06ceccc 100644 --- a/docs/configure-plugins/pii-redaction/about.mdx +++ b/docs/configure-plugins/pii-redaction/about.mdx @@ -8,8 +8,8 @@ position: 1 SPDX-License-Identifier: Apache-2.0 */} Use the PII redaction plugin when you want first-party privacy controls for -managed NeMo Relay LLM and tool observability surfaces through the shared -plugin system. +managed NeMo Relay LLM, tool, mark, and generic scope observability surfaces +through the shared plugin system. The built-in plugin component uses `kind = "pii_redaction"` and is available as a first-party NeMo Relay plugin. @@ -30,7 +30,8 @@ The plugin supports these backend modes: Start here when you need to: -- Remove sensitive fields from emitted tool or LLM payloads. +- Remove sensitive fields from emitted tool, LLM, mark, or generic scope + payloads. - Replace sensitive text with a deterministic marker such as `[REDACTED]`. - Hash matching values before observability exporters or subscribers receive them. @@ -66,10 +67,11 @@ The difference between middleware and `pii_redaction` is that middleware is the ## Current Scope -The built-in plugin currently exposes four managed sanitize surfaces: +The built-in plugin exposes five managed sanitize surfaces: - `input` - `output` +- `mark` - `tool_input` - `tool_output` @@ -88,6 +90,11 @@ The current backend boundary is intentional: - Managed LLM surfaces use the selected built-in codec so redaction can target normalized Relay request and response shapes such as `/messages/0/content` and `/message`. +- `mark` sanitizes `data`, `category_profile`, and `metadata` independently on + every mark event. It defaults to `true`; set `mark = false` to opt out. +- `input` and `output` also sanitize those three fields on non-tool, non-LLM + scope starts and ends. Tool and LLM scope envelopes remain on their + specialized sanitizer paths to avoid applying the same mask or hash twice. ## Observability Boundary diff --git a/docs/configure-plugins/pii-redaction/configuration.mdx b/docs/configure-plugins/pii-redaction/configuration.mdx index 1d2c16af6..fca02822f 100644 --- a/docs/configure-plugins/pii-redaction/configuration.mdx +++ b/docs/configure-plugins/pii-redaction/configuration.mdx @@ -66,8 +66,9 @@ The top-level PII redaction object contains: |---|---| | `version` | PII redaction config schema version. Defaults to `1`. | | `mode` | Backend mode. Current values are `builtin` and `local_model`. | -| `input` | Enables managed LLM request sanitization. | -| `output` | Enables managed LLM response sanitization. | +| `input` | Enables managed LLM request sanitization and non-tool/LLM scope-start field sanitization. | +| `output` | Enables managed LLM response sanitization and non-tool/LLM scope-end field sanitization. | +| `mark` | Enables mark `data`, `category_profile`, and `metadata` sanitization. Defaults to `true`. | | `tool_input` | Enables sanitization of emitted tool-request observability payloads. | | `tool_output` | Enables sanitization of emitted tool-response observability payloads. | | `priority` | Guardrail priority. Lower values run earlier. | @@ -87,6 +88,7 @@ The following table compares the available PII redaction backends: | Built-in component kind and config validation | Supported | Supported | | Managed LLM `input` | Supported | Not implemented | | Managed LLM `output` | Supported | Not implemented | +| Mark and generic scope event fields | Supported | Not implemented | | Managed `tool_input` | Supported | Not implemented | | Managed `tool_output` | Supported | Not implemented | | Built-in actions | `remove`, `redact`, `regex_replace`, `hash`, `mask` | N/A | @@ -131,6 +133,7 @@ mode = "builtin" codec = "openai_chat" input = true output = true +mark = true tool_input = true tool_output = true @@ -158,6 +161,12 @@ This example configures the built-in backend for: - LLM response redaction from the normalized response path `/message` - tool argument redaction at `/api_key` - tool result redaction at `/result/secret` +- mark and non-tool/LLM scope event redaction across `data`, + `category_profile`, and `metadata` + +`mark` is enabled when omitted. Set `mark = false` only when marks are known to +contain no sensitive observability fields or another sanitizer owns that +boundary. ### CLI Editor Support @@ -168,6 +177,7 @@ Use the editor when you want to: - Toggle the component on or off - Choose `builtin` or `local_model` +- Enable or opt out of mark sanitization - Set the LLM `codec` - Edit `builtin` action settings such as `action`, `target_paths`, `pattern`, `detector`, `replacement`, and masking fields @@ -277,6 +287,10 @@ The plugin uses different payload boundaries for tools and LLMs: - LLMs use the selected built-in codec. Prefer normalized Relay paths such as: - `/messages/0/content` for request message content - `/message` for the normalized assistant response text +- Marks and non-tool/LLM scopes sanitize `data`, `category_profile`, and + `metadata` as separate JSON values. A target path is evaluated independently + inside each value; it does not start with `/data`, `/category_profile`, or + `/metadata`. The current implementation also preserves provider-shaped response-path compatibility for the supported codecs, but normalized LLM paths are the @@ -327,14 +341,23 @@ string leaf in the selected payload boundary. ## Observability Semantics -The built-in plugin uses sanitize guardrails. +The built-in plugin uses sanitize guardrails and event sanitizers. That means: -- The real provider response value remains unchanged. -- The plugin sanitizes emitted NeMo Relay start and end event payloads. -- The plugin populates `annotated_response` from the sanitized end-event payload - when a response codec is available. +- the real provider response value is unchanged +- the emitted NeMo Relay start, end, or mark event observability fields are + sanitized +- `annotated_response` is populated from the sanitized end-event payload when a + response codec is provided +- subscribers, ATOF, ATIF, OpenTelemetry, and OpenInference receive the + sanitized event; raw values removed by the plugin do not bypass through an + exporter + +For tool and LLM scope events, the generic scope sanitizer leaves the shared +fields unchanged because the specialized tool or LLM sanitizer already owns +that lifecycle boundary. This prevents repeated hashing or masking. Generic +scope categories use `input` for starts and `output` for ends. ## Local Model Mode diff --git a/docs/instrument-applications/adding-scopes-and-marks.mdx b/docs/instrument-applications/adding-scopes-and-marks.mdx index ce7ede293..d40270f0f 100644 --- a/docs/instrument-applications/adding-scopes-and-marks.mdx +++ b/docs/instrument-applications/adding-scopes-and-marks.mdx @@ -200,7 +200,9 @@ Use this checklist before running the pattern in production traffic. - Keep scope names stable enough for filtering and dashboards. - Use marks for business checkpoints, not verbose debug logging. - Include only JSON-compatible `data`, `metadata`, `input`, and `output` payloads. -- Do not attach sensitive payloads unless sanitize guardrails or exporter filters will remove them. +- Do not attach sensitive payloads unless a mark or scope + [event sanitizer](/reference/event-sanitizers) removes them before subscriber + and exporter delivery. - Propagate the active scope stack when work crosses thread or worker boundaries. ## Next Steps diff --git a/docs/reference/event-sanitizers.mdx b/docs/reference/event-sanitizers.mdx new file mode 100644 index 000000000..bfc7ed786 --- /dev/null +++ b/docs/reference/event-sanitizers.mdx @@ -0,0 +1,181 @@ +--- +title: "Event Sanitizers" +description: "Sanitize mark and scope event fields before subscribers and exporters receive them." +position: 3 +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Event sanitizers rewrite observability fields on mark and scope events before +the events reach subscribers or exporters. Use them for event data that is not +covered by the tool and LLM request or response sanitizer APIs. + +## Sanitizer Surfaces + +NeMo Relay provides three registries: + +| Registry | Events | +|---|---| +| Mark sanitizer | Every mark, including explicit, pending, plugin-runtime, and streaming marks | +| Scope-start sanitizer | Every scope category with `scope_category = "start"` | +| Scope-end sanitizer | Every scope category with `scope_category = "end"` | + +Scope sanitizers run for all categories, including `tool` and `llm`. For tool +and LLM events, they run after the specialized request or response sanitizers. +This ordering lets a general event policy inspect the already-sanitized event +fields without changing the real request or response. + +## Callback Contract + +The callback receives: + +- The complete event as immutable context. +- An `EventSanitizeFields` value containing `data`, `category_profile`, and + `metadata`. + +Return the three observability fields that should be applied. Event identity, +parentage, timestamps, kind, scope category, semantic category, attributes, +input/output lifecycle meaning, and schemas cannot be changed through this +API. Set a field to `null` or `None` to remove it. Preserve fields that your +policy does not change. + +Registries are priority ordered. Lower priorities run first, and each callback +receives the fields returned by the preceding callback. Invalid callback +results fail open and preserve the current fields. Binding-specific callback +error inspection remains available where the binding exposes it. + +## Registration Lifetimes + +Event sanitizers use the same ownership model as other middleware: + +- Global registrations remain active until explicitly deregistered. +- Scope-local registrations are inherited by descendants and are removed when + the owning scope closes. +- Plugin-context registrations are qualified to the component instance and + roll back if activation fails or the plugin configuration is cleared. + +## Register a Global Mark Sanitizer + +The following examples replace mark data and remove mark metadata while +preserving the category profile. + + + +```python +import nemo_relay +from nemo_relay import guardrails + +def sanitize_mark(event, fields: nemo_relay.EventSanitizeFields): + return { + "data": {"checkpoint": event.name}, + "category_profile": fields["category_profile"], + "metadata": None, + } + +guardrails.register_mark_sanitize("safe-marks", 100, sanitize_mark) +try: + nemo_relay.scope.event("planning", data={"secret": "value"}) +finally: + guardrails.deregister_mark_sanitize("safe-marks") +``` + + + +```js +const relay = require("nemo-relay-node"); + +relay.registerMarkSanitizeGuardrail("safe-marks", 100, (event, fields) => ({ + data: { checkpoint: event.name }, + categoryProfile: fields.categoryProfile, + metadata: null, +})); + +try { + relay.event("planning", null, { secret: "value" }); +} finally { + relay.deregisterMarkSanitizeGuardrail("safe-marks"); +} +``` + + + +```rust +use nemo_relay::api::registry::{ + deregister_mark_sanitize_guardrail, register_mark_sanitize_guardrail, +}; +use serde_json::json; +use std::sync::Arc; + +register_mark_sanitize_guardrail( + "safe-marks", + 100, + Arc::new(|event, mut fields| { + fields.data = Some(json!({"checkpoint": event.name()})); + fields.metadata = None; + fields + }), +)?; + +// Emit marks while the registration is active. + +deregister_mark_sanitize_guardrail("safe-marks")?; +``` + + + +## API Names + +Use the matching start and end APIs when the policy applies to duration-bearing +scope events. + +| Level | Python | Node.js | Rust | +|---|---|---|---| +| Global mark | `guardrails.register_mark_sanitize` | `registerMarkSanitizeGuardrail` | `register_mark_sanitize_guardrail` | +| Global start | `guardrails.register_scope_sanitize_start` | `registerScopeSanitizeStartGuardrail` | `register_scope_sanitize_start_guardrail` | +| Global end | `guardrails.register_scope_sanitize_end` | `registerScopeSanitizeEndGuardrail` | `register_scope_sanitize_end_guardrail` | +| Scope-local mark | `scope_local.register_mark_sanitize` | `scopeRegisterMarkSanitizeGuardrail` | `scope_register_mark_sanitize_guardrail` | +| Scope-local start | `scope_local.register_scope_sanitize_start` | `scopeRegisterScopeSanitizeStartGuardrail` | `scope_register_scope_sanitize_start_guardrail` | +| Scope-local end | `scope_local.register_scope_sanitize_end` | `scopeRegisterScopeSanitizeEndGuardrail` | `scope_register_scope_sanitize_end_guardrail` | + +Each registration API has a corresponding `deregister` API. Python +scope-local helpers accept a scope handle. Node.js and Rust scope-local helpers +accept the owning scope UUID. + +Plugin contexts expose the same three surfaces as +`register_mark_sanitize_guardrail`, +`register_scope_sanitize_start_guardrail`, and +`register_scope_sanitize_end_guardrail`, using each binding's naming +convention. + +## Dynamic Plugins + +Rust native plugins receive the three methods on their typed plugin context. +Python `grpc-v1` workers receive the same methods on `PluginContext`; callbacks +may be synchronous or asynchronous. The worker protocol remains `grpc-v1` and +adds these stable registration surface values: + +- `MARK_SANITIZE_GUARDRAIL = 30` +- `SCOPE_SANITIZE_START_GUARDRAIL = 31` +- `SCOPE_SANITIZE_END_GUARDRAIL = 32` + +Dynamic plugins should declare only the surfaces they install and should use +the plugin context so failed activation rolls back every registration. + +## Experimental C and Go Bindings + +The source-first C API uses `NemoRelayEventSanitizeCb` and exports global, +scope-local, and plugin-context registration functions for all three surfaces. +Global names begin with `nemo_relay_register_`; scope-local names begin with +`nemo_relay_scope_register_`. + +The Go binding exposes `EventSanitizeFields`, `EventSanitizeFunc`, global +`Register*SanitizeGuardrail` helpers, scope-local +`ScopeRegister*SanitizeGuardrail` helpers, and the same methods on +`PluginContext`. The `guardrails` package provides shorter aliases. + +## Related Topics + +- [Middleware](/about-nemo-relay/concepts/middleware) +- [Events](/about-nemo-relay/concepts/events) +- [Register Plugin Behavior](/build-plugins/language-binding/register-behavior) +- [PII Redaction Configuration](/configure-plugins/pii-redaction/configuration) diff --git a/docs/reference/tool-execution-intercept-outcomes.mdx b/docs/reference/tool-execution-intercept-outcomes.mdx index ca177fe83..7567f6a23 100644 --- a/docs/reference/tool-execution-intercept-outcomes.mdx +++ b/docs/reference/tool-execution-intercept-outcomes.mdx @@ -29,10 +29,12 @@ does not expose downstream pending marks. Relay retains those marks, combines them in effective middleware order, and materializes them only after the final outcome succeeds. -There is no mark-specific registration path. Use the existing global, -scope-local, or plugin-context tool execution registration APIs and return the -canonical outcome from every registered callback. Relay rejects legacy raw -intercept returns at public and dynamic-plugin boundaries. +Use the existing global, scope-local, or plugin-context tool execution +registration APIs to produce pending marks, and return the canonical outcome +from every registered callback. Use a +[mark event sanitizer](/reference/event-sanitizers) when the emitted pending +mark's `data`, `category_profile`, or `metadata` must be sanitized. Legacy raw +intercept returns are rejected at public and dynamic-plugin boundaries. ## Managed Lifecycle @@ -44,7 +46,8 @@ mark a timestamp after the tool end event. If execution or an intercept fails, Relay emits the error end event and discards accumulated pending marks. Pending marks are never included in the -application-visible tool result or passed to sanitize-response guardrails. +application-visible tool result or passed to tool sanitize-response +guardrails. They pass through the mark sanitizer registry when materialized. ## Binding Contract diff --git a/go/nemo_relay/README.md b/go/nemo_relay/README.md index 81baaba83..39985b134 100644 --- a/go/nemo_relay/README.md +++ b/go/nemo_relay/README.md @@ -40,7 +40,8 @@ primary supported surfaces. - **Scope, tool, and LLM helpers**: Managed lifecycle APIs backed by the shared Rust runtime. - **Middleware APIs**: Guardrails and intercepts for request rewriting, - blocking, sanitization, and execution wrapping. + blocking, sanitization, and execution wrapping, including mark and scope + event sanitizers at global, scope-local, and plugin-context levels. - **Event subscribers**: Runtime lifecycle callbacks for observability and diagnostics. - **Convenience subpackages**: Short imports for scopes, tools, LLM calls, diff --git a/python/nemo_relay/README.md b/python/nemo_relay/README.md index 6d6e2fac9..c377f6e7d 100644 --- a/python/nemo_relay/README.md +++ b/python/nemo_relay/README.md @@ -41,7 +41,8 @@ runtime semantics as the Rust and Node.js surfaces. - **Scope, tool, and LLM helpers**: Managed boundaries that emit lifecycle events and run middleware in a consistent order. - **Middleware APIs**: Guardrails and intercepts for tool and LLM requests, - responses, and execution. + responses, and execution, plus mark and scope event sanitizers for `data`, + `category_profile`, and `metadata`. - **Subscribers and exporters**: Event consumers for observability and diagnostics. - **Plugin and typed helpers**: Public modules for plugins, codecs, typed diff --git a/skills/nemo-relay-export-atif-trajectories/SKILL.md b/skills/nemo-relay-export-atif-trajectories/SKILL.md index 73cceb6f8..d890af256 100644 --- a/skills/nemo-relay-export-atif-trajectories/SKILL.md +++ b/skills/nemo-relay-export-atif-trajectories/SKILL.md @@ -50,8 +50,8 @@ live OTLP spans. - ATIF exports the full event buffer collected so far. - Consecutive tool observations can be merged into one system observation step. -- Trajectories reflect sanitized event payloads, not raw secrets that - sanitize guardrails removed before event emission. +- Trajectories reflect sanitized event payloads, not raw secrets that tool, + LLM, mark, or scope event sanitizers removed before event emission. - Response codecs can improve LLM end annotations, but they do not change the caller-visible LLM response. From d3845c2cafdeb3aa1f149a969e5771083f6525ac Mon Sep 17 00:00:00 2001 From: Will Killian Date: Tue, 14 Jul 2026 22:40:42 -0400 Subject: [PATCH 2/9] docs: align event sanitizer guidance Signed-off-by: Will Killian --- .agents/skills/add-middleware/SKILL.md | 23 ++++---- docs/about-nemo-relay/concepts/middleware.mdx | 40 +++++++++----- .../pii-redaction/configuration.mdx | 55 ++++++++++--------- docs/reference/event-sanitizers.mdx | 16 +++--- 4 files changed, 74 insertions(+), 60 deletions(-) diff --git a/.agents/skills/add-middleware/SKILL.md b/.agents/skills/add-middleware/SKILL.md index 13dd674b6..6512b3db5 100644 --- a/.agents/skills/add-middleware/SKILL.md +++ b/.agents/skills/add-middleware/SKILL.md @@ -31,14 +31,14 @@ Decide these before editing code: final response output? - Is the callback fallible, and how should callback failures propagate? - Does it need both global and scope-local registration? -- What should subscribers observe in `event.input` and `event.output` after this +- What should subscribers and exporters observe in the event payload after this middleware runs? - If this is an event sanitizer, which of `data`, `category_profile`, and `metadata` may change, and is the event used only as immutable context? ## Pipeline Order -See `docs/about/concepts/middleware.md` for the full diagrams. +See `docs/about-nemo-relay/concepts/middleware.mdx` for the full diagrams. - **Tool execute**: conditional guardrails -> request intercepts -> sanitize request (for events) @@ -47,8 +47,8 @@ See `docs/about/concepts/middleware.md` for the full diagrams. conditional guardrails -> request intercepts -> sanitize request (for events) | execution intercept chain(callable) -> sanitize response - **Mark and scope events**: - specialized tool/LLM sanitizer (when applicable) -> mark or scope event - sanitizer -> subscriber dispatch + specialized tool or LLM sanitizer (when applicable) -> mark or scope event + sanitizer -> subscriber and exporter dispatch ## Core Steps @@ -78,8 +78,11 @@ Follow the pattern of `tool_sanitize_request_chain` or `tool_request_intercepts_ 5. Wire the chain into the execute path. -Update `crates/core/src/api/tool.rs` or `crates/core/src/api/llm.rs` to call -the new chain method at the appropriate pipeline stage. +Update the relevant lifecycle owner to call the new chain method at the +appropriate pipeline stage. Tool and LLM paths live in +`crates/core/src/api/tool.rs` and `crates/core/src/api/llm.rs`; shared mark and +scope event sanitization lives in `crates/core/src/api/shared.rs` and is called +from `crates/core/src/api/scope.rs`. 6. Expose the new middleware surface in every affected binding. @@ -92,7 +95,7 @@ Follow the `add-binding-feature` skill for the cross-binding implementation chec - [ ] Ordering by priority - [ ] Callback error propagation - [ ] Scope-local registration, inheritance, and cleanup on pop -- [ ] Event input/output semantics after middleware mutation +- [ ] Event payload semantics after middleware mutation - [ ] Mark and scope event field semantics, including immutable identity fields - [ ] Parity coverage in every affected binding @@ -103,7 +106,7 @@ Follow the `add-binding-feature` skill for the cross-binding implementation chec - Runtime state and chain builders: `crates/core/src/api/runtime/state.rs` - Scope-local registry merging: `crates/core/src/context/registries.rs` - Registry: `crates/core/src/registry.rs` -- Pipeline docs: `docs/about/concepts/middleware.md` -- Architecture docs: `docs/about/architecture.md` -- Registration examples: `docs/instrument-applications/advanced-guide.md` +- Pipeline docs: `docs/about-nemo-relay/concepts/middleware.mdx` +- Architecture docs: `docs/about-nemo-relay/architecture.mdx` +- Registration examples: `docs/instrument-applications/advanced-guide.mdx` - Validation: `validate-change` diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index 08cc96d38..10289500c 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -140,9 +140,10 @@ sequenceDiagram participant Req as Request Intercepts participant Exec as Execution Intercepts participant Callback as Real Callback - participant San as Sanitize Guardrails + participant San as Tool / LLM Sanitizers + participant EventSan as Scope Event Sanitizers participant Dispatch as Async Subscriber Dispatcher - participant Subs as Subscribers + participant Consumers as Subscribers / Exporters Caller->>Runtime: managed tool or LLM call Runtime->>Cond: decide whether work may proceed @@ -152,32 +153,37 @@ sequenceDiagram else allowed Runtime->>Req: rewrite the real request Runtime->>San: sanitize emitted start payload + Runtime->>EventSan: sanitize start event fields Runtime->>Dispatch: enqueue start event before execution - Dispatch-->>Subs: deliver start event later + Dispatch-->>Consumers: deliver start event later Runtime->>Exec: wrap execution Exec->>Callback: invoke callback Callback-->>Exec: return real result Exec-->>Runtime: continue Runtime->>San: sanitize emitted end payload + Runtime->>EventSan: sanitize end event fields Runtime->>Dispatch: enqueue end event - Dispatch-->>Subs: deliver end event later + Dispatch-->>Consumers: deliver end event later Runtime-->>Caller: return real result end ``` 1. Conditional-execution guardrails 2. Request intercepts -3. Sanitize-request guardrails and emit the start event -4. Execution intercepts -5. The real callback, unless an execution intercept replaces it -6. Sanitize-response guardrails and emit the end event +3. Tool or LLM sanitize-request guardrails +4. Scope-start event sanitizers and start-event emission +5. Execution intercepts +6. The real callback, unless an execution intercept replaces it +7. Tool or LLM sanitize-response guardrails +8. Scope-end event sanitizers and end-event emission For streaming LLM flows, the same pre-execution order applies: the runtime applies `sanitize-request` guardrails and emits the LLM start event before the stream execution intercept chain runs. Stream execution intercepts are the execution family for streaming provider callbacks. The runtime then collects chunks and finalizes the stream before `sanitize-response` guardrails rewrite -the emitted end-event payload at item 6. +the emitted end-event payload and scope-end event sanitizers run at items 7 and +8. This ordering is what makes the semantic split between intercepts and guardrails important: @@ -221,10 +227,12 @@ flowchart TB direction TB SanitizeRequestGuardrails[/Sanitize Request Guardrail/] SanitizeResponseGuardrails[/Sanitize Response Guardrail/] + ScopeStartSanitizers[/Scope-Start Event Sanitizers/] + ScopeEndSanitizers[/Scope-End Event Sanitizers/] StartEvent[Emit Start Event] EndEvent[Emit End Event] Dispatcher[["Async Subscriber Dispatcher"]] - EventSubscribers[["Event Subscribers"]] + EventConsumers[["Subscribers / Exporters"]] end end @@ -235,9 +243,10 @@ flowchart TB ConditionalExecutionGuardrails -->|"(rejected event)"| Dispatcher ConditionalExecutionGuardrails -->|"(rejected)"| RaiseException ConditionalExecutionGuardrails -->|"(passed)"| RequestIntercepts - SanitizeRequestGuardrails -->|Sanitized Start Payload| StartEvent + SanitizeRequestGuardrails -->|Sanitized Start Payload| ScopeStartSanitizers + ScopeStartSanitizers -->|Sanitized Event Fields| StartEvent StartEvent --> Dispatcher - Dispatcher --> EventSubscribers + Dispatcher --> EventConsumers StartEvent -->|Before Execution Intercepts| HasExecutionIntercept RequestIntercepts -.->|Real Request| HasExecutionIntercept @@ -256,13 +265,14 @@ flowchart TB Finalizer -.->|Aggregated Response| SanitizeResponseGuardrails Finalizer o--o|shared state| Collector - SanitizeResponseGuardrails -->|Sanitized End Payload| EndEvent + SanitizeResponseGuardrails -->|Sanitized End Payload| ScopeEndSanitizers + ScopeEndSanitizers -->|Sanitized Event Fields| EndEvent EndEvent --> Dispatcher class Execution,Invocation,Streaming,Observability,Request,Response grey-lightest; - class Dispatcher,EventSubscribers,StartEvent,EndEvent teal-lightest; + class Dispatcher,EventConsumers,StartEvent,EndEvent teal-lightest; class RequestIntercepts,HasExecutionIntercept,ExecutionIntercepts yellow-lightest; - class ConditionalExecutionGuardrails,SanitizeRequestGuardrails,SanitizeResponseGuardrails green-lightest; + class ConditionalExecutionGuardrails,SanitizeRequestGuardrails,SanitizeResponseGuardrails,ScopeStartSanitizers,ScopeEndSanitizers green-lightest; class RaiseException red-lightest; class DefaultCallable,InterceptResult,Collector,Finalizer magenta-lightest; ``` diff --git a/docs/configure-plugins/pii-redaction/configuration.mdx b/docs/configure-plugins/pii-redaction/configuration.mdx index fca02822f..160a1ca3a 100644 --- a/docs/configure-plugins/pii-redaction/configuration.mdx +++ b/docs/configure-plugins/pii-redaction/configuration.mdx @@ -46,8 +46,8 @@ the plugin contract. Use this matrix when deciding whether to use the built-in `pii_redaction` plugin or raw sanitize-guardrail middleware directly. -| Area | `pii_redaction` plugin | Raw middleware | -|---|---|---| +| Area | `pii_redaction` Plugin | Raw Middleware | +| --- | --- | --- | | Primary UX | Declarative config through `plugins.toml`, bindings helpers, and the CLI editor | Callback registration in application code | | Best fit | Reusable privacy policy shared across apps or teams | App-specific logic tied to runtime state or custom heuristics | | Built-in actions | `remove`, `redact`, `regex_replace`, `hash`, `mask` | You implement the behavior yourself | @@ -63,15 +63,15 @@ plugin or raw sanitize-guardrail middleware directly. The top-level PII redaction object contains: | Field | Purpose | -|---|---| +| --- | --- | | `version` | PII redaction config schema version. Defaults to `1`. | -| `mode` | Backend mode. Current values are `builtin` and `local_model`. | -| `input` | Enables managed LLM request sanitization and non-tool/LLM scope-start field sanitization. | -| `output` | Enables managed LLM response sanitization and non-tool/LLM scope-end field sanitization. | +| `mode` | Backend mode. Current values are `builtin` and `local_model`. Defaults to `builtin`. | +| `input` | Enables managed LLM request sanitization and non-tool, non-LLM scope-start field sanitization. Defaults to `true`. | +| `output` | Enables managed LLM response sanitization and non-tool, non-LLM scope-end field sanitization. Defaults to `true`. | | `mark` | Enables mark `data`, `category_profile`, and `metadata` sanitization. Defaults to `true`. | -| `tool_input` | Enables sanitization of emitted tool-request observability payloads. | -| `tool_output` | Enables sanitization of emitted tool-response observability payloads. | -| `priority` | Guardrail priority. Lower values run earlier. | +| `tool_input` | Enables sanitization of emitted tool-request observability payloads. Defaults to `true`. | +| `tool_output` | Enables sanitization of emitted tool-response observability payloads. Defaults to `true`. | +| `priority` | Guardrail priority. Lower values run earlier. Defaults to `100`. | | `codec` | Managed LLM provider codec. Required when `input` or `output` is enabled. | | `builtin` | Built-in backend settings used when `mode = "builtin"`. | | `local` | Local-backend settings used when `mode = "local_model"`. | @@ -84,7 +84,7 @@ At least one managed redaction surface must be enabled. The following table compares the available PII redaction backends: | Area | `builtin` | `local_model` | -|---|---|---| +| --- | --- | --- | | Built-in component kind and config validation | Supported | Supported | | Managed LLM `input` | Supported | Not implemented | | Managed LLM `output` | Supported | Not implemented | @@ -161,8 +161,8 @@ This example configures the built-in backend for: - LLM response redaction from the normalized response path `/message` - tool argument redaction at `/api_key` - tool result redaction at `/result/secret` -- mark and non-tool/LLM scope event redaction across `data`, - `category_profile`, and `metadata` +- mark-event and non-tool, non-LLM scope-event redaction at matching target paths within + `data`, `category_profile`, and `metadata` `mark` is enabled when omitted. Set `mark = false` only when marks are known to contain no sensitive observability fields or another sanitizer owns that @@ -195,7 +195,7 @@ use raw middleware. The `builtin` section contains: | Field | Purpose | -|---|---| +| --- | --- | | `action` | Sanitization action. Current values are `remove`, `redact`, `regex_replace`, `hash`, and `mask`. | | `target_paths` | Exact JSON-pointer paths to sanitize. Empty means every matching string leaf. | | `pattern` | Regex pattern used when `action = "regex_replace"` or `action = "redact"`. | @@ -287,7 +287,7 @@ The plugin uses different payload boundaries for tools and LLMs: - LLMs use the selected built-in codec. Prefer normalized Relay paths such as: - `/messages/0/content` for request message content - `/message` for the normalized assistant response text -- Marks and non-tool/LLM scopes sanitize `data`, `category_profile`, and +- Marks and non-tool, non-LLM scopes sanitize `data`, `category_profile`, and `metadata` as separate JSON values. A target path is evaluated independently inside each value; it does not start with `/data`, `/category_profile`, or `/metadata`. @@ -345,19 +345,20 @@ The built-in plugin uses sanitize guardrails and event sanitizers. That means: -- the real provider response value is unchanged -- the emitted NeMo Relay start, end, or mark event observability fields are - sanitized -- `annotated_response` is populated from the sanitized end-event payload when a - response codec is provided -- subscribers, ATOF, ATIF, OpenTelemetry, and OpenInference receive the - sanitized event; raw values removed by the plugin do not bypass through an - exporter - -For tool and LLM scope events, the generic scope sanitizer leaves the shared -fields unchanged because the specialized tool or LLM sanitizer already owns -that lifecycle boundary. This prevents repeated hashing or masking. Generic -scope categories use `input` for starts and `output` for ends. +- The real provider request and response values are unchanged. +- The emitted NeMo Relay start, end, or mark event observability fields are + sanitized. +- `annotated_response` is regenerated from the sanitized end-event payload when + the configured response codec can decode that payload. +- Subscribers and exporters consume only the sanitized canonical event stream. + Each exporter can omit or transform fields as part of its normal projection, + but raw values removed by the plugin do not bypass the sanitizer. + +For tool and LLM scope events, the PII redaction scope sanitizer leaves `data`, +`category_profile`, and `metadata` unchanged because the specialized tool or +LLM sanitizer already owns that lifecycle boundary. This prevents repeated +hashing or masking. Other scope categories use `input` for starts and `output` +for ends. ## Local Model Mode diff --git a/docs/reference/event-sanitizers.mdx b/docs/reference/event-sanitizers.mdx index bfc7ed786..92944db92 100644 --- a/docs/reference/event-sanitizers.mdx +++ b/docs/reference/event-sanitizers.mdx @@ -15,7 +15,7 @@ covered by the tool and LLM request or response sanitizer APIs. NeMo Relay provides three registries: | Registry | Events | -|---|---| +| --- | --- | | Mark sanitizer | Every mark, including explicit, pending, plugin-runtime, and streaming marks | | Scope-start sanitizer | Every scope category with `scope_category = "start"` | | Scope-end sanitizer | Every scope category with `scope_category = "end"` | @@ -35,14 +35,14 @@ The callback receives: Return the three observability fields that should be applied. Event identity, parentage, timestamps, kind, scope category, semantic category, attributes, -input/output lifecycle meaning, and schemas cannot be changed through this -API. Set a field to `null` or `None` to remove it. Preserve fields that your -policy does not change. +semantic input and output meaning, and schemas cannot be changed through this +API. Set a field to `None` in Rust or Python, or `null` in Node.js, to remove +it. Preserve fields that your policy does not change. Registries are priority ordered. Lower priorities run first, and each callback -receives the fields returned by the preceding callback. Invalid callback -results fail open and preserve the current fields. Binding-specific callback -error inspection remains available where the binding exposes it. +receives the fields returned by the preceding callback. Invalid binding +callback results fail open and preserve the current fields. Node.js records +these failures for inspection through `getLastCallbackError()`. ## Registration Lifetimes @@ -129,7 +129,7 @@ Use the matching start and end APIs when the policy applies to duration-bearing scope events. | Level | Python | Node.js | Rust | -|---|---|---|---| +| --- | --- | --- | --- | | Global mark | `guardrails.register_mark_sanitize` | `registerMarkSanitizeGuardrail` | `register_mark_sanitize_guardrail` | | Global start | `guardrails.register_scope_sanitize_start` | `registerScopeSanitizeStartGuardrail` | `register_scope_sanitize_start_guardrail` | | Global end | `guardrails.register_scope_sanitize_end` | `registerScopeSanitizeEndGuardrail` | `register_scope_sanitize_end_guardrail` | From 4916972845f972efbcbb1bda43f15fb14bfcb673 Mon Sep 17 00:00:00 2001 From: Will Killian Date: Tue, 14 Jul 2026 23:24:25 -0400 Subject: [PATCH 3/9] docs: clarify event sanitizer replacement semantics Signed-off-by: Will Killian --- .agents/skills/add-middleware/SKILL.md | 2 +- docs/about-nemo-relay/concepts/middleware.mdx | 14 ++++++++------ .../pii-redaction/configuration.mdx | 2 +- docs/reference/event-sanitizers.mdx | 10 +++++++--- .../references/atif.md | 8 ++++---- 5 files changed, 21 insertions(+), 15 deletions(-) diff --git a/.agents/skills/add-middleware/SKILL.md b/.agents/skills/add-middleware/SKILL.md index 6512b3db5..63ed37fca 100644 --- a/.agents/skills/add-middleware/SKILL.md +++ b/.agents/skills/add-middleware/SKILL.md @@ -93,7 +93,7 @@ Follow the `add-binding-feature` skill for the cross-binding implementation chec - [ ] Registration and duplicate-name behavior - [ ] Deregistration and no-op missing-name behavior - [ ] Ordering by priority -- [ ] Callback error propagation +- [ ] Callback failure policy, including fail-open behavior when required - [ ] Scope-local registration, inheritance, and cleanup on pop - [ ] Event payload semantics after middleware mutation - [ ] Mark and scope event field semantics, including immutable identity fields diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index 10289500c..8b4011b63 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -13,9 +13,8 @@ calls and sanitizes emitted mark and scope events. ## What Middleware Is -Middleware is the runtime behavior that runs around tool and LLM execution. -NeMo Relay uses middleware to control, transform, or observe work at specific -lifecycle points. +Middleware controls or transforms tool and LLM execution and sanitizes emitted +events. NeMo Relay applies each surface at a specific lifecycle point. Middleware is organized by lifecycle meaning rather than as one undifferentiated hook system. @@ -141,7 +140,7 @@ sequenceDiagram participant Exec as Execution Intercepts participant Callback as Real Callback participant San as Tool / LLM Sanitizers - participant EventSan as Scope Event Sanitizers + participant EventSan as Mark / Scope Event Sanitizers participant Dispatch as Async Subscriber Dispatcher participant Consumers as Subscribers / Exporters @@ -149,6 +148,7 @@ sequenceDiagram Runtime->>Cond: decide whether work may proceed alt blocked + Runtime->>EventSan: sanitize guardrail scopes and rejection mark Cond-->>Caller: reject execution else allowed Runtime->>Req: rewrite the real request @@ -227,6 +227,7 @@ flowchart TB direction TB SanitizeRequestGuardrails[/Sanitize Request Guardrail/] SanitizeResponseGuardrails[/Sanitize Response Guardrail/] + MarkSanitizers[/Mark Event Sanitizers/] ScopeStartSanitizers[/Scope-Start Event Sanitizers/] ScopeEndSanitizers[/Scope-End Event Sanitizers/] StartEvent[Emit Start Event] @@ -240,7 +241,8 @@ flowchart TB Request --> ConditionalExecutionGuardrails RequestIntercepts -->|Transformed Request| SanitizeRequestGuardrails - ConditionalExecutionGuardrails -->|"(rejected event)"| Dispatcher + ConditionalExecutionGuardrails -->|"(rejection mark)"| MarkSanitizers + MarkSanitizers -->|Sanitized Mark Fields| Dispatcher ConditionalExecutionGuardrails -->|"(rejected)"| RaiseException ConditionalExecutionGuardrails -->|"(passed)"| RequestIntercepts SanitizeRequestGuardrails -->|Sanitized Start Payload| ScopeStartSanitizers @@ -272,7 +274,7 @@ flowchart TB class Execution,Invocation,Streaming,Observability,Request,Response grey-lightest; class Dispatcher,EventConsumers,StartEvent,EndEvent teal-lightest; class RequestIntercepts,HasExecutionIntercept,ExecutionIntercepts yellow-lightest; - class ConditionalExecutionGuardrails,SanitizeRequestGuardrails,SanitizeResponseGuardrails,ScopeStartSanitizers,ScopeEndSanitizers green-lightest; + class ConditionalExecutionGuardrails,SanitizeRequestGuardrails,SanitizeResponseGuardrails,MarkSanitizers,ScopeStartSanitizers,ScopeEndSanitizers green-lightest; class RaiseException red-lightest; class DefaultCallable,InterceptResult,Collector,Finalizer magenta-lightest; ``` diff --git a/docs/configure-plugins/pii-redaction/configuration.mdx b/docs/configure-plugins/pii-redaction/configuration.mdx index 160a1ca3a..d3991a7e6 100644 --- a/docs/configure-plugins/pii-redaction/configuration.mdx +++ b/docs/configure-plugins/pii-redaction/configuration.mdx @@ -46,7 +46,7 @@ the plugin contract. Use this matrix when deciding whether to use the built-in `pii_redaction` plugin or raw sanitize-guardrail middleware directly. -| Area | `pii_redaction` Plugin | Raw Middleware | +| Area | `pii_redaction` plugin | Raw middleware | | --- | --- | --- | | Primary UX | Declarative config through `plugins.toml`, bindings helpers, and the CLI editor | Callback registration in application code | | Best fit | Reusable privacy policy shared across apps or teams | App-specific logic tied to runtime state or custom heuristics | diff --git a/docs/reference/event-sanitizers.mdx b/docs/reference/event-sanitizers.mdx index 92944db92..c24d4771f 100644 --- a/docs/reference/event-sanitizers.mdx +++ b/docs/reference/event-sanitizers.mdx @@ -36,8 +36,10 @@ The callback receives: Return the three observability fields that should be applied. Event identity, parentage, timestamps, kind, scope category, semantic category, attributes, semantic input and output meaning, and schemas cannot be changed through this -API. Set a field to `None` in Rust or Python, or `null` in Node.js, to remove -it. Preserve fields that your policy does not change. +API. The return value is a complete replacement, not a patch. Start from the +provided fields and return all three. Omitting a field can clear it in bindings +that accept a partial object. Set a field to `None` in Rust or Python, `null` in +Node.js, or a zero-value `json.RawMessage` in Go to remove it. Registries are priority ordered. Lower priorities run first, and each callback receives the fields returned by the preceding callback. Invalid binding @@ -171,7 +173,9 @@ Global names begin with `nemo_relay_register_`; scope-local names begin with The Go binding exposes `EventSanitizeFields`, `EventSanitizeFunc`, global `Register*SanitizeGuardrail` helpers, scope-local `ScopeRegister*SanitizeGuardrail` helpers, and the same methods on -`PluginContext`. The `guardrails` package provides shorter aliases. +`PluginContext`. The `guardrails` package provides shorter aliases. A returned +`EventSanitizeFields` replaces all three fields, so copy the provided value and +modify only the fields that should change. ## Related Topics diff --git a/skills/nemo-relay-plugin-observability/references/atif.md b/skills/nemo-relay-plugin-observability/references/atif.md index ab7399a6c..a22c951ba 100644 --- a/skills/nemo-relay-plugin-observability/references/atif.md +++ b/skills/nemo-relay-plugin-observability/references/atif.md @@ -19,8 +19,8 @@ live OTLP spans. ## Embedded ATIF Semantics - ATIF export translates NeMo Relay events into ATIF v1.7 trajectory data. -- LLM start events become `user` steps; message content is extracted from the - `LLMRequest.content` payload when possible. +- LLM start events become `user` steps. ATIF uses the latest user message from + the request annotation when possible, then falls back to the raw event input. - LLM end events become `agent` steps with response content, model metadata, token metrics, reasoning fields, and promoted `tool_calls` when the response uses a supported tool-call shape. @@ -28,8 +28,8 @@ live OTLP spans. promoted from the preceding LLM end response. - Tool end events become `system` observations. Observations are correlated to promoted tool calls by function name and source call ID when available. -- Mark events with data become `system` steps. Scope start/end events are - structural and are not emitted as trajectory steps. +- Point-in-time mark events and scope start/end events are structural and are + not emitted as trajectory steps. - Scope nesting becomes ancestry metadata on exported steps. - Nested agent scopes become embedded `subagent_trajectories` with `subagent_trajectory_ref` observations in the parent trajectory. From c6ab8c08c8f09fdfe56075ac121a4eb794e264cd Mon Sep 17 00:00:00 2001 From: Will Killian Date: Tue, 14 Jul 2026 23:28:44 -0400 Subject: [PATCH 4/9] docs: title case sanitizer table headers Signed-off-by: Will Killian --- docs/configure-plugins/pii-redaction/configuration.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configure-plugins/pii-redaction/configuration.mdx b/docs/configure-plugins/pii-redaction/configuration.mdx index d3991a7e6..160a1ca3a 100644 --- a/docs/configure-plugins/pii-redaction/configuration.mdx +++ b/docs/configure-plugins/pii-redaction/configuration.mdx @@ -46,7 +46,7 @@ the plugin contract. Use this matrix when deciding whether to use the built-in `pii_redaction` plugin or raw sanitize-guardrail middleware directly. -| Area | `pii_redaction` plugin | Raw middleware | +| Area | `pii_redaction` Plugin | Raw Middleware | | --- | --- | --- | | Primary UX | Declarative config through `plugins.toml`, bindings helpers, and the CLI editor | Callback registration in application code | | Best fit | Reusable privacy policy shared across apps or teams | App-specific logic tied to runtime state or custom heuristics | From d31508d8a81c0c361c74f0462a2cf73649168d32 Mon Sep 17 00:00:00 2001 From: Will Killian Date: Tue, 14 Jul 2026 23:31:15 -0400 Subject: [PATCH 5/9] docs: add complete sanitizer list lead-ins Signed-off-by: Will Killian --- crates/node/README.md | 4 ++++ docs/reference/event-sanitizers.mdx | 2 ++ go/nemo_relay/README.md | 4 ++++ python/nemo_relay/README.md | 4 ++++ 4 files changed, 14 insertions(+) diff --git a/crates/node/README.md b/crates/node/README.md index 3ff1cca3d..c3820a132 100644 --- a/crates/node/README.md +++ b/crates/node/README.md @@ -25,6 +25,8 @@ should install it from npm rather than depend on the Rust crate directly. ## Why Use It? +Use the Node.js binding for the following tasks: + - **Own execution context in Node.js**: Group agent, tool, and LLM work into one scope tree from JavaScript or TypeScript. - **Put policy around callbacks**: Register guardrails and intercepts for @@ -37,6 +39,8 @@ should install it from npm rather than depend on the Rust crate directly. ## What You Get +The Node.js package provides the following capabilities: + - **npm package for Node.js**: A Node.js 24 or newer package backed by a napi-rs native extension. - **Managed tool and LLM execution**: Helpers that emit lifecycle events and diff --git a/docs/reference/event-sanitizers.mdx b/docs/reference/event-sanitizers.mdx index c24d4771f..fb629a744 100644 --- a/docs/reference/event-sanitizers.mdx +++ b/docs/reference/event-sanitizers.mdx @@ -179,6 +179,8 @@ modify only the fields that should change. ## Related Topics +For related concepts and configuration, refer to these topics: + - [Middleware](/about-nemo-relay/concepts/middleware) - [Events](/about-nemo-relay/concepts/events) - [Register Plugin Behavior](/build-plugins/language-binding/register-behavior) diff --git a/go/nemo_relay/README.md b/go/nemo_relay/README.md index 51201a2f5..813b7f45e 100644 --- a/go/nemo_relay/README.md +++ b/go/nemo_relay/README.md @@ -31,6 +31,8 @@ primary supported surfaces. ## Why Use It? +Use the Go binding for the following tasks: + - **Use NeMo Relay from Go**: Group agent, tool, and LLM work into the same scope and lifecycle model as the Rust runtime. - **Bridge through CGo and FFI**: Consume the shared runtime through the @@ -42,6 +44,8 @@ primary supported surfaces. ## What You Get +The Go package provides the following capabilities: + - **Scope, tool, and LLM helpers**: Managed lifecycle APIs backed by the shared Rust runtime. - **Middleware APIs**: Guardrails and intercepts for request rewriting, diff --git a/python/nemo_relay/README.md b/python/nemo_relay/README.md index c377f6e7d..0927b4f04 100644 --- a/python/nemo_relay/README.md +++ b/python/nemo_relay/README.md @@ -26,6 +26,8 @@ runtime semantics as the Rust and Node.js surfaces. ## Why Use It? +Use the Python binding for the following tasks: + - **Own execution context in Python**: Group agent, tool, and LLM work into one scope tree from Python application code. - **Package policy around callbacks**: Use guardrails and intercepts to block @@ -38,6 +40,8 @@ runtime semantics as the Rust and Node.js surfaces. ## What You Get +The Python package provides the following capabilities: + - **Scope, tool, and LLM helpers**: Managed boundaries that emit lifecycle events and run middleware in a consistent order. - **Middleware APIs**: Guardrails and intercepts for tool and LLM requests, From b8b984c0b6b4d162e7ec09a5a69c9e4e3681090f Mon Sep 17 00:00:00 2001 From: Will Killian Date: Tue, 14 Jul 2026 23:38:49 -0400 Subject: [PATCH 6/9] docs: harmonize shared ATIF guidance Signed-off-by: Will Killian --- .../nemo-relay-plugin-observability/references/atif.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/skills/nemo-relay-plugin-observability/references/atif.md b/skills/nemo-relay-plugin-observability/references/atif.md index a22c951ba..05558e2c1 100644 --- a/skills/nemo-relay-plugin-observability/references/atif.md +++ b/skills/nemo-relay-plugin-observability/references/atif.md @@ -19,8 +19,14 @@ live OTLP spans. ## Embedded ATIF Semantics - ATIF export translates NeMo Relay events into ATIF v1.7 trajectory data. -- LLM start events become `user` steps. ATIF uses the latest user message from - the request annotation when possible, then falls back to the raw event input. +- LLM start events become `user` steps. ATIF extracts the latest user message + from the request annotation when possible. It does not export the annotation's + complete history as the step message. Each owning agent scope starts fresh, + and a `compaction` mark refreshes it. The first subsequent LLM start annotation + retains complete history. Later starts retain system instructions, the latest + user message, and every following assistant or tool message. When a request + codec supplies an annotation, the event input uses the same projection. + Provider execution remains unchanged. - LLM end events become `agent` steps with response content, model metadata, token metrics, reasoning fields, and promoted `tool_calls` when the response uses a supported tool-call shape. From 8230db83c5f462398aee3c679d8f1b4d7d608a1b Mon Sep 17 00:00:00 2001 From: Will Killian Date: Wed, 15 Jul 2026 09:24:42 -0400 Subject: [PATCH 7/9] docs: use possibility wording in middleware skill Signed-off-by: Will Killian --- .agents/skills/add-middleware/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/add-middleware/SKILL.md b/.agents/skills/add-middleware/SKILL.md index 63ed37fca..78fff533d 100644 --- a/.agents/skills/add-middleware/SKILL.md +++ b/.agents/skills/add-middleware/SKILL.md @@ -34,7 +34,7 @@ Decide these before editing code: - What should subscribers and exporters observe in the event payload after this middleware runs? - If this is an event sanitizer, which of `data`, `category_profile`, and - `metadata` may change, and is the event used only as immutable context? + `metadata` can change, and is the event used only as immutable context? ## Pipeline Order From a9ad740b81ce9190e7118541d9959d6a4ad53dce Mon Sep 17 00:00:00 2001 From: Will Killian <2007799+willkill07@users.noreply.github.com> Date: Wed, 15 Jul 2026 18:56:44 -0400 Subject: [PATCH 8/9] Update .agents/skills/add-middleware/SKILL.md Co-authored-by: lvojtku Signed-off-by: Will Killian <2007799+willkill07@users.noreply.github.com> --- .agents/skills/add-middleware/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/add-middleware/SKILL.md b/.agents/skills/add-middleware/SKILL.md index 78fff533d..40a38bbfd 100644 --- a/.agents/skills/add-middleware/SKILL.md +++ b/.agents/skills/add-middleware/SKILL.md @@ -38,7 +38,7 @@ Decide these before editing code: ## Pipeline Order -See `docs/about-nemo-relay/concepts/middleware.mdx` for the full diagrams. +Refer to `docs/about-nemo-relay/concepts/middleware.mdx` for the full diagrams. - **Tool execute**: conditional guardrails -> request intercepts -> sanitize request (for events) From 4993c7de9fd2e575e6cf6188f2dadaeae919e6df Mon Sep 17 00:00:00 2001 From: Will Killian Date: Wed, 15 Jul 2026 19:06:08 -0400 Subject: [PATCH 9/9] docs: make event sanitizer guide more conversational Signed-off-by: Will Killian --- docs/reference/event-sanitizers.mdx | 112 +++++++++++++++------------- 1 file changed, 60 insertions(+), 52 deletions(-) diff --git a/docs/reference/event-sanitizers.mdx b/docs/reference/event-sanitizers.mdx index fb629a744..feadbd37b 100644 --- a/docs/reference/event-sanitizers.mdx +++ b/docs/reference/event-sanitizers.mdx @@ -6,13 +6,15 @@ position: 3 {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} -Event sanitizers rewrite observability fields on mark and scope events before -the events reach subscribers or exporters. Use them for event data that is not -covered by the tool and LLM request or response sanitizer APIs. +Event sanitizers let you remove or replace observability data on mark and +scope events before the runtime sends the events to subscribers or exporters. +Use them for event data that the tool and LLM request or response sanitizer +APIs do not cover. They do not change the real request, response, or callback +result. ## Sanitizer Surfaces -NeMo Relay provides three registries: +Choose a registry based on the event that you want to sanitize. | Registry | Events | | --- | --- | @@ -20,41 +22,45 @@ NeMo Relay provides three registries: | Scope-start sanitizer | Every scope category with `scope_category = "start"` | | Scope-end sanitizer | Every scope category with `scope_category = "end"` | -Scope sanitizers run for all categories, including `tool` and `llm`. For tool -and LLM events, they run after the specialized request or response sanitizers. -This ordering lets a general event policy inspect the already-sanitized event -fields without changing the real request or response. +Scope sanitizers apply to every scope category, including `tool` and `llm`. +For tool and LLM events, the specialized request or response sanitizers run +first. Your scope sanitizer then receives the already-sanitized event fields, +which lets a general event policy inspect them without changing the real +request or response. ## Callback Contract -The callback receives: +When the runtime calls a sanitizer, it provides the following values: - The complete event as immutable context. -- An `EventSanitizeFields` value containing `data`, `category_profile`, and +- An `EventSanitizeFields` value with `data`, `category_profile`, and `metadata`. -Return the three observability fields that should be applied. Event identity, -parentage, timestamps, kind, scope category, semantic category, attributes, -semantic input and output meaning, and schemas cannot be changed through this -API. The return value is a complete replacement, not a patch. Start from the -provided fields and return all three. Omitting a field can clear it in bindings -that accept a partial object. Set a field to `None` in Rust or Python, `null` in -Node.js, or a zero-value `json.RawMessage` in Go to remove it. +Return the replacement values for those three observability fields. The return +value replaces the fields; it does not patch them. Start with the supplied +fields, modify the fields that need to change, and return all three. In +bindings that accept a partial object, omitting a field can clear it. To remove +a field explicitly, use `None` in Rust or Python, `null` in Node.js, or a +zero-value `json.RawMessage` in Go. -Registries are priority ordered. Lower priorities run first, and each callback -receives the fields returned by the preceding callback. Invalid binding -callback results fail open and preserve the current fields. Node.js records -these failures for inspection through `getLastCallbackError()`. +Sanitizers can change only `data`, `category_profile`, and `metadata`. They +cannot change event identity, parentage, timestamps, kind, scope category, +semantic category, attributes, semantic input and output meaning, or schemas. + +Registries run in priority order. Lower priorities run first, and each +callback receives the fields returned by the callback before it. Invalid +binding callback results fail open and preserve the current fields. Node.js +records these failures for `getLastCallbackError()`. ## Registration Lifetimes -Event sanitizers use the same ownership model as other middleware: +Where you register a sanitizer determines how long it stays active. -- Global registrations remain active until explicitly deregistered. -- Scope-local registrations are inherited by descendants and are removed when - the owning scope closes. -- Plugin-context registrations are qualified to the component instance and - roll back if activation fails or the plugin configuration is cleared. +- Global registrations stay active until you explicitly deregister them. +- Scope-local registrations are inherited by descendant scopes and are removed + when the owning scope closes. +- Plugin-context registrations belong to the component instance. The runtime + rolls them back if activation fails or you clear the plugin configuration. ## Register a Global Mark Sanitizer @@ -127,21 +133,21 @@ deregister_mark_sanitize_guardrail("safe-marks")?; ## API Names -Use the matching start and end APIs when the policy applies to duration-bearing -scope events. +Use the matching start and end APIs when a policy applies to duration-bearing +scope events. The following APIs register sanitizers at each level. | Level | Python | Node.js | Rust | | --- | --- | --- | --- | -| Global mark | `guardrails.register_mark_sanitize` | `registerMarkSanitizeGuardrail` | `register_mark_sanitize_guardrail` | -| Global start | `guardrails.register_scope_sanitize_start` | `registerScopeSanitizeStartGuardrail` | `register_scope_sanitize_start_guardrail` | -| Global end | `guardrails.register_scope_sanitize_end` | `registerScopeSanitizeEndGuardrail` | `register_scope_sanitize_end_guardrail` | -| Scope-local mark | `scope_local.register_mark_sanitize` | `scopeRegisterMarkSanitizeGuardrail` | `scope_register_mark_sanitize_guardrail` | -| Scope-local start | `scope_local.register_scope_sanitize_start` | `scopeRegisterScopeSanitizeStartGuardrail` | `scope_register_scope_sanitize_start_guardrail` | -| Scope-local end | `scope_local.register_scope_sanitize_end` | `scopeRegisterScopeSanitizeEndGuardrail` | `scope_register_scope_sanitize_end_guardrail` | +| Global Mark | `guardrails.register_mark_sanitize` | `registerMarkSanitizeGuardrail` | `register_mark_sanitize_guardrail` | +| Global Start | `guardrails.register_scope_sanitize_start` | `registerScopeSanitizeStartGuardrail` | `register_scope_sanitize_start_guardrail` | +| Global End | `guardrails.register_scope_sanitize_end` | `registerScopeSanitizeEndGuardrail` | `register_scope_sanitize_end_guardrail` | +| Scope-Local Mark | `scope_local.register_mark_sanitize` | `scopeRegisterMarkSanitizeGuardrail` | `scope_register_mark_sanitize_guardrail` | +| Scope-Local Start | `scope_local.register_scope_sanitize_start` | `scopeRegisterScopeSanitizeStartGuardrail` | `scope_register_scope_sanitize_start_guardrail` | +| Scope-Local End | `scope_local.register_scope_sanitize_end` | `scopeRegisterScopeSanitizeEndGuardrail` | `scope_register_scope_sanitize_end_guardrail` | -Each registration API has a corresponding `deregister` API. Python -scope-local helpers accept a scope handle. Node.js and Rust scope-local helpers -accept the owning scope UUID. +Each registration API has a matching deregistration API. Python scope-local +helpers accept a scope handle. Node.js and Rust scope-local helpers accept the +owning scope UUID. Plugin contexts expose the same three surfaces as `register_mark_sanitize_guardrail`, @@ -151,35 +157,37 @@ convention. ## Dynamic Plugins -Rust native plugins receive the three methods on their typed plugin context. -Python `grpc-v1` workers receive the same methods on `PluginContext`; callbacks -may be synchronous or asynchronous. The worker protocol remains `grpc-v1` and -adds these stable registration surface values: +Rust native plugins receive the three registration methods on their typed +plugin context. Python `grpc-v1` workers receive the same methods on +`PluginContext`, and their callbacks can be synchronous or asynchronous. The +worker protocol remains `grpc-v1` and defines the following stable registration +surface values: - `MARK_SANITIZE_GUARDRAIL = 30` - `SCOPE_SANITIZE_START_GUARDRAIL = 31` - `SCOPE_SANITIZE_END_GUARDRAIL = 32` -Dynamic plugins should declare only the surfaces they install and should use -the plugin context so failed activation rolls back every registration. +Declare only the surfaces that your dynamic plugin installs. Register them +through the plugin context so the runtime rolls back every registration if +activation fails. ## Experimental C and Go Bindings -The source-first C API uses `NemoRelayEventSanitizeCb` and exports global, +The source-first C API uses `NemoRelayEventSanitizeCb`. It provides global, scope-local, and plugin-context registration functions for all three surfaces. -Global names begin with `nemo_relay_register_`; scope-local names begin with -`nemo_relay_scope_register_`. +Global names start with `nemo_relay_register_`, and scope-local names start +with `nemo_relay_scope_register_`. -The Go binding exposes `EventSanitizeFields`, `EventSanitizeFunc`, global +The Go binding provides `EventSanitizeFields`, `EventSanitizeFunc`, global `Register*SanitizeGuardrail` helpers, scope-local `ScopeRegister*SanitizeGuardrail` helpers, and the same methods on -`PluginContext`. The `guardrails` package provides shorter aliases. A returned -`EventSanitizeFields` replaces all three fields, so copy the provided value and -modify only the fields that should change. +`PluginContext`. The `guardrails` package provides shorter aliases. Because a +returned `EventSanitizeFields` replaces all three fields, copy the supplied +value and modify only the fields that should change. ## Related Topics -For related concepts and configuration, refer to these topics: +For related concepts and configuration, refer to the following topics: - [Middleware](/about-nemo-relay/concepts/middleware) - [Events](/about-nemo-relay/concepts/events)