feat(supervisor): isolate and declassify authenticated MCP discovery responses

[prekshivyas]

Problem Statement

OpenShell can authorize MCP requests and resolve provider credential placeholders immediately before an upstream request is sent, but it has no equivalent trust boundary on the response path. JSON-RPC responses and MCP SSE events are currently relayed to the sandbox unchanged.

That leaves no safe primitive for authenticated discovery such as tools/list when the caller must prove all of the following:

1. The real credential is resolved and used only inside the trusted OpenShell boundary.
2. Raw credential-dependent upstream bytes never become sandbox- or agent-visible.
3. Only an explicitly defined, bounded discovery result is released.
4. Discovery does not grant permission to invoke a tool.
5. Malformed, oversized, uncorrelated, stale, or policy-inconsistent responses fail closed.

This is blocking consumers that need to bootstrap an authenticated MCP provider without first giving an agent access to the provider's raw discovery response. One concrete consumer is NVIDIA/NemoClaw: NemoClaw#6901 and the security review on NemoClaw#6904 require authenticated tool discovery without exposing credential-bearing values or trusting unvalidated upstream content.

This is not merely exact-secret redaction. A credential-aware or compromised upstream can encode, transform, or derive a secret into otherwise valid strings, and arbitrary tool descriptions/schemas may contain untrusted instructions. OpenShell needs an explicit isolation and declassification contract.

Proposed Design

Add a provider-bound, protocol-aware MCP discovery boundary owned by the trusted OpenShell runtime. The preferred implementation direction is a supervisor-owned response gate, because the supervisor already owns TLS termination, request policy evaluation, credential injection, and the live MCP session. A host/control-plane operation may initiate the discovery and return the typed result to an authenticated operator, but the protocol parsing and response isolation should remain next to the session in the supervisor.

The design should satisfy these invariants:

1. Provider-bound operation. Discovery is authorized against an existing sandbox, effective policy endpoint, and attached provider. It must not accept an arbitrary URL, hostname, Authorization value, or credential from the caller.

2. Explicit trusted recipient. If a host/control-plane API returns discovered names, it does so only to an authenticated, authorized operator under an explicit declassification mode. Raw upstream bytes are never returned and never enter the sandbox.

3. Protocol-aware correlation. Track the relevant MCP initialization, negotiated protocol version, session, request ID, response ID, pagination state, and SSE events. Do not treat generic HTTP success as valid MCP discovery.

4. Normalize and re-encode. Parse a complete response, validate it against the selected MCP profile, select only allowed fields, apply bounds, and encode a new typed result. Do not release slices of the original response body.

5. Separate discovery from invocation authority. Intersect discovered tools with the effective tools/call policy. A discovery result can describe what exists but must never expand what the sandbox may call.

6. Explicit output modes. At minimum, distinguish:

   • an operator-authorized declassification mode that may return normalized tool names to the trusted caller; and
   • a stronger verification mode that compares against credential-independent, operator-managed or signed catalog metadata and returns only a locally sourced catalog or fixed verdict.

   Tool names selected by an authenticated server are still credential-dependent content. Validation, exact-secret removal, or a names-only projection does not make those names credential-independent.

7. Fail closed and bound work. Reject malformed, unsupported, oversized, compressed-but-unsupported, uncorrelated, policy-stale, or incomplete discovery responses. Bound total bytes, JSON depth, field lengths, tool count, schema size, pages/cursors, SSE event size, and time.

8. Audit safely. Emit OCSF allow/deny/error events and useful dimensions such as provider, endpoint, protocol version, count, validation result, and policy generation without logging credentials, raw response content, descriptions, or schemas.

A policy-declared exact hostname must not be used as the content-trust signal. Today that declaration controls network reachability/SSRF behavior; it does not establish response integrity, MCP authorization scope, or permission to declassify tool definitions.

Current Architecture

At pinned OpenShell main commit a2cd5f8edae6d4587b10f965080f5a9cf0851afe, the current path is:

sandbox request
  -> CONNECT identity / SSRF / L4 policy
  -> TLS termination
  -> MCP JSON-RPC request parsing and method policy
  -> PRE_CREDENTIALS request middleware
  -> supervisor SecretResolver injects the real credential
  -> authenticated upstream MCP server
  -> HTTP framing only
  -> raw JSON / SSE bytes forwarded to sandbox

The request path has a trusted credential boundary. The response path does not yet have a corresponding validation/declassification boundary.

Technical investigation

• crates/openshell-supervisor-network/src/proxy.rs:1181 and :1218 create the L7 evaluation context containing the supervisor-only SecretResolver.
• crates/openshell-supervisor-network/src/proxy.rs:1292 and crates/openshell-supervisor-network/src/l7/relay.rs:245 dispatch protocol: mcp to the JSON-RPC relay after TLS termination.
• crates/openshell-supervisor-network/src/l7/relay.rs:1108 parses and evaluates the sandbox-to-server request.
• crates/openshell-supervisor-network/src/l7/jsonrpc.rs:67 recognizes the SSE receive-stream GET and otherwise buffers only the request body.
• crates/openshell-supervisor-network/src/l7/jsonrpc.rs:89 retains calls/batch/stream flags but does not retain the JSON-RPC IDs needed for robust request/response correlation.
• crates/openshell-supervisor-network/src/l7/relay.rs:1229 and :1272 pass the request into the shared REST relay with the secret resolver.
• crates/openshell-supervisor-network/src/l7/rest.rs:616 and :667 resolve credential placeholders in outbound headers and write the authenticated request upstream.
• crates/openshell-supervisor-network/src/l7/rest.rs:947 enters the generic response relay.
• crates/openshell-supervisor-network/src/l7/rest.rs:2687-2858 parses HTTP framing, then forwards content-length, chunked, and text/event-stream bodies without MCP-aware validation. The SSE path can remain open until EOF and has no discovery-specific idle bound.
• crates/openshell-supervisor-network/src/l7/relay.rs:1267-1278 documents the future response/SSE inspection seam before the generic relay, but no trusted response contract is implemented.
• architecture/sandbox.md:52-73, docs/reference/policy-schema.mdx:289-301, and docs/sandboxes/policies.mdx:623-633 describe the same limitation.

The behavior is also present in release v0.0.85; this is not just an unreleased-main regression.

Affected Components

Component	Likely files
MCP policy/declassification contract	proto/sandbox.proto:173-210
YAML/protobuf round trip	crates/openshell-policy/src/lib.rs:171-204,547-571
Provider profile round trip	crates/openshell-providers/src/profiles.rs:210-243,930-1001
Runtime endpoint configuration	crates/openshell-supervisor-network/src/l7/mod.rs:95-136,203-267
MCP request/response/session parser	crates/openshell-supervisor-network/src/l7/jsonrpc.rs:18-98,198-295
MCP relay and response gate	crates/openshell-supervisor-network/src/l7/relay.rs:1108-1293
HTTP framing and SSE transport	crates/openshell-supervisor-network/src/l7/rest.rs:616-959,2670-2856
Connection context/dispatch	crates/openshell-supervisor-network/src/proxy.rs:1181-1244,1292-1452
Gateway/control plane, if exposed there	proto/openshell.proto and the corresponding gateway/CLI handlers
Documentation	architecture/sandbox.md, architecture/security-policy.md, docs/reference/policy-schema.mdx, docs/sandboxes/policies.mdx
Integration/E2E	e2e/mcp-conformance/, e2e/rust/tests/forward_proxy_jsonrpc_l7.rs

Estimated scope is roughly 10-18 files for a supervisor-local gate and wider if a new gateway/CLI discovery operation or persistent catalog is added. No meaningful SELinux/AppArmor-specific impact is expected for a supervisor-local parser. A new host service, socket, or persisted catalog requires separate LSM and filesystem-label review.

Existing Work and Duplicate Check

No searched issue or PR provides this complete boundary.

• #1938, merged, added MCP request governance. It intentionally preserves response frames and server-to-client messages without policy inspection. It establishes the gap but does not close it.
• #2174 proposes version-aware MCP wire profiles and discusses response/SSE introspection and trusted annotations. It is a likely prerequisite/partial overlap, but it does not define credential-derived response isolation, catalog declassification, cache scoping, or closure of raw sandbox bypass paths.
• #1733 and merged #2027 added HTTP_REQUEST/PRE_CREDENTIALS. Response-body scanning is explicitly outside the v1 middleware scope.
• #2011 covers external request middleware transport/authentication, not a response operation or MCP-specific semantics.
• #1272 proposed generic request/response filters and was closed in favor of Sandbox egress middleware #1733. Generic content scanning alone cannot provide MCP correlation or a safe declassification contract.
• #2169 adds request content guarding, not response isolation.
• #1694 proposes a post-credential request hook, not a response boundary.
• Closed feat: agent-readable policy manifest for tool/endpoint discovery #1386 concerns static policy-manifest discovery and rejected generating MCP tool definitions because policy lacks API semantics.

Bypass Paths the Design Must Close

A safer host-side discovery API does not solve the boundary if a protected sandbox can still obtain equivalent raw content by:

• issuing tools/list directly;
• opening the MCP SSE receive stream;
• selecting an alternate endpoint/path;
• using plaintext or tls: skip;
• reaching the provider through a non-L7 credential relay;
• receiving list-change notifications, server requests, or equivalent tool definitions on a live stream;
• reusing a stale session, catalog, credential revision, or policy generation.

For providers using the protected discovery mode, these paths must be intercepted, denied, or routed through the same response gate.

Identity, Cache, and Freshness Requirements

If discovery results are cached or shared, the cache key and authorization check must include at least:

• provider instance and endpoint/path, not hostname alone;
• credential identity and credential revision;
• sandbox, tenant, and principal scope where authorization differs;
• negotiated MCP protocol profile and session;
• effective tool-call policy generation;
• pagination completeness;
• catalog freshness and revocation state.

A gateway-owned catalog is viable when discovery is deliberately configuration-time and shared, but it broadens the gateway into an outbound MCP client. That would require explicit decisions for gateway egress, TLS/redirect/SSRF behavior, credential use, lifecycle/session ownership, retries/timeouts, and multi-tenant cache isolation. Prefer keeping live protocol handling in the supervisor and returning only a typed result through the control plane.

Required Tests

• Correlate tools/list responses to request IDs; reject missing, duplicate, wrong, null, or stale IDs as appropriate to the selected MCP version.
• Validate success/error responses, version-specific batch rules, initialization/session binding, protocol-version headers, and pagination.
• Bound response bytes, tool count, JSON depth, schemas, cursor length, page count, and repeated/cyclic cursors.
• Strip annotations by default or preserve them only under explicit provider/profile trust policy. MCP annotations are hints, not authorization facts.
• Reject duplicate or invalid tool names and prove the result is intersected with the effective tools/call policy.
• Cover content-length, chunked, split-frame, malformed UTF-8/JSON, unsupported content encoding, and truncated bodies.
• Cover SSE comments, multiline data, split events, oversized events, idle gaps, notifications, server requests, list-changed flows, and no partial release of an invalid event.
• Prove direct tools/list, receive-stream GET, alternate paths, plaintext, tls: skip, and non-L7 credential relay cannot bypass the boundary for a protected provider.
• Use a fake authenticated upstream that observes the resolved Authorization header and attempts exact, encoded, and derived credential reflection. Assert the sandbox receives only the defined normalized output and logs contain no secret.
• Test credential rotation, policy hot reload, stale sessions/catalogs, sandbox isolation, and cache-key separation.
• Keep supported MCP conformance scenarios passing.
• Add OCSF allow/deny/error assertions without raw response content.

Acceptance Criteria

• An authorized caller can request authenticated MCP discovery only for an existing provider-bound endpoint under an effective sandbox policy.
• The real provider credential is resolved and used only inside the trusted OpenShell boundary.
• No raw discovery response bytes or partial SSE events reach the sandbox or caller.
• The implementation correlates and validates the selected MCP version/session/request and emits a newly encoded, typed result.
• The output/declassification mode is explicit, documented, and covered by threat-model tests.
• Discovery never grants invocation authority and is intersected with the effective tools/call policy.
• Protected providers cannot bypass the gate through direct discovery, SSE, alternate routes, or stale state.
• Resource, pagination, and time bounds are enforced and failures are closed.
• Credential rotation and policy changes invalidate relevant state.
• Audit events contain useful metadata but no raw response, schema, description, or secret.
• The policy schema, provider profiles, architecture/security docs, user docs, and MCP conformance/E2E coverage are updated.
• The relationship to Define version-aware MCP wire profiles for Streamable HTTP and JSON-RPC #2174 is resolved explicitly: dependency, combined implementation, or documented separation.

Alternatives Considered

Forward the raw authenticated response to the sandbox

Rejected. This is the current behavior and violates the isolation requirement.

Return only tool names

Insufficient by itself. Tool names are still selected by the authenticated upstream and remain credential-dependent content. Names-only can be a valid explicit declassification mode for a trusted operator, but it is not credential-independent verification.

Remove exact credential strings from the response

Rejected as a security boundary. Exact redaction does not cover encoding, derivation, semantic leakage, or tool poisoning.

Trust every response from a policy-declared hostname

Rejected. Network destination authorization is not a content-integrity or declassification decision.

Use generic response middleware only

Useful as a future complement for content scanning, but insufficient to own MCP versioning, request/response correlation, sessions, pagination, SSE framing, policy intersection, and invocation authorization. External middleware would also receive sensitive credential-derived content and therefore needs a stronger trust/authentication contract than the current pre-credential request hook.

Discover in the gateway and cache globally

Possible, but substantially broadens gateway responsibilities and creates egress, session, freshness, revocation, and multi-tenant isolation concerns. If selected, the catalog must remain provider/principal/policy scoped and raw sandbox discovery must still be closed.

Require an operator-managed or signed catalog

This provides the strongest credential-independent verification mode and should remain an available design. It may not fit dynamic providers, so it can coexist with an explicitly authorized operator-declassification mode.

Risks and Open Questions

1. Which threat model is required: accidental verbatim reflection, compromised upstream, malicious upstream/tool poisoning, or all of them?
2. Which fields may be declassified in each mode: names, descriptions, input schemas, output schemas, annotations, icons, and metadata?
3. Is the first consumer an authenticated host operator, a sandbox process, or both?
4. Should protected discovery be a new policy mode on an MCP endpoint, a provider profile capability, or both?
5. Does implementation depend on the version/session profile work in Define version-aware MCP wire profiles for Streamable HTTP and JSON-RPC #2174?
6. How are SSE list-change notifications and server-initiated messages handled after initial discovery?
7. Is a strict locally sourced/verified catalog required for high-assurance use cases?
8. What are the default byte, count, depth, pagination, and idle-time bounds?
9. How should policy or credential rotation invalidate live sessions and cached catalogs?
10. If a generic response middleware operation is later added, what sensitive-data contract and trust model applies to that middleware?

Agent Investigation

Investigation covered current OpenShell main at a2cd5f8edae6d4587b10f965080f5a9cf0851afe, release v0.0.85, request and response relay code, policy/provider protobuf round trips, middleware RFC/protobufs, architecture and policy documentation, MCP conformance/E2E coverage, and adjacent open/merged/closed issues and PRs. The proposed direction was independently reviewed with particular attention to hostname trust, bypass paths, protocol correlation, cache identity, and gateway responsibility.

Complexity: High
Confidence: Medium

Checklist

• I searched for existing issues and PRs covering this capability.
• I reviewed the current request, credential-injection, HTTP response, JSON-RPC, and SSE paths.
• I identified affected policy, provider, supervisor, gateway/control-plane, documentation, and test surfaces.
• I documented alternatives, security invariants, bypass paths, risks, and acceptance criteria.
• No security vulnerability details or exploit reproduction steps are included; this is a design/feature gap.
• Maintainers choose the response trust model and approve the implementation direction.

The OpenShell contributor triage workflow has applied state:triage-needed because the author does not have repository write permission. The investigation is complete and ready for human review; a maintainer can apply the appropriate area labels and state:review-ready after triage. Only a maintainer should apply state:agent-ready after the contract and threat model are approved.