AuthBridge forward-proxy buffers entire response body, breaking streaming MCP tool calls

[kellyaa]

Summary

The AuthBridge forward-proxy fully buffers outbound response bodies via io.ReadAll whenever any plugin in the outbound pipeline declares ReadsBody or WritesBody. For MCP servers using the Streamable HTTP transport, tools/call responses can be returned as Content-Type: text/event-stream (chunked-transfer body with SSE-framed JSON-RPC messages) and stay open for the full duration of slow tool execution. Buffering converts that streaming response into a blocking Client.Do read, which is then bounded by the hard-coded 30s http.Client.Timeout and produces 502 bad gateway (or appears to the agent as a hang followed by retry).

This affects every realistic outbound pipeline because mcp-parser, inference-parser, and ibac all declare ReadsBody: true.

Reproduction

Deploy any agent + MCP server pair where MCP tool calls take longer than ~30s of upstream work, with an outbound pipeline containing mcp-parser (and/or ibac/inference-parser). Observed with the exgentic tau2 benchmark, where tool calls invoke a user-simulator LLM:

./deploy-and-evaluate.sh --benchmark tau2 --agent tool_calling \
  --plugin-preset ibac-only --model openai/azure/gpt-5.5

AuthBridge logs show the request being admitted by the pipeline and then the body read failing:

17:51:21.467  mcp-parser: request   method=tools/call isAction=true
17:51:35.619  forward-proxy: response body read error
              host=exgentic-mcp-tau2-mcp:8000
              error="context deadline exceeded ... while reading body"
17:51:56.416  forward-proxy: response body read error  context canceled

The agent sees the tool call as never-returning and stalls.

Root cause

authlib/listener/forwardproxy/server.go:

• Lines 277–291 — when OutboundPipeline.NeedsBody() is true, the entire response body is read with io.ReadAll(io.LimitReader(resp.Body, maxBodySize+1)) before any plugin runs and before any byte is forwarded downstream:

  if s.OutboundPipeline.NeedsBody() && resp.Body != nil {
      respBody, err := io.ReadAll(io.LimitReader(resp.Body, maxBodySize+1))
      ...
      pctx.ResponseBody = respBody
      resp.Body = io.NopCloser(bytes.NewReader(respBody))
  }

• Line 89 — the outbound HTTP client has Timeout: 30 * time.Second, which Go applies to the entire request lifecycle including body read. With buffering on, this caps the lifetime of any streaming MCP tools/call to 30s regardless of whether it's actually making progress.

• Line 25 — maxBodySize = 1 << 20 (1 MiB). Even if the timeout were raised, large streaming responses (long tau2 simulator turns, large MCP results) would still hit 502 response body too large.

• Line 344 — the downstream io.Copy(w, resp.Body) is a no-op stream because resp.Body is already a fully-buffered bytes.Reader by then.

Plugins declaring ReadsBody: true (all on the outbound path):

• authlib/plugins/mcpparser/plugin.go:91
• authlib/plugins/ibac/plugin.go:301
• authlib/plugins/inferenceparser/plugin.go:30

Why a bigger timeout / bigger buffer is not the right fix

Raising http.Client.Timeout and maxBodySize masks the symptom for short tool calls but:

• Buffered tool-call responses block the agent until the upstream call fully completes, defeating the streaming-HTTP transport the agent and MCP server already negotiate.
• IBAC's verdict on tools/call is decided at request time (before the upstream call goes out); response-side body inspection in IBAC is auxiliary (audit/event recording), so there is no semantic reason to gate the downstream wire on a fully-buffered response.
• mcp-parser's response-side parsing operates per JSON-RPC message, which composes naturally with frame-by-frame streaming.

Proposed fix

Special-case streaming responses in the forward-proxy response handler:

• Detect Content-Type: text/event-stream (or any chunked, non-application/json response) and switch from the buffered branch to a streaming pass-through.
• For each completed JSON-RPC message in the SSE-framed body, run the existing response-phase plugin inspectors (mcp-parser, ibac, inference-parser) over that message, then immediately write the frame to the downstream ResponseWriter and Flush().
• Keep the existing buffered path for application/json one-shot responses — they're small, plugins want the full envelope, and latency cost is negligible.
• Once the streaming path exists, the 30s http.Client.Timeout becomes a connect/header ceiling only (which is fine; tau2 servers return headers immediately), and maxBodySize no longer applies to the wire response, only to whatever slice a plugin actually inspects.

The Streamable HTTP transport is already in use end-to-end:

• Agent: mcp.client.streamable_http.streamable_http_client (in the exgentic agent's mcp_wrapper.py).
• MCP server: FastMCP's streamable_http_app().

So no agent or MCP-server changes are needed — only the proxy needs to stop converting streaming responses into buffered ones.

IBAC compatibility

The proposed fix does not affect IBAC's decision-making. IBAC's body inspection is request-side only:

• describeAction() at authlib/plugins/ibac/plugin.go:598 builds the judge prompt from pctx.Method/Scheme/Host/Path, pctx.Body (request body excerpt), pctx.Extensions.MCP (parsed MCP method/tool/args from mcp-parser), and pctx.Extensions.Inference (when JudgeInference=true). All four are populated during the request phase, before forwarding upstream.
• OnRequest() at plugin.go:333 is the only place the judge is called and the only place IBAC can emit a Reject. The verdict is final before the upstream MCP server starts producing a response.
• OnResponse() at plugin.go:564 is a literal no-op (return pipeline.Action{Type: pipeline.Continue}) — IBAC does not inspect the response body today.

IBAC's ReadsBody: true capability covers its use of pctx.Body (the request body). The fact that the same flag also triggers response-side io.ReadAll is a coincidence of the current proxy implementation, not something IBAC requires. Request-side buffering remains unchanged under the proposed fix.

If a future IBAC feature wanted to inspect tool-call results, the streaming path would compose better than the current buffered one — each JSON-RPC result arrives as a discrete frame instead of being demultiplexed from a buffered SSE blob.

Severity

High for any deployment running ibac or mcp-parser against MCP servers whose tool calls do meaningful upstream work. Currently makes the tau2 benchmark unusable through the AuthBridge sidecar.

[huang195]

Traced the code end-to-end — confirming the bug and the streaming direction, with a few refinements. Scope note up front: this plans the fix without mid-stream response denial. Response-phase plugins here are observability-only today (IBAC decides at request time, OnResponse is a no-op), so the goal is "stream gracefully without breaking observability," and per-message response enforcement is a clean follow-up the design leaves room for.

1. Diagnosis confirmed

forwardproxy/server.go buffers via io.ReadAll(LimitReader(resp.Body, maxBodySize+1)) when NeedsBody(), runs RunResponse once over the buffer, then io.Copys an already-buffered reader. With Client.Timeout=30s covering the body read and maxBodySize=1<<20, a slow/large tools/call SSE response 502s. ✔

2. Streaming is the right fix (not bigger buffers)

Raising the timeout/cap risks OOM on large results and still blocks the transport. Streaming bounds memory to one frame and removes the dependence on a total deadline. Enforcement is unaffected: IBAC gates the call at request time, before any response streams.

3. The timeout must be changed explicitly

http.Client.Timeout covers reading the body, so it does not become a "header-only ceiling" just by switching to io.Copy. Replace it with Transport.ResponseHeaderTimeout (time-to-headers) + a read/idle deadline to kill hung streams. Without this, streaming still dies at 30s.

4. Branch on Content-Type, per response

Per the Streamable HTTP spec the server chooses application/json vs text/event-stream per response (client Accepts both), so the same tool may return JSON on one call and SSE on the next. Decide at header time off Content-Type — reliable, no heuristics. Useful framing: a response is always a stream of ≥1 complete JSON-RPC messages; application/json is just the one-message case.

5. The plugins split two ways — this is the real work

Plugin	Response handling	Stream-native?
ibac	OnResponse no-op (gates at request)	✔
mcp-parser	each tools/call result is one complete frame	✔
inference-parser	concatenates token deltas; usage only in the final chunk	✖ aggregates
a2a-parser	assembles artifact + terminal status across message/stream events	✖ aggregates

So 2 of 4 aggregate. Running their current OnResponse per-frame yields fragmented/overwritten output and N noisy events. They need to fold each frame into running state and finalize at end-of-stream — a real (if contained) rework, not a drop-in.

6. Interface: a frame hook as an optional interface — no core-interface growth

The codebase already uses optional, type-asserted interfaces (Configurable/Initializer/Readier/Shutdowner/Finisher). Add streaming the same way; existing plugins are untouched. Model the response uniformly as frames:

// optional
type StreamingResponder interface {
    OnResponseFrame(ctx context.Context, pctx *Context, frame []byte, last bool) Action
}

• mcp-parser implements only this (per-message). inference-parser/a2a-parser fold each frame and finalize when last.
• OnRequest stays — MCP requests are one-shot JSON and that hook feeds IBAC; it's not the streaming problem.
• application/json is delivered as a single last=true frame, so stream-native plugins need one response code path, not two. (A later, larger cleanup could slim the core Plugin interface so plugins implement only the hooks their protocol needs — out of scope here.)

7. Scope for this PR

In: stream text/event-stream frame-by-frame (http.Flusher, no Content-Length); header-only timeout; frame-native plugin handling (mcp-parser per-message, inference/a2a fold+finalize); keep the buffered path for application/json.

Out (for now): mid-stream response denial. Since nothing rejects on the response, the proxy can forward+flush each frame and feed plugins for recording — no inspect-before-forward gating needed. The OnResponseFrame hook leaves the door open to add per-message enforcement later without rework.

8. Also applies inbound

The same buffering exists in reverseproxy.modifyResponse for A2A message/stream → a2a-parser, so the streaming path/contract should be shared across both listeners. WritesBody/SetResponseBody is incompatible with streaming (no plugin does it today — guard it).

[kellyaa]

Implementation plan

Picking up @huang195's refinements (thanks — these change the shape of the work). Posting the plan before I start so the design calls are reviewable.

Scope

In: stream text/event-stream responses frame-by-frame in both forward and reverse proxies; replace the outbound http.Client.Timeout with a header-only ceiling + idle deadline; add an optional StreamingResponder plugin interface; convert the aggregating plugins to fold-and-finalize.

Out: mid-stream response enforcement (no plugin rejects on the response today — the new hook leaves the door open for it later).

Steps

1. Branch & worktree

   • Worktree off main of kagenti-extensions, branch fix/477-streaming-mcp-responses. DCO + signed commits (-s -S).

2. Outbound HTTP client (authlib/listener/forwardproxy/server.go:65-95)

   • Drop Client.Timeout (covers body read; incompatible with streaming).
   • Add Transport.ResponseHeaderTimeout (~30s — preserves the existing time-to-headers guard).
   • On the streaming read path, apply a per-read idle deadline so a wedged upstream is killed even when bytes are flowing slowly.

3. Optional plugin interface in authlib/pipeline/:

   type StreamingResponder interface {
       OnResponseFrame(ctx context.Context, pctx *Context, frame []byte, last bool) Action
   }

   Type-asserted opt-in (same pattern as Configurable/Initializer/Readier/Shutdowner/Finisher). No core-interface churn; non-streaming plugins are untouched. Plumb a RunResponseFrame helper through pipeline.Holder.

4. Response handler — branch on Content-Type per response

   • text/event-stream: streaming path. Read SSE frames; for each complete JSON-RPC message call OnResponseFrame(..., last=false) then write+Flush downstream. On EOF, call once more with last=true (empty frame) so aggregating plugins can finalize. RunResponse is NOT invoked after the streaming path completes — aggregating plugins move their finalization logic into OnResponseFrame(last=true). (Decision: avoids confusing legacy plugins that don't migrate; clean contract.)
   • application/json: existing buffered path preserved as today. Additionally, for plugins implementing StreamingResponder, deliver the single buffered body as one last=true frame so streaming-aware plugins use a single code path (the "stream of ≥1 messages" framing).
   • WritesBody=true guard: if any plugin in the outbound pipeline declares WritesBody, refuse the streaming path (semantically incompatible — we can't rewrite a body we've already started forwarding) and fall back to the existing buffered path with a warning log. No current plugin sets WritesBody; this is the guard @huang195 called out.

5. maxBodySize reshape

   • Streaming path: per-frame cap of 1 MiB (matches today's total cap; keeps a memory ceiling per JSON-RPC message). No total cap on the stream — that's the point of streaming.
   • Buffered path: unchanged (1 MiB total).

6. Convert aggregating plugins

   • inference-parser: implement OnResponseFrame that decodes each data: chunk's choices[].delta, accumulates content, and collects usage from the final chunk. On last=true, populate pctx.Extensions.Inference exactly as today's OnResponse would. Existing OnResponse becomes a no-op when OnResponseFrame ran. Because the JSON path delivers a single last=true frame, the same code works for both shapes.
   • a2a-parser: implement OnResponseFrame that parses each message/stream event and folds artifact + terminal status into running state; finalize on last=true.
   • mcp-parser: implement OnResponseFrame for per-message recording (one JSON-RPC result == one frame). Drop the buffered OnResponse body parse in favor of the framed one.
   • ibac: unchanged — OnResponse already a no-op.

7. Mirror in authlib/listener/reverseproxy/

   • Inbound A2A message/stream responses use the same buffering today (reverseproxy.modifyResponse). Apply the same Content-Type branch and reuse the OnResponseFrame plumbing + SSE frame reader. The streaming contract is shared across forward and reverse proxies.

8. Tests

   • Unit: SSE frame reader (split/multi-frame chunks, comment lines, missing trailing newline, frame > 1 MiB rejected).
   • Unit: each updated plugin's OnResponseFrame (per-message for mcp-parser; fold-and-finalize for inference-parser / a2a-parser).
   • Integration in forwardproxy/server_test.go: mock upstream that emits SSE over >30s with idle gaps — verify downstream sees frames as they arrive, plugins receive each frame with correct last flag, no 30s timeout fires, and WritesBody=true in pipeline forces the buffered fallback.
   • Mirror integration tests in reverseproxy/.
   • go test ./... under authlib/.

9. PR

   • Against main, references AuthBridge forward-proxy buffers entire response body, breaking streaming MCP tool calls #477. DCO sign-off; Assisted-By trailer.

Design calls flagged for review

• Don't call RunResponse after streaming completes — aggregating plugins finalize via OnResponseFrame(last=true). Cleaner contract; legacy plugins that don't migrate aren't called on streaming responses.
• Per-frame maxBodySize=1 MiB on the streaming path; no total cap — bounds memory per message while not defeating streaming.
• WritesBody=true forces buffered fallback rather than failing the request — preserves correctness; logs visibility.

Will follow up with the PR.