Skip to content

feat(ai-gemini): native structuredOutputStream for geminiText and geminiTextInteractions #570

Description

@tombeckenham

Background

Both Gemini text adapters currently fall through to fallbackStructuredOutputStream in @tanstack/ai (packages/typescript/ai/src/activities/chat/index.ts:1796-1869) when called with chat({ outputSchema, stream: true }):

  • geminiText()packages/typescript/ai-gemini/src/adapters/text.ts
  • geminiTextInteractions() (experimental) — packages/typescript/ai-gemini/src/experimental/text-interactions/adapter.ts

Neither implements the optional structuredOutputStream adapter method, so the core helper calls the non-streaming structuredOutput(), buffers the entire JSON, then re-emits it as a single synthetic TEXT_MESSAGE_CONTENT chunk followed by a structured-output.complete CUSTOM event.

This is "stream-compatible" but not truly streaming — users see one big chunk arrive at once, not incremental JSON deltas.

For comparison, the OpenAI-base-derived adapters (openai, openrouter chat-completions + responses, grok, groq) all implement structuredOutputStream natively — see packages/typescript/openai-base/src/adapters/{chat-completions-text.ts:231,responses-text.ts:270}.

What to implement

geminiText()

Use client.models.generateContentStream(...) with responseMimeType: 'application/json' and responseSchema (already used by the non-streaming structuredOutput() at text.ts:181). Yield incremental TEXT_MESSAGE_CONTENT chunks as JSON fragments arrive. On stream end, parse the accumulated text and emit the terminal structured-output.complete CUSTOM event with { object, raw }.

geminiTextInteractions() (experimental)

Use client.interactions.create({ stream: true, response_mime_type: 'application/json', response_format: outputSchema }) (current shape already lives in the non-streaming structuredOutput() at adapter.ts:152-154). Reuse the existing translateInteractionEvents SSE translator — content.delta events with delta.type === 'text' carry the incremental JSON. Emit the terminal structured-output.complete CUSTOM event before RUN_FINISHED, matching the existing gemini.interactionId ordering invariant.

Tests to add (packages/typescript/ai-gemini/tests/)

  • text-adapter.test.ts — call chat({ adapter: geminiText(...), outputSchema, stream: true }), assert ≥2 TEXT_MESSAGE_CONTENT chunks (proves incremental streaming, not buffered), and that the terminal structured-output.complete CUSTOM event carries the parsed object.
  • text-interactions-adapter.test.ts — same shape against geminiTextInteractions(); also assert gemini.interactionId is still emitted before RUN_FINISHED when structured streaming.

E2E

The structured-output-stream feature already exists in testing/e2e/src/lib/feature-support.ts. Adding gemini to its support set + a fixture under testing/e2e/fixtures/structured-output-stream/ should be enough — the existing structured-output-stream.spec.ts covers the contentDeltaCount > 1 invariant via the route's onChunk counter.

Out of scope

  • Image / audio / video output modalities through structured streaming — those flow through geminiImage/geminiSpeech and have their own adapter surfaces.
  • Tool-call interleaving inside structured-output streams — the OpenAI-base implementations also don't mix tool calls into a single structured stream; same constraint here.

Context

Surfaced during PR review of #502 (feat(ai-gemini): add geminiTextInteractions() adapter for stateful Interactions API). #502 itself ships the non-streaming structuredOutput() only; this issue tracks the streaming variant for both Gemini adapters as a follow-up.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions