-
Notifications
You must be signed in to change notification settings - Fork 12
Add Gemini buddy example #190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
4f884e6
99c1862
f209d3b
f539d14
d689631
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| FROM node:22-slim | ||
|
|
||
| # The buddy: Google's Gemini CLI. | ||
| RUN npm install -g @google/gemini-cli | ||
|
|
||
| WORKDIR /srv | ||
| COPY package.json buddy_server.mjs ./ | ||
| RUN npm install | ||
|
|
||
| # Dev-only: replace <API_KEY> with a Google AI Studio free-tier key before `make kind`. | ||
| # Keep the edit local so it is never committed: | ||
| # git update-index --skip-worktree development/examples/gemini-buddy-mcp/Dockerfile | ||
| ENV GEMINI_API_KEY=<API_KEY> | ||
| ENV GEMINI_CLI_TRUST_WORKSPACE=true | ||
|
|
||
| EXPOSE 7780 | ||
| CMD ["node", "buddy_server.mjs"] | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,105 @@ | ||||||||||||||||||||||
| import express from "express"; | ||||||||||||||||||||||
| import { spawn } from "node:child_process"; | ||||||||||||||||||||||
| import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; | ||||||||||||||||||||||
| import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js"; | ||||||||||||||||||||||
| import { z } from "zod"; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| const PORT = 7780; | ||||||||||||||||||||||
| const MODEL = "gemini-3.1-flash-lite"; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| // The buddy's personality: warm, funny, talks like a friend. | ||||||||||||||||||||||
| const BUDDY_SYSTEM = | ||||||||||||||||||||||
| "You're a laid-back, funny buddy - not a formal assistant. " + | ||||||||||||||||||||||
| "Talk like a close friend hanging out: warm, casual, a little cheeky, quick with a joke. " + | ||||||||||||||||||||||
| "Use contractions and the odd emoji, and keep replies short and punchy. " + | ||||||||||||||||||||||
| "You'll get the conversation so far on stdin, lines prefixed 'User:' and 'Buddy:'. " + | ||||||||||||||||||||||
| "Reply with ONLY the buddy's next turn - no prefix, no narration."; | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| // sessionId -> [{ role: "user" | "assistant", text }] | ||||||||||||||||||||||
| const sessions = new Map(); | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| function transcript(id) { | ||||||||||||||||||||||
| if (!sessions.has(id)) sessions.set(id, []); | ||||||||||||||||||||||
| return sessions.get(id); | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| // Render the running transcript to pipe to gemini on stdin. | ||||||||||||||||||||||
| function render(turns) { | ||||||||||||||||||||||
| return turns.map((t) => `${t.role === "user" ? "User" : "Buddy"}: ${t.text}`).join("\n"); | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
Comment on lines
+27
to
+29
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If a user message contains newlines, the rendered transcript will contain lines that are not prefixed with Consider sanitizing or replacing newlines in the message text to keep the transcript structure clean and predictable. function render(turns) {
return turns.map((t) => {
const role = t.role === "user" ? "User" : "Buddy";
const cleanText = t.text.replace(/\\r?\\n/g, " ");
return `${role}: ${cleanText}`;
}).join("\\n");
} |
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| // One non-interactive turn: persona via -p, transcript on stdin. | ||||||||||||||||||||||
| function runGemini(text) { | ||||||||||||||||||||||
| return new Promise((resolve, reject) => { | ||||||||||||||||||||||
| const child = spawn("gemini", ["-m", MODEL, "-p", BUDDY_SYSTEM]); | ||||||||||||||||||||||
| let stdout = ""; | ||||||||||||||||||||||
| let stderr = ""; | ||||||||||||||||||||||
| child.stdout.on("data", (d) => { stdout += d; }); | ||||||||||||||||||||||
| child.stderr.on("data", (d) => { stderr += d; }); | ||||||||||||||||||||||
|
Comment on lines
+35
to
+38
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When reading from To prevent this, set the encoding of the streams to
Suggested change
|
||||||||||||||||||||||
| child.on("error", reject); | ||||||||||||||||||||||
| child.stdin.on("error", reject); // fail the request instead of crashing on EPIPE | ||||||||||||||||||||||
| child.on("close", (exitCode) => { | ||||||||||||||||||||||
| if (exitCode === 0) resolve(stdout.trim()); | ||||||||||||||||||||||
| else reject(new Error(stderr.trim() || `gemini exited with code ${exitCode}`)); | ||||||||||||||||||||||
| }); | ||||||||||||||||||||||
| child.stdin.write(text); | ||||||||||||||||||||||
| child.stdin.end(); | ||||||||||||||||||||||
| }); | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| const server = new McpServer({ name: "gemini-buddy", version: "1.0.0" }); | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| server.registerTool( | ||||||||||||||||||||||
| "chat", | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| description: | ||||||||||||||||||||||
| "Chat with your Gemini buddy. Send one message; the buddy remembers the " + | ||||||||||||||||||||||
| "conversation across calls (per session_id), so you never resend history. " + | ||||||||||||||||||||||
| "Returns the buddy's reply.", | ||||||||||||||||||||||
| inputSchema: { | ||||||||||||||||||||||
| message: z.string().describe("Your next message to the buddy."), | ||||||||||||||||||||||
| session_id: z.string().optional().describe("Conversation id; defaults to 'default'."), | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| async ({ message, session_id }) => { | ||||||||||||||||||||||
| const id = session_id ?? "default"; | ||||||||||||||||||||||
| const turns = transcript(id); | ||||||||||||||||||||||
| turns.push({ role: "user", text: message }); | ||||||||||||||||||||||
| try { | ||||||||||||||||||||||
| const reply = await runGemini(render(turns)); | ||||||||||||||||||||||
| turns.push({ role: "assistant", text: reply }); | ||||||||||||||||||||||
| return { content: [{ type: "text", text: reply }] }; | ||||||||||||||||||||||
| } catch (err) { | ||||||||||||||||||||||
| turns.pop(); // drop the turn we couldn't answer so it can be retried | ||||||||||||||||||||||
| return { content: [{ type: "text", text: String(err?.message ?? err) }], isError: true }; | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
|
Comment on lines
+64
to
+76
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since the server handles requests asynchronously and stores session history in a shared in-memory While this is a development example, it is worth noting that in a production environment, you should serialize requests per session (e.g., using a promise chain or queue per session ID) or use a persistent, transactional database. |
||||||||||||||||||||||
| ); | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| server.registerTool( | ||||||||||||||||||||||
| "reset", | ||||||||||||||||||||||
| { | ||||||||||||||||||||||
| description: "Forget a conversation and start fresh with your buddy.", | ||||||||||||||||||||||
| inputSchema: { | ||||||||||||||||||||||
| session_id: z.string().optional().describe("Conversation id to clear; defaults to 'default'."), | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| async ({ session_id }) => { | ||||||||||||||||||||||
| const id = session_id ?? "default"; | ||||||||||||||||||||||
| sessions.delete(id); | ||||||||||||||||||||||
| return { content: [{ type: "text", text: `Okay, wiped the '${id}' chat. Clean slate! 🧽` }] }; | ||||||||||||||||||||||
| }, | ||||||||||||||||||||||
| ); | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| // Stateless Streamable HTTP: a fresh transport per request. | ||||||||||||||||||||||
| const app = express(); | ||||||||||||||||||||||
| app.use(express.json()); | ||||||||||||||||||||||
| app.post("/mcp", async (req, res) => { | ||||||||||||||||||||||
| const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined }); | ||||||||||||||||||||||
| res.on("close", () => { transport.close(); }); | ||||||||||||||||||||||
| await server.connect(transport); | ||||||||||||||||||||||
| await transport.handleRequest(req, res, req.body); | ||||||||||||||||||||||
| }); | ||||||||||||||||||||||
| app.listen(PORT, "0.0.0.0", () => { | ||||||||||||||||||||||
| console.log(`gemini-buddy MCP server on :${PORT}/mcp`); | ||||||||||||||||||||||
| }); | ||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "name": "gemini-buddy-mcp", | ||
| "version": "1.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "dependencies": { | ||
| "@modelcontextprotocol/sdk": "^1.0.0", | ||
| "express": "^4.19.0", | ||
| "zod": "^3.23.0" | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| version: "v1alpha1" | ||
| attenuation: | ||
| policies: [] | ||
| services: | ||
| - type: "mcp" | ||
| name: "gemini-buddy" | ||
| description: "A friendly, funny Gemini-backed buddy you can chat with over multiple turns" | ||
| target_url: "http://127.0.0.1:7780/mcp" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| --- | ||
| title: "Gemini Buddy" | ||
| linkTitle: "Gemini Buddy" | ||
| weight: 20 | ||
| --- | ||
|
|
||
| Hold a real multi-turn conversation with a *second* LLM exposed as an ordinary | ||
| mesh service — where your agent never resends the conversation, because the | ||
| service remembers its own side. | ||
|
|
||
| Source: [`development/examples/gemini-buddy-mcp/`](https://github.com/google/sam/tree/main/development/examples/gemini-buddy-mcp). | ||
|
|
||
| ## The idea | ||
|
|
||
| Sometimes you want your agent to *talk to another model* — a second opinion, a | ||
| peer to argue a design with, a specialist that builds up its own context over | ||
| many turns. The naive way is to carry both conversations in the orchestrator: | ||
| your agent would have to store the other model's transcript and replay all of it | ||
| on every turn. | ||
|
|
||
| This use case shows you don't have to. A **buddy** is a plain MCP service backed | ||
| by the Gemini CLI that **owns its own conversation**. Your agent sends one | ||
| message per turn and gets one reply; the buddy keeps the running transcript | ||
| server-side, keyed by a `session_id`. It's built from nothing but an ordinary | ||
| `sam-node` MCP service — the conversational evolution of the one-shot | ||
| [`code-reviewer`](https://github.com/google/sam/tree/main/development/examples/code-reviewer-mcp) | ||
| example. | ||
|
|
||
| ## Two conversations, two places | ||
|
|
||
| The confusion this example clears up: talking to another LLM does *not* mean | ||
| maintaining two conversations from your side. | ||
|
|
||
| - **Your conversation** (you ↔ your agent) lives in your agent's context. | ||
| - **The buddy's conversation** lives inside the buddy service process. | ||
|
|
||
| When your agent calls the buddy's `chat` tool it sends **only the next message**, | ||
| never the history. The buddy appends it to that session's transcript, replays the | ||
| whole transcript *locally* to a one-shot `gemini -p` call, stores the reply, and | ||
| returns just that reply. The reply naturally becomes part of your agent's context | ||
| like any other tool result — so neither side ever double-maintains the other's | ||
| transcript. | ||
|
|
||
| That makes the buddy service the **conversation owner** and the model a | ||
| swappable, stateless backend: the same service works for any CLI (`gemini`, | ||
| `claude`, `codex`, …) by changing one `spawn` line. | ||
|
|
||
| ## The pieces | ||
|
|
||
| - **`gemini-buddy` service** — a normal MCP service exposing two tools: | ||
| - `chat` — takes `{ message, session_id? }` (session defaults to `"default"`), | ||
| appends the message to that session's in-memory transcript, runs one | ||
| `gemini -p` turn with the transcript on stdin, and returns the buddy's reply. | ||
| - `reset` — takes `{ session_id? }` and wipes that session so the next `chat` | ||
| starts fresh. | ||
| - **Orchestrator** — any mesh MCP client (an agent harness or a custom program). | ||
| It discovers the buddy with `find_remote_tools(gemini-buddy)` and drives it with | ||
| `call_remote_tool(peer, chat, …)` — one message at a time. | ||
|
|
||
| ## What you can do with it | ||
|
|
||
| - **Second opinion / peer** — bounce a design or a tricky bug off a different | ||
| model across several turns without babysitting its context. | ||
| - **Specialist that accumulates context** — let the buddy build up domain | ||
| knowledge over many messages (walk it through a subsystem, then keep asking) | ||
| that your own agent never has to reload. | ||
| - **A reusable pattern** — swap the Gemini CLI for any other model CLI; the | ||
| service structure (own the transcript, shell a stateless one-shot per turn) is | ||
| identical. `reset` gives you clean, isolated sessions on demand. | ||
|
|
||
| ## Try it on kind | ||
|
|
||
| The repository ships a [kind](https://kind.sigs.k8s.io/)-based local mesh that | ||
| brings the buddy up with one command. | ||
|
|
||
| ### 1. Set an LLM key for the buddy image | ||
|
|
||
| The buddy shells out to the Gemini CLI, so set your API key on the API-key `ENV` | ||
| line in `development/examples/gemini-buddy-mcp/Dockerfile` before building (a free | ||
| Google AI Studio key is fine for the demo). | ||
|
|
||
| ### 2. Mesh layout | ||
|
|
||
| Host the buddy on one node in `development/kind/mesh-config.yaml`: | ||
|
|
||
| ```yaml | ||
| node-a: # bare node (orchestrator entry) | ||
| node-b: gemini-buddy-mcp # the buddy | ||
| node-c: | ||
| node-d: | ||
| node-e: | ||
| ``` | ||
|
|
||
| ### 3. Bring the mesh up and start a local orchestrator node | ||
|
|
||
| ```bash | ||
| make build # builds ./bin/sam-node (once) | ||
| make kind-up # hub + buddy (node-b) | ||
| make kind-local-node # local sam-node enrolled in the mesh — LEAVE RUNNING | ||
| ``` | ||
|
|
||
| `kind-local-node` runs in the foreground in its own shell and exposes the mesh | ||
| MCP tools at **`http://127.0.0.1:9099/mcp`** (bearer token `devtoken`) — no | ||
| `kubectl port-forward` needed. This local node is your orchestrator's entry point | ||
| into the mesh. | ||
|
|
||
| ### 4. Point your harness at the local node | ||
|
|
||
| Add the local node as an MCP server in whatever harness you drive the mesh from. | ||
| The specifics differ per harness (some use a JSON/TOML config file, others a UI), | ||
| but the settings are always the same: | ||
|
|
||
| - **Transport:** HTTP (Streamable HTTP / `http`) | ||
| - **URL:** `http://127.0.0.1:9099/mcp` | ||
| - **Header:** `Authorization: Bearer devtoken` | ||
|
|
||
| For example, harnesses that use the common `mcpServers` JSON config (Claude Code, | ||
| Cursor, and others) would add: | ||
|
|
||
| ```json | ||
| { | ||
| "mcpServers": { | ||
| "sam-mesh": { | ||
| "type": "http", | ||
| "url": "http://127.0.0.1:9099/mcp", | ||
| "headers": { "Authorization": "Bearer devtoken" } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Consult your harness's MCP documentation for its exact config format. Once | ||
| connected, the mesh exposes `find_remote_tools` and `call_remote_tool` as tools | ||
| your agent (or program) can call. | ||
|
|
||
| ### 5. Have a conversation | ||
|
|
||
| Just tell your agent what you want — the mesh tool descriptions and the buddy's | ||
| own instructions steer it to discover the service and route each turn, so you | ||
| don't have to name any tools. A natural first prompt: | ||
|
|
||
| > Find the `gemini-buddy` service in the mesh, then introduce yourself and tell it | ||
| > I'm planning a trip to Kyoto in spring and that I'm slightly obsessed with ramen. | ||
|
|
||
| Then, in a *separate* message, prove the buddy remembers without you resending | ||
| anything: | ||
|
|
||
| > Ask the buddy where I'm headed and what food I'm into. Don't remind it — just ask. | ||
|
|
||
| The buddy answers correctly even though your agent never re-sent the first turn: | ||
| the transcript lives in the service, not in your context. Finally, wipe it: | ||
|
|
||
| > Reset the conversation with the buddy, then ask it where I'm going again. | ||
|
|
||
| After the reset it no longer knows — the service dropped that session's | ||
| transcript. | ||
|
|
||
| Once you've got the hang of it, hand the whole loop to your agent and have fun | ||
| watching two models converse: | ||
|
|
||
| > Find the `gemini-buddy` service on the SAM mesh, reset its session, then have a | ||
| > 10-turn conversation with it — one question per turn, each building on its last | ||
| > answer, and show me every question-and-answer pair as you go. | ||
|
|
||
| ## Configuration | ||
|
|
||
| | var | default | used by | | ||
| |-----|---------|---------| | ||
| | `GEMINI_API_KEY` | *(placeholder — set in the Dockerfile)* | buddy | | ||
| | `GEMINI_CLI_TRUST_WORKSPACE` | `true` | buddy | | ||
|
|
||
| The listen port (`7780`) and model (`gemini-3.1-flash-lite`) are constants at the | ||
| top of `buddy_server.mjs`; change them there if needed. |
Uh oh!
There was an error while loading. Please reload this page.