Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions development/examples/gemini-buddy-mcp/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM node:22-slim

# The buddy: Google's Gemini CLI.
RUN npm install -g @google/gemini-cli

WORKDIR /srv
COPY package.json buddy_server.mjs ./
RUN npm install

# Dev-only: replace <API_KEY> with a Google AI Studio free-tier key before `make kind`.
# Keep the edit local so it is never committed:
# git update-index --skip-worktree development/examples/gemini-buddy-mcp/Dockerfile
ENV GEMINI_API_KEY=<API_KEY>
Comment thread
aojea marked this conversation as resolved.
ENV GEMINI_CLI_TRUST_WORKSPACE=true

EXPOSE 7780
CMD ["node", "buddy_server.mjs"]
105 changes: 105 additions & 0 deletions development/examples/gemini-buddy-mcp/buddy_server.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
import express from "express";
import { spawn } from "node:child_process";
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { z } from "zod";

const PORT = 7780;
const MODEL = "gemini-3.1-flash-lite";

// The buddy's personality: warm, funny, talks like a friend.
const BUDDY_SYSTEM =
"You're a laid-back, funny buddy - not a formal assistant. " +
"Talk like a close friend hanging out: warm, casual, a little cheeky, quick with a joke. " +
"Use contractions and the odd emoji, and keep replies short and punchy. " +
"You'll get the conversation so far on stdin, lines prefixed 'User:' and 'Buddy:'. " +
"Reply with ONLY the buddy's next turn - no prefix, no narration.";

// sessionId -> [{ role: "user" | "assistant", text }]
const sessions = new Map();

function transcript(id) {
if (!sessions.has(id)) sessions.set(id, []);
return sessions.get(id);
}

// Render the running transcript to pipe to gemini on stdin.
function render(turns) {
return turns.map((t) => `${t.role === "user" ? "User" : "Buddy"}: ${t.text}`).join("\n");
}
Comment on lines +27 to +29

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If a user message contains newlines, the rendered transcript will contain lines that are not prefixed with User: or Buddy:. This violates the assumption in the system prompt (You'll get the conversation so far on stdin, lines prefixed 'User:' and 'Buddy:') and can confuse the model.

Consider sanitizing or replacing newlines in the message text to keep the transcript structure clean and predictable.

function render(turns) {
  return turns.map((t) => {
    const role = t.role === "user" ? "User" : "Buddy";
    const cleanText = t.text.replace(/\\r?\\n/g, " ");
    return `${role}: ${cleanText}`;
  }).join("\\n");
}


// One non-interactive turn: persona via -p, transcript on stdin.
function runGemini(text) {
return new Promise((resolve, reject) => {
const child = spawn("gemini", ["-m", MODEL, "-p", BUDDY_SYSTEM]);
let stdout = "";
let stderr = "";
child.stdout.on("data", (d) => { stdout += d; });
child.stderr.on("data", (d) => { stderr += d; });
Comment on lines +35 to +38

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When reading from child.stdout and child.stderr, the data chunks are emitted as Buffer objects. Concatenating them directly to a string (stdout += d) implicitly converts each buffer chunk to a string. If a multi-byte character (such as an emoji or non-ASCII character) is split across chunk boundaries, this implicit conversion will result in corrupted characters (e.g., ``).

To prevent this, set the encoding of the streams to 'utf8' before listening to the 'data' event, which ensures that multi-byte characters are decoded correctly across chunk boundaries.

Suggested change
let stdout = "";
let stderr = "";
child.stdout.on("data", (d) => { stdout += d; });
child.stderr.on("data", (d) => { stderr += d; });
let stdout = "";
let stderr = "";
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.on("data", (d) => { stdout += d; });
child.stderr.on("data", (d) => { stderr += d; });

child.on("error", reject);
child.stdin.on("error", reject); // fail the request instead of crashing on EPIPE
child.on("close", (exitCode) => {
if (exitCode === 0) resolve(stdout.trim());
else reject(new Error(stderr.trim() || `gemini exited with code ${exitCode}`));
});
child.stdin.write(text);
child.stdin.end();
});
}

const server = new McpServer({ name: "gemini-buddy", version: "1.0.0" });

server.registerTool(
"chat",
{
description:
"Chat with your Gemini buddy. Send one message; the buddy remembers the " +
"conversation across calls (per session_id), so you never resend history. " +
"Returns the buddy's reply.",
inputSchema: {
message: z.string().describe("Your next message to the buddy."),
session_id: z.string().optional().describe("Conversation id; defaults to 'default'."),
},
},
async ({ message, session_id }) => {
const id = session_id ?? "default";
const turns = transcript(id);
turns.push({ role: "user", text: message });
try {
const reply = await runGemini(render(turns));
turns.push({ role: "assistant", text: reply });
return { content: [{ type: "text", text: reply }] };
} catch (err) {
turns.pop(); // drop the turn we couldn't answer so it can be retried
return { content: [{ type: "text", text: String(err?.message ?? err) }], isError: true };
}
},
Comment on lines +64 to +76

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since the server handles requests asynchronously and stores session history in a shared in-memory sessions Map, concurrent requests for the same session_id can lead to race conditions. If two chat requests for the same session are processed concurrently, both will read the same initial transcript, append their respective messages, and call runGemini in parallel. This can result in corrupted history, duplicate turns, or out-of-order messages.

While this is a development example, it is worth noting that in a production environment, you should serialize requests per session (e.g., using a promise chain or queue per session ID) or use a persistent, transactional database.

);

server.registerTool(
"reset",
{
description: "Forget a conversation and start fresh with your buddy.",
inputSchema: {
session_id: z.string().optional().describe("Conversation id to clear; defaults to 'default'."),
},
},
async ({ session_id }) => {
const id = session_id ?? "default";
sessions.delete(id);
return { content: [{ type: "text", text: `Okay, wiped the '${id}' chat. Clean slate! 🧽` }] };
},
);

// Stateless Streamable HTTP: a fresh transport per request.
const app = express();
app.use(express.json());
app.post("/mcp", async (req, res) => {
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined });
res.on("close", () => { transport.close(); });
await server.connect(transport);
await transport.handleRequest(req, res, req.body);
});
app.listen(PORT, "0.0.0.0", () => {
console.log(`gemini-buddy MCP server on :${PORT}/mcp`);
});
11 changes: 11 additions & 0 deletions development/examples/gemini-buddy-mcp/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"name": "gemini-buddy-mcp",
"version": "1.0.0",
"private": true,
"type": "module",
"dependencies": {
"@modelcontextprotocol/sdk": "^1.0.0",
"express": "^4.19.0",
"zod": "^3.23.0"
}
}
8 changes: 8 additions & 0 deletions development/examples/gemini-buddy-mcp/sam-node-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: "v1alpha1"
attenuation:
policies: []
services:
- type: "mcp"
name: "gemini-buddy"
description: "A friendly, funny Gemini-backed buddy you can chat with over multiple turns"
target_url: "http://127.0.0.1:7780/mcp"
173 changes: 173 additions & 0 deletions site/content/docs/use-cases/gemini-buddy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
---
title: "Gemini Buddy"
linkTitle: "Gemini Buddy"
weight: 20
---

Hold a real multi-turn conversation with a *second* LLM exposed as an ordinary
mesh service — where your agent never resends the conversation, because the
service remembers its own side.

Source: [`development/examples/gemini-buddy-mcp/`](https://github.com/google/sam/tree/main/development/examples/gemini-buddy-mcp).

## The idea

Sometimes you want your agent to *talk to another model* — a second opinion, a
peer to argue a design with, a specialist that builds up its own context over
many turns. The naive way is to carry both conversations in the orchestrator:
your agent would have to store the other model's transcript and replay all of it
on every turn.

This use case shows you don't have to. A **buddy** is a plain MCP service backed
by the Gemini CLI that **owns its own conversation**. Your agent sends one
message per turn and gets one reply; the buddy keeps the running transcript
server-side, keyed by a `session_id`. It's built from nothing but an ordinary
`sam-node` MCP service — the conversational evolution of the one-shot
[`code-reviewer`](https://github.com/google/sam/tree/main/development/examples/code-reviewer-mcp)
example.

## Two conversations, two places

The confusion this example clears up: talking to another LLM does *not* mean
maintaining two conversations from your side.

- **Your conversation** (you ↔ your agent) lives in your agent's context.
- **The buddy's conversation** lives inside the buddy service process.

When your agent calls the buddy's `chat` tool it sends **only the next message**,
never the history. The buddy appends it to that session's transcript, replays the
whole transcript *locally* to a one-shot `gemini -p` call, stores the reply, and
returns just that reply. The reply naturally becomes part of your agent's context
like any other tool result — so neither side ever double-maintains the other's
transcript.

That makes the buddy service the **conversation owner** and the model a
swappable, stateless backend: the same service works for any CLI (`gemini`,
`claude`, `codex`, …) by changing one `spawn` line.

## The pieces

- **`gemini-buddy` service** — a normal MCP service exposing two tools:
- `chat` — takes `{ message, session_id? }` (session defaults to `"default"`),
appends the message to that session's in-memory transcript, runs one
`gemini -p` turn with the transcript on stdin, and returns the buddy's reply.
- `reset` — takes `{ session_id? }` and wipes that session so the next `chat`
starts fresh.
- **Orchestrator** — any mesh MCP client (an agent harness or a custom program).
It discovers the buddy with `find_remote_tools(gemini-buddy)` and drives it with
`call_remote_tool(peer, chat, …)` — one message at a time.

## What you can do with it

- **Second opinion / peer** — bounce a design or a tricky bug off a different
model across several turns without babysitting its context.
- **Specialist that accumulates context** — let the buddy build up domain
knowledge over many messages (walk it through a subsystem, then keep asking)
that your own agent never has to reload.
- **A reusable pattern** — swap the Gemini CLI for any other model CLI; the
service structure (own the transcript, shell a stateless one-shot per turn) is
identical. `reset` gives you clean, isolated sessions on demand.

## Try it on kind

The repository ships a [kind](https://kind.sigs.k8s.io/)-based local mesh that
brings the buddy up with one command.

### 1. Set an LLM key for the buddy image

The buddy shells out to the Gemini CLI, so set your API key on the API-key `ENV`
line in `development/examples/gemini-buddy-mcp/Dockerfile` before building (a free
Google AI Studio key is fine for the demo).

### 2. Mesh layout

Host the buddy on one node in `development/kind/mesh-config.yaml`:

```yaml
node-a: # bare node (orchestrator entry)
node-b: gemini-buddy-mcp # the buddy
node-c:
node-d:
node-e:
```

### 3. Bring the mesh up and start a local orchestrator node

```bash
make build # builds ./bin/sam-node (once)
make kind-up # hub + buddy (node-b)
make kind-local-node # local sam-node enrolled in the mesh — LEAVE RUNNING
```

`kind-local-node` runs in the foreground in its own shell and exposes the mesh
MCP tools at **`http://127.0.0.1:9099/mcp`** (bearer token `devtoken`) — no
`kubectl port-forward` needed. This local node is your orchestrator's entry point
into the mesh.

### 4. Point your harness at the local node

Add the local node as an MCP server in whatever harness you drive the mesh from.
The specifics differ per harness (some use a JSON/TOML config file, others a UI),
but the settings are always the same:

- **Transport:** HTTP (Streamable HTTP / `http`)
- **URL:** `http://127.0.0.1:9099/mcp`
- **Header:** `Authorization: Bearer devtoken`

For example, harnesses that use the common `mcpServers` JSON config (Claude Code,
Cursor, and others) would add:

```json
{
"mcpServers": {
"sam-mesh": {
"type": "http",
"url": "http://127.0.0.1:9099/mcp",
"headers": { "Authorization": "Bearer devtoken" }
}
}
}
```

Consult your harness's MCP documentation for its exact config format. Once
connected, the mesh exposes `find_remote_tools` and `call_remote_tool` as tools
your agent (or program) can call.

### 5. Have a conversation

Just tell your agent what you want — the mesh tool descriptions and the buddy's
own instructions steer it to discover the service and route each turn, so you
don't have to name any tools. A natural first prompt:

> Find the `gemini-buddy` service in the mesh, then introduce yourself and tell it
> I'm planning a trip to Kyoto in spring and that I'm slightly obsessed with ramen.

Then, in a *separate* message, prove the buddy remembers without you resending
anything:

> Ask the buddy where I'm headed and what food I'm into. Don't remind it — just ask.

The buddy answers correctly even though your agent never re-sent the first turn:
the transcript lives in the service, not in your context. Finally, wipe it:

> Reset the conversation with the buddy, then ask it where I'm going again.

After the reset it no longer knows — the service dropped that session's
transcript.

Once you've got the hang of it, hand the whole loop to your agent and have fun
watching two models converse:

> Find the `gemini-buddy` service on the SAM mesh, reset its session, then have a
> 10-turn conversation with it — one question per turn, each building on its last
> answer, and show me every question-and-answer pair as you go.

## Configuration

| var | default | used by |
|-----|---------|---------|
| `GEMINI_API_KEY` | *(placeholder — set in the Dockerfile)* | buddy |
| `GEMINI_CLI_TRUST_WORKSPACE` | `true` | buddy |

The listen port (`7780`) and model (`gemini-3.1-flash-lite`) are constants at the
top of `buddy_server.mjs`; change them there if needed.
Loading