diff --git a/.changeset/devtools-memory-inspector.md b/.changeset/devtools-memory-inspector.md new file mode 100644 index 000000000..5fd1e33aa --- /dev/null +++ b/.changeset/devtools-memory-inspector.md @@ -0,0 +1,29 @@ +--- +'@tanstack/ai-memory': minor +'@tanstack/ai-event-client': minor +'@tanstack/ai-client': minor +'@tanstack/ai-devtools-core': minor +--- + +**Surface server-side memory state in the TanStack AI DevTools.** + +The DevTools panel now has a **Memory** tab for any chat wired with +`memoryMiddleware`. It shows, per scope (session), an operations timeline (each +turn's recall — query, fragment count, injected system-prompt size, whether +memory tools were exposed, duration) and the current stored records/facts when +the adapter implements the optional `inspect`/`listFacts` methods. + +Because memory runs on the server (whose event bus never reaches the browser), +the middleware transports its state to the panel over the chat stream as a +`memory:state` `CUSTOM` event, which `@tanstack/ai-client`'s devtools bridge +re-emits as browser `memory:*` events — the same pattern generation results use. +The snapshot reflects memory as of the start of each turn; opening the panel +mid-conversation replays the latest state so the tab isn't empty. + +- `@tanstack/ai-memory` — `memoryMiddleware` injects a `memory:state` `CUSTOM` + chunk carrying recall metrics + an `inspect`/`listFacts` snapshot; exports + `MEMORY_STATE_EVENT` and `MemoryStateEventValue`. +- `@tanstack/ai-event-client` — adds the `memory:snapshot` devtools event. +- `@tanstack/ai-client` — the chat devtools bridge re-emits `memory:*` from the + transported chunk and replays the last snapshot on `devtools:request-state`. +- `@tanstack/ai-devtools-core` — new Memory tab + per-scope memory store slice. diff --git a/.changeset/memory-middleware.md b/.changeset/memory-middleware.md new file mode 100644 index 000000000..6e9cd9f31 --- /dev/null +++ b/.changeset/memory-middleware.md @@ -0,0 +1,44 @@ +--- +'@tanstack/ai': minor +'@tanstack/ai-event-client': minor +'@tanstack/ai-memory': minor +--- + +**Add server-side memory via a `recall`/`save` adapter contract in `@tanstack/ai-memory`.** + +Memory is now a single, provider-agnostic contract with two verbs — `recall` and +`save` — which is the shape every memory backend (in-process, Redis, and hosted +vendors) naturally exposes. `memoryMiddleware` recalls relevant memory into the +system prompt (and optionally injects vendor tools) before the model runs, then +defers `save` of the finished turn via `ctx.defer` so streaming is never blocked. +Extraction, ranking, and rendering live inside each adapter — the middleware is thin. + +`@tanstack/ai-memory` (new package) — everything ships here: + +- Root: `memoryMiddleware`, the `MemoryAdapter` contract + (`recall` / `save` / optional `inspect` / `listFacts`), and the `MemoryScope` / + `MemoryTurn` / `RecallResult` / `SaveReceipt` types. +- `@tanstack/ai-memory/in-memory` → `inMemory()` — zero-dependency adapter for dev, + tests, and single-process demos. Pass an `embedder` for semantic scoring and/or an + `extract` function to persist derived facts. +- `@tanstack/ai-memory/redis` → `redis({ redis, prefix? })` — production adapter for + plain Redis. `ioredis` wires in directly; `redis` (node-redis v4+) via the + `fromNodeRedis(client)` wrapper. Both are optional peer dependencies. +- `@tanstack/ai-memory/hindsight` → `hindsight()`, `@tanstack/ai-memory/mem0` → + `mem0()`, `@tanstack/ai-memory/honcho` → `honcho()` — hosted-vendor adapters. Their + SDKs (`@vectorize-io/hindsight-client`, `@honcho-ai/sdk`) are optional peers loaded + lazily; mem0 talks to its server over plain HTTP (no SDK). Vendors can expose LLM + tools through `recall` (e.g. hindsight's retain/recall/reflect). +- A shared `recall`/`save` contract-test suite (`@tanstack/ai-memory/tests/contract`) + that any adapter — including third-party ones — can run. + +`@tanstack/ai`: + +- **Removes the (unreleased) `@tanstack/ai/memory` subpath.** The middleware, + contract, and helpers all moved to `@tanstack/ai-memory`. + +`@tanstack/ai-event-client`: + +- The five `memory:*` devtools events (`memory:retrieve:started` / `:completed`, + `memory:persist:started` / `:completed`, `memory:error`) now carry recall/save + payloads (adapter id, fragment/receipt counts, `phase: 'recall' | 'save'`). diff --git a/docs/config.json b/docs/config.json index 592302cbc..de57b5fb6 100644 --- a/docs/config.json +++ b/docs/config.json @@ -27,7 +27,8 @@ { "label": "Devtools", "to": "getting-started/devtools", - "addedAt": "2026-04-15" + "addedAt": "2026-04-15", + "updatedAt": "2026-07-22" }, { "label": "Quick Start: Vue", @@ -460,6 +461,40 @@ } ] }, + { + "label": "Memory", + "children": [ + { + "label": "Overview", + "to": "memory/overview", + "addedAt": "2026-07-21", + "updatedAt": "2026-07-22" + }, + { + "label": "Quickstart", + "to": "memory/quickstart", + "addedAt": "2026-07-21", + "updatedAt": "2026-07-22" + }, + { + "label": "Adapters", + "to": "memory/adapters", + "addedAt": "2026-07-21", + "updatedAt": "2026-07-22" + }, + { + "label": "Custom Adapter", + "to": "memory/custom-adapter", + "addedAt": "2026-07-21", + "updatedAt": "2026-07-22" + }, + { + "label": "Operating", + "to": "memory/operating", + "addedAt": "2026-07-22" + } + ] + }, { "label": "Advanced", "children": [ diff --git a/docs/getting-started/devtools.md b/docs/getting-started/devtools.md index 259363da6..6569c087b 100644 --- a/docs/getting-started/devtools.md +++ b/docs/getting-started/devtools.md @@ -22,6 +22,7 @@ TanStack Devtools is a unified devtools panel for inspecting and debugging TanSt - **Tool Call Inspection** - Inspect input and output of tool calls. - **Tool Fixture Replay** - Build tool payloads from a tool's standard-schema input, append the result into chat messages, and save fixtures in localStorage for repeated UI iteration. - **State Visualization** - Visualize chat state and message history. +- **Memory Inspector** - For chats wired with `memoryMiddleware`, see what memory recalled and injected each turn plus the current stored records and facts. - **Error Tracking** - Monitor errors and exceptions in AI interactions. ## Hook Dashboard @@ -74,6 +75,15 @@ When a `useChat` hook receives tools, the devtools panel lists those tools and t Applying a tool fixture appends the tool call and result into the real chat messages for that hook. Saved fixtures are stored in browser localStorage under the AI devtools namespace so they are available the next time you open the panel. +## Memory Inspector + +When a chat is wired with [`memoryMiddleware`](../memory/overview.md), the hook's **Memory** tab shows what the server-side memory backend did for that conversation, grouped by scope (session): + +- **Operations timeline** - Each turn's recall: the query, how many fragments came back, how many characters were injected into the system prompt, whether memory-provided tools were exposed, and the recall duration. +- **Stored records & facts** - The current contents of the memory store for the scope, when the adapter implements the optional `inspect`/`listFacts` methods (the built-in `inMemory()` and `redis()` adapters do). Adapters without introspection still show the operations timeline. + +Because memory runs on the server, its state is transported to the panel over the chat stream (a `CUSTOM` event the client re-emits) rather than a separate channel — the same way generation results reach the panel. The snapshot reflects memory as of the start of each turn, so a turn's own writes appear in the next turn's snapshot. Opening the panel after a turn replays the latest memory state, so the tab is populated even when you open devtools mid-conversation. + ## Event Sources Client-visible state is emitted by the headless client. Server-only details, such as middleware and provider stream events that never exist on the client, are emitted from the server counterpart. Events include a source descriptor and stable envelope id so the panel can link related events and avoid displaying duplicates. diff --git a/docs/memory/adapters.md b/docs/memory/adapters.md new file mode 100644 index 000000000..97f6695aa --- /dev/null +++ b/docs/memory/adapters.md @@ -0,0 +1,200 @@ +--- +title: Adapters +id: memory-adapters +order: 3 +description: "Every built-in and vendor memory adapter in @tanstack/ai-memory, with all of their options and an example of each: inMemory, redis, hindsight, mem0, honcho." +keywords: + - tanstack ai + - memory + - adapters + - inMemory + - redis + - hindsight + - mem0 + - honcho + - options +--- + +Every adapter implements the same `recall`/`save` contract, so they're interchangeable +in `memoryMiddleware`. This page is the full option reference: each adapter's options with +an example of each. + +- [Common options](#common-options), shared by `inMemory()` and `redis()` +- Adapters: [`inMemory()`](#inmemory), [`redis()`](#redis), [`hindsight()`](#hindsight), [`mem0()`](#mem0), [`honcho()`](#honcho) + +## Common options + +`inMemory()` and `redis()` are both client-side rankers built on the same pipeline, so +they share these options. + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `topK` | `number` | `6` | Max hits returned by `recall`. | +| `minScore` | `number` | `0.15` | Drop hits scoring below this. | +| `kinds` | `Array` | all | Restrict recall to these record kinds (`'message'`, `'summary'`, `'fact'`, `'preference'`). | +| `embedder` | `{ embed(text): Promise }` | none | Enable semantic scoring (embeds on both `recall` and `save`). | +| `extract` | `(turn, scope) => ExtractedFact[]` | none | Persist derived facts on `save`, alongside the raw turn. | +| `render` | `(hits) => string` | built-in | Replace the prompt renderer. | + +Every option, in one adapter: + +```ts +import { inMemory } from '@tanstack/ai-memory/in-memory' + +// `embedText` stands in for your embedding client (OpenAI, Cohere, a local model). +declare function embedText(text: string): Promise> + +const memory = inMemory({ + topK: 8, // return up to 8 hits + minScore: 0.2, // ignore weak matches + kinds: ['message', 'fact', 'preference'], // skip summaries + embedder: { embed: embedText }, // semantic + lexical scoring + extract: (turn) => [ + // store a derived fact in addition to the raw turn + { text: `User said: ${turn.user}`, kind: 'fact', importance: 0.8 }, + ], + render: (hits) => + // custom prompt block instead of the default renderer + `What I remember:\n${hits.map((h) => `- ${h.record.text}`).join('\n')}`, +}) +``` + +**`extract`** returns `ExtractedFact[]` (`{ text, kind?, importance?, metadata? }`). Return +`undefined` for a no-op. It's where an LLM-based fact extractor plugs in without the +adapter taking a hard dependency on any model. + +**`embedder`** is invoked on the recall path (to embed the query) and again on save (to +embed stored text). Without it, scoring is lexical + recency only. + +## `inMemory()` + +Zero-dependency, `Map`-backed. Takes only the [common options](#common-options) above. +Records vanish on restart, so use it for dev, tests, and single-process demos. + +```ts +import { inMemory } from '@tanstack/ai-memory/in-memory' + +const memory = inMemory() // all options are optional +``` + +## `redis()` + +Plain-Redis adapter. Adds two options to the [common options](#common-options), and +requires a client. + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `redis` | `RedisLike` | (required) | Your Redis client (`ioredis`, or node-redis via `fromNodeRedis`). | +| `prefix` | `string` | `'tanstack-ai:memory'` | Key namespace. | + +```ts +import Redis from 'ioredis' +import { redis } from '@tanstack/ai-memory/redis' + +const memory = redis({ + redis: new Redis(process.env.REDIS_URL ?? 'redis://localhost:6379'), // required + prefix: 'myapp:memory', // key namespace + topK: 8, // common options apply here too + minScore: 0.2, +}) +``` + +Using **node-redis** (`redis` package) instead of `ioredis`? Its camelCase API doesn't +match `RedisLike`, so wrap it with `fromNodeRedis`: + +```ts +import { createClient } from 'redis' +import { redis, fromNodeRedis } from '@tanstack/ai-memory/redis' + +const client = createClient({ url: process.env.REDIS_URL }) +await client.connect() + +const memory = redis({ redis: fromNodeRedis(client) }) +``` + +`ioredis` and `redis` are both optional peer dependencies. Install whichever you use. + +## `hindsight()` + +Hosted adapter backed by Hindsight. Owns extraction/ranking server-side and exposes +`retain`/`recall`/`reflect` LLM tools through `recall`. `@vectorize-io/hindsight-client` +is an optional peer, loaded lazily. + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `user` | `string` | `scope.userId` | Durable user id used in the bank key (`{user}__{sessionId}`). | +| `baseUrl` | `string` | `HINDSIGHT_URL` / `http://localhost:8888` | Server URL. | +| `budget` | `'low' \| 'mid' \| 'high'` | `'mid'` | Recall budget. | +| `onToolRetain` | `(receipt) => void` | none | Fired when the model calls `hindsight_retain`. | +| `onToolRecall` | `(query, result) => void` | none | Fired when the model calls `hindsight_recall`. | + +```ts +import { hindsight } from '@tanstack/ai-memory/hindsight' + +const memory = hindsight({ + user: 'alice', // bank = alice__{sessionId} + baseUrl: 'https://hindsight.internal', // default: HINDSIGHT_URL + budget: 'high', // deeper recall + onToolRetain: (receipt) => console.log('model retained', receipt.ok), + onToolRecall: (query, result) => + console.log('model recalled', query, result.fragments?.length), +}) +``` + +## `mem0()` + +Hosted adapter backed by a mem0 server, over plain HTTP (no SDK peer). Requires a running +mem0 server. + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `user` | `string` | `scope.userId` / `'demo-user'` | mem0 `user_id`. | +| `baseUrl` | `string` | `MEM0_URL` / `http://localhost:8000` | Server URL. | +| `apiKey` | `string` | `MEM0_ADMIN_API_KEY` | Bearer token. | +| `rerank` | `boolean` | `true` | Ask mem0 to rerank search results. | +| `threshold` | `number` | `0.1` | Minimum search score. | + +```ts +import { mem0 } from '@tanstack/ai-memory/mem0' + +const memory = mem0({ + user: 'alice', // mem0 user_id + baseUrl: 'https://mem0.internal', // default: MEM0_URL + apiKey: process.env.MEM0_ADMIN_API_KEY, // bearer token + rerank: true, // rerank results + threshold: 0.2, // stricter score floor +}) +``` + +## `honcho()` + +Hosted adapter backed by Honcho. `recall` returns a synthesized dialectic answer over the +user's representation (no discrete fragments). `@honcho-ai/sdk` is an optional peer, +loaded lazily. + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `user` | `string` | `scope.userId` / `'demo-user'` | User peer id. | +| `baseURL` | `string` | `HONCHO_URL` / `http://localhost:8001` | Server URL. | +| `workspaceId` | `string` | `HONCHO_APP_NAME` / `'ai-memory'` | Workspace id. | +| `apiKey` | `string` | `HONCHO_API_KEY` / `'dev-no-auth'` | API key. | +| `assistantId` | `string` | `'assistant'` | Assistant peer id. | + +```ts +import { honcho } from '@tanstack/ai-memory/honcho' + +const memory = honcho({ + user: 'alice', // user peer + baseURL: 'https://honcho.internal', // default: HONCHO_URL + workspaceId: 'my-app', // default: HONCHO_APP_NAME + apiKey: process.env.HONCHO_API_KEY, // default: 'dev-no-auth' + assistantId: 'support-bot', // default: 'assistant' +}) +``` + +## Where to go next + +- [Overview](./overview): the `recall`/`save` contract and how a turn flows +- [Quickstart](./quickstart): wire an adapter into a real `chat()` call +- [Operating memory](./operating): options, telemetry, devtools events, and failures +- [Custom Adapter](./custom-adapter): implement `recall`/`save` for a backend that isn't shipped diff --git a/docs/memory/custom-adapter.md b/docs/memory/custom-adapter.md new file mode 100644 index 000000000..d11bc142f --- /dev/null +++ b/docs/memory/custom-adapter.md @@ -0,0 +1,192 @@ +--- +title: Custom Adapter +id: memory-custom-adapter +order: 4 +description: "Write a recall/save MemoryAdapter for a backend that isn't shipped, such as pgvector, MongoDB, DynamoDB, or a hosted memory service. Two methods, one shared contract test." +keywords: + - tanstack ai + - memory + - custom adapter + - MemoryAdapter + - recall + - save + - pgvector + - contract suite +--- + +You have a backend in mind (pgvector, MongoDB, DynamoDB, a hosted memory API) and the +built-in `inMemory()` / `redis()` adapters don't fit. A memory adapter is just an object +with two methods, `recall` and `save`, so this is a short guide. + +> **First time looking at memory?** Start with the [Overview](./overview) for what the +> contract is and how the middleware uses it. + +## The contract + +```ts +// The MemoryAdapter contract, from `@tanstack/ai-memory`: +import type { MemoryAdapter } from '@tanstack/ai-memory' +``` + +```ts +// The shape of the contract, shown for reference. +import type { + MemoryFact, + MemoryScope, + MemorySnapshot, + MemoryTurn, + RecallResult, + SaveReceipt, +} from '@tanstack/ai-memory' + +interface MemoryAdapter { + id: string + recall(scope: MemoryScope, query: string): Promise + save(scope: MemoryScope, turn: MemoryTurn): Promise> + inspect?(scope: MemoryScope): Promise // optional (devtools) + listFacts?(scope: MemoryScope): Promise> // optional (devtools) +} +``` + +Two rules the middleware relies on: + +1. **`recall` decides relevance.** Return a rendered `systemPrompt` (empty string when + there's nothing), plus optional `fragments`, `tools`, and `toolGuidance`. Ranking + strategy is entirely yours: lexical, vector, hybrid, or vendor-native. +2. **`save` owns extraction.** Turn the `{ user, assistant }` turn into whatever you + persist. Return one `SaveReceipt` per underlying write. + +Scope isolation is your responsibility: a `recall` for one `scope` must never surface +another scope's data. + +## Step 1: Scaffold + +```ts +import type { + MemoryAdapter, + MemoryScope, + MemoryTurn, + RecallResult, + SaveReceipt, +} from '@tanstack/ai-memory' + +// node-postgres' Pool, minimally. In your project, use the real type instead: +// import type { Pool } from 'pg' +type Pool = { + query: ( + text: string, + values: Array, + ) => Promise<{ rows: Array<{ text: string }> }> +} + +// Your embedding client. Swap in OpenAI, Cohere, a local model, etc. +type Embed = (text: string) => Promise> + +export function pgvectorMemory(options: { pool: Pool; embed: Embed }): MemoryAdapter { + const { pool, embed } = options + return { + id: 'pgvector', + + async save(scope: MemoryScope, turn: MemoryTurn): Promise> { + const rows = [ + { role: 'user', text: turn.user }, + { role: 'assistant', text: turn.assistant }, + ] + for (const row of rows) { + const vector = await embed(row.text) + await pool.query( + `INSERT INTO memory (session_id, user_id, role, text, embedding) + VALUES ($1, $2, $3, $4, $5)`, + [scope.sessionId, scope.userId ?? null, row.role, row.text, JSON.stringify(vector)], + ) + } + return [{ ok: true }] + }, + + async recall(scope: MemoryScope, query: string): Promise { + const q = await embed(query) + const { rows } = await pool.query( + `SELECT text, 1 - (embedding <=> $1::vector) AS score + FROM memory + WHERE session_id = $2 AND ($3::text IS NULL OR user_id = $3) + ORDER BY score DESC + LIMIT 6`, + [JSON.stringify(q), scope.sessionId, scope.userId ?? null], + ) + const fragments = rows.map((r) => ({ text: r.text, source: 'pgvector' })) + const systemPrompt = fragments.length + ? `Relevant memory:\n${fragments.map((f) => `- ${f.text}`).join('\n')}` + : '' + return { systemPrompt, fragments } + }, + } +} +``` + +The shape generalizes: every method takes a `scope`, does its backend-specific work, +and keeps scopes isolated. For a backend without native search, load the scope's +records and rank them yourself. + +## Step 2: Run the contract suite + +`@tanstack/ai-memory/tests/contract` exports `runMemoryAdapterContract`. Point it at a +factory that returns a fresh adapter. It verifies the save then recall round-trip, scope +isolation, empty recall, receipt shape, and the optional introspection methods. + +```ts ignore +// ignore: imports the `../src/pgvector` module you wrote in Step 1. +// tests/pgvector.test.ts +import { runMemoryAdapterContract } from '@tanstack/ai-memory/tests/contract' +import { pgvectorMemory } from '../src/pgvector' + +runMemoryAdapterContract('pgvectorMemory', async () => { + const pool = makeCleanPool() // truncate between tests for a fresh adapter + return pgvectorMemory({ pool, embed }) +}) +``` + +## Step 3: Wire it into `memoryMiddleware` + +Once the suite is green, the adapter is interchangeable with the built-ins: + +```ts ignore +// ignore: imports the `./pgvector` module you wrote in Step 1, and assumes +// `messages` / `scope` from your app. +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { pgvectorMemory } from './pgvector' + +const memory = pgvectorMemory({ pool, embed }) + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + middleware: [memoryMiddleware({ adapter: memory, scope })], +}) +``` + +The middleware never inspects the adapter's internals. `recall`/`save` is the entire +interface. + +## Exposing tools (optional) + +`recall` can return `tools` and `toolGuidance` to give the model direct control over +memory (this is how the `hindsight()` adapter exposes retain/recall/reflect tools). The +middleware merges them into the run's tools and injects the guidance ahead of the +recalled prompt. Return `tools: []` (or omit it) when your adapter exposes none. + +## Pitfalls + +- **Keep scopes isolated.** If you serialize scope into a composite key, escape your + delimiter so a `sessionId`/`userId` containing it can't collide with another scope. +- **`recall` must not throw for an empty scope.** Return `{ systemPrompt: '' }`. +- **Extraction lives in `save`.** Don't expect the middleware to derive facts. The raw + turn is handed to you; store or summarize it however you like. + +## Where to go next + +- [Overview](./overview): the `recall`/`save` contract, scope, and how a turn flows +- [Adapters](./adapters): the built-in and vendor adapters, with every option +- [Quickstart](./quickstart): wire `memoryMiddleware` into a real `chat()` call +- [Operating memory](./operating): options, telemetry, devtools events, and failures diff --git a/docs/memory/operating.md b/docs/memory/operating.md new file mode 100644 index 000000000..718a4301b --- /dev/null +++ b/docs/memory/operating.md @@ -0,0 +1,100 @@ +--- +title: Operating +id: memory-operating +order: 5 +description: "Run memoryMiddleware in production: configure its options, add onRecall/onSave telemetry, watch recall and save in devtools, and rely on non-fatal failure handling that never breaks a chat run." +keywords: + - tanstack ai + - memory + - middleware options + - telemetry + - devtools + - observability + - save-only +--- + +Memory is wired into your `chat()` call. Now you want to see whether it actually recalls +anything, get that activity into your own logs, and know that a slow or broken store +won't take down the chat. This page covers the middleware's options and how to observe +and operate it. + +New to memory? Start with the [Overview](./overview) and [Quickstart](./quickstart) first. + +## `memoryMiddleware` options + +| Option | Type | Default | Purpose | +|--------|------|---------|---------| +| `adapter` | `MemoryAdapter` | (required) | The backend to `recall` from and `save` to. | +| `scope` | `MemoryScope \| (ctx) => MemoryScope` | (required) | Isolation scope, static or derived per request. | +| `role` | `'recall+save' \| 'save-only'` | `'recall+save'` | `'save-only'` persists turns without recalling or injecting. | +| `onRecall` | `({ scope, query, result }) => void` | none | App telemetry after each `recall`. | +| `onSave` | `({ scope, turn, receipts }) => void` | none | App telemetry after each deferred `save`. | + +Every option in one place: + +```ts +import { memoryMiddleware } from '@tanstack/ai-memory' +import { inMemory } from '@tanstack/ai-memory/in-memory' + +const mw = memoryMiddleware({ + adapter: inMemory(), + // Function form derives scope per request. `ctx.threadId` is the stable + // per-conversation id; add `userId` from your server-validated session. + scope: (ctx) => ({ sessionId: ctx.threadId }), + role: 'recall+save', // or 'save-only' to persist without injecting + onRecall: ({ query, result }) => { + console.log('recalled', result.fragments?.length ?? 0, 'hits for', query) + }, + onSave: ({ receipts }) => { + console.log('saved', receipts.filter((r) => r.ok).length, 'records') + }, +}) +``` + +## Persist without recalling + +By default the middleware both recalls and saves (`role: 'recall+save'`). Set +`role: 'save-only'` to persist each turn without reading memory back or injecting anything +into the prompt. Use it to build up a user's history before you turn recall on, or on +routes where you want to record turns but not shape the current answer. + +## Telemetry with onRecall and onSave + +The devtools events below are for watching memory during development. For telemetry that +ships with your app, use the `onRecall` and `onSave` callbacks shown above. `onRecall` +fires after each recall with the query and result, so you can count hits. `onSave` fires +after each deferred save with the write receipts, so you can count writes. Send those to +your metrics client instead of `console.log`. + +## Watch it in devtools + +The AI DevTools has a **Memory** tab for any chat wired with `memoryMiddleware`. It +shows each turn's recall (query, fragment count, characters injected, recall duration) +and, for adapters that implement `inspect`/`listFacts` (the built-in `inMemory()` and +`redis()` do), the current stored records and facts. See the +[Memory Inspector](../getting-started/devtools#memory-inspector) for what it renders and +how server-side memory reaches the panel. + +Under the hood the middleware emits these events on `aiEventClient` (from +`@tanstack/ai-event-client`). The panel reads them, and you can subscribe directly: + +| Event | When | +|-------|------| +| `memory:retrieve:started` | Recall begins | +| `memory:retrieve:completed` | Recall returns (fragment count, whether tools were injected) | +| `memory:persist:started` | A deferred save begins | +| `memory:persist:completed` | A save completes (receipt count) | +| `memory:error` | A `recall` or `save` threw (`phase: 'recall'` or `'save'`) | + +## Failures are non-fatal + +A memory failure never breaks a chat run. A throwing `recall` or `save` emits +`memory:error` and the run continues with degraded memory: recall returns nothing, and a +failed save is dropped. Streaming is never blocked, and a failed save never fails the +turn. This means a flaky store degrades the experience instead of taking down the chat. + +## Where to go next + +- [Overview](./overview): the `recall`/`save` contract and how a turn flows +- [Adapters](./adapters): every adapter's options, with an example of each +- [Custom Adapter](./custom-adapter): implement `recall`/`save` for a backend that isn't shipped diff --git a/docs/memory/overview.md b/docs/memory/overview.md new file mode 100644 index 000000000..5f254a22f --- /dev/null +++ b/docs/memory/overview.md @@ -0,0 +1,128 @@ +--- +title: Overview +id: memory-overview +order: 1 +description: "Give a TanStack AI chat() call memory across turns and sessions. memoryMiddleware recalls relevant memory into the prompt before the model runs, then saves each finished turn through a pluggable adapter." +keywords: + - tanstack ai + - memory + - long-term memory + - retrieval + - persistence + - middleware + - rag + - personalization +--- + +Your assistant forgets everything the moment a session ends. A user tells it their name +this week; next week it asks again. `memoryMiddleware` fixes that. It gives a `chat()` +run memory that survives across turns and across sessions. + +It works in two moves. Before the model runs, it **recalls** relevant memory from a +pluggable adapter and adds it to the system prompt. After the run finishes, it **saves** +the turn. The save is deferred, so it never blocks streaming. + +Reach for it when you need recall across turns or sessions. To keep the last few messages +of the same request, just pass them in `messages`. Memory is overkill for that. + +Everything lives in `@tanstack/ai-memory`: the middleware, the adapter contract, and the +built-in and vendor adapters. + +> Want a copy-paste setup? See the [Quickstart](./quickstart). Building an adapter for a +> backend that isn't shipped? See the [Custom Adapter](./custom-adapter) guide. + +## When to reach for it + +| Need | Use this | +|------|----------| +| "Remember what the user told me last week" | Memory middleware with a persistent adapter | +| "Each user has their own context" | Memory middleware with a scoped adapter | +| "Use a hosted memory service (mem0, Honcho, Hindsight)" | The matching vendor adapter | +| Keep the last few turns in the same request | Pass them in `messages`, skip memory | + +## The contract: recall and save + +A memory adapter has one identifier and two verbs. Extraction, ranking, rendering, and +storage are all the adapter's job. The middleware never looks inside a record. + +| Member | Purpose | +|--------|---------| +| `id` | Stable identifier used in logs and devtools. | +| `recall(scope, query)` | Return what's relevant to `query` within `scope`: a rendered `systemPrompt`, optional `fragments`, and optional LLM `tools` plus `toolGuidance`. | +| `save(scope, turn)` | Persist a finished `{ user, assistant }` turn. Extraction happens here. Returns one `SaveReceipt` per write. | +| `inspect(scope)?` | Optional. A full snapshot for a devtools panel. | +| `listFacts(scope)?` | Optional. A flat fact list for a devtools panel. | + +```ts +// The MemoryAdapter contract, from `@tanstack/ai-memory`: +import type { MemoryAdapter } from '@tanstack/ai-memory' +``` + +Built-in adapters, each a tree-shakeable subpath: + +```ts +import { inMemory } from '@tanstack/ai-memory/in-memory' +import { redis } from '@tanstack/ai-memory/redis' +``` + +Vendor adapters: + +```ts +import { hindsight } from '@tanstack/ai-memory/hindsight' +import { mem0 } from '@tanstack/ai-memory/mem0' +import { honcho } from '@tanstack/ai-memory/honcho' +``` + +See [Adapters](./adapters) for every adapter and its options. + +## How a turn flows + +1. **Recall** runs before the model, during the run's `init` phase. + `adapter.recall(scope, userText)` returns memory, and the middleware adds the + `systemPrompt`, `toolGuidance`, and any `tools` to the run. +2. **Save** runs after the stream finishes, deferred through `ctx.defer` so it never + blocks the response. The middleware hands the `{ user, assistant }` turn to + `adapter.save(scope, turn)`. + +To add telemetry, watch memory in devtools, persist without recalling, or handle +failures, see [Operating memory](./operating). + +## Scope and security + +`MemoryScope` is the isolation boundary. It is session-centric, with an optional durable +user id: + +```ts +// The MemoryScope type, from `@tanstack/ai-memory`: +type MemoryScope = { + sessionId: string + userId?: string +} +``` + +Always derive scope on the server from trusted state. Accepting `userId` from the request +body is how one user reads another user's memory. The function form of `scope` runs per +request and only sees what your server attached to the chat context: + +```ts +import { memoryMiddleware } from '@tanstack/ai-memory' +import type { MemoryAdapter } from '@tanstack/ai-memory' + +declare const adapter: MemoryAdapter +declare function getSession(ctx: unknown): { threadId: string; userId: string } + +memoryMiddleware({ + adapter, + scope: (ctx) => { + const session = getSession(ctx) // your server-validated session + return { sessionId: session.threadId, userId: session.userId } + }, +}) +``` + +## Next steps + +- [Quickstart](./quickstart): wire `memoryMiddleware` into a real `chat()` call +- [Adapters](./adapters): every adapter's options, with an example of each +- [Custom Adapter](./custom-adapter): implement `recall`/`save` for a backend that isn't shipped +- [Operating memory](./operating): options, telemetry, devtools events, and failure behavior diff --git a/docs/memory/quickstart.md b/docs/memory/quickstart.md new file mode 100644 index 000000000..20e4c52e8 --- /dev/null +++ b/docs/memory/quickstart.md @@ -0,0 +1,159 @@ +--- +title: Quickstart +id: memory-quickstart +order: 2 +description: "Add cross-session memory to a TanStack AI chat() call: install the package, pick a recall/save adapter, wire memoryMiddleware, and derive scope server-side." +keywords: + - tanstack ai + - memory + - quickstart + - in-memory adapter + - redis adapter + - chat middleware +--- + +You have a working `chat()` call and you want it to remember context across turns or +sessions. By the end of this guide, `memoryMiddleware` recalls relevant memory into the +prompt and saves each finished turn through a real adapter, scoped safely from your +server-validated session. + +> Want the full contract first? See the [Overview](./overview). + +## Step 1: Install the package + +```bash +pnpm add @tanstack/ai-memory +``` + +`@tanstack/ai-memory` ships `memoryMiddleware`, the `MemoryAdapter` contract, and the +built-in and vendor adapters (each on its own subpath). + +## Step 2: Pick an adapter + +> **In-memory:** `inMemory()` is zero-dependency and stores records in a `Map`. Use it +> for local development, tests, and single-process demos. Records vanish on restart. +> +> **Redis:** `redis({ redis })` persists across restarts and shares state across +> processes. Bring your own client (`ioredis`, or `redis` via `fromNodeRedis`). +> +> **Vendors:** `hindsight()`, `mem0()`, and `honcho()` delegate to a hosted memory service. + +Custom adapters implement the `recall`/`save` contract. See [Custom Adapter](./custom-adapter). + +## Step 3: Wire `memoryMiddleware` into `chat()` + +Start with the in-memory adapter, the fastest path to a working setup: + +```ts +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { inMemory } from '@tanstack/ai-memory/in-memory' + +const memory = inMemory() + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages: [{ role: 'user', content: 'Hello' }], + middleware: [ + memoryMiddleware({ + adapter: memory, + scope: { sessionId: 'demo-thread', userId: 'alice' }, + }), + ], +}) +``` + +Each turn, the middleware recalls relevant memory into the system prompt (lexical scoring +by default), then deferred-saves the user and assistant turn after the stream finishes. + +When you're ready to ship, swap the adapter and keep everything else the same: + +```ts +import Redis from 'ioredis' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { redis } from '@tanstack/ai-memory/redis' +import type { MemoryScope } from '@tanstack/ai-memory' + +declare const scope: MemoryScope // from Step 5 + +const client = new Redis(process.env.REDIS_URL ?? 'redis://localhost:6379') +const memory = redis({ redis: client }) + +memoryMiddleware({ adapter: memory, scope }) +``` + +> Using a hosted service? Swap `inMemory()` for `hindsight({ user })`, `mem0({ user })`, +> or `honcho({ user })`. The middleware wiring is identical. The adapter maps +> `recall`/`save` onto the vendor API. + +## Step 4: Semantic scoring (optional) + +The built-in adapters score lexically by default. Pass an `embedder` for semantic recall +when scopes grow large or queries don't share keywords with stored text: + +```ts +import OpenAI from 'openai' +import { inMemory } from '@tanstack/ai-memory/in-memory' + +const openai = new OpenAI() + +const memory = inMemory({ + embedder: { + async embed(text) { + const result = await openai.embeddings.create({ + model: 'text-embedding-3-small', + input: text, + }) + const embedding = result.data[0]?.embedding + if (!embedding) throw new Error('embedding request returned no vector') + return embedding + }, + }, +}) +``` + +## Step 5: Derive scope server-side + +`scope` is the isolation boundary. Static scopes are fine for fixtures, but in any real +app derive scope per request from server-validated session data, never from the request +body. + +```ts +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { memoryMiddleware } from '@tanstack/ai-memory' +import type { ModelMessage } from '@tanstack/ai' +import type { MemoryAdapter } from '@tanstack/ai-memory' + +// From earlier steps / your auth layer. +declare const messages: Array +declare const memory: MemoryAdapter +declare const session: { userId: string; threadId: string } +declare function getSession(ctx: unknown): { threadId: string; userId: string } + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + context: { session }, // attached by your auth middleware, not from req.body + middleware: [ + memoryMiddleware({ + adapter: memory, + scope: (ctx) => { + const session = getSession(ctx) + return { sessionId: session.threadId, userId: session.userId } + }, + }), + ], +}) +``` + +On the client, nothing changes. `useChat` (or your connection adapter) consumes the +stream exactly as before. Memory is entirely server-side. + +## Where to go next + +- [Overview](./overview): the `recall`/`save` contract, scope, and how a turn flows +- [Adapters](./adapters): every adapter's options, with an example of each +- [Operating memory](./operating): options, telemetry, devtools events, and failures +- [Custom Adapter](./custom-adapter): implement `recall`/`save` for a backend that isn't shipped diff --git a/kiira.config.ts b/kiira.config.ts index 07fb98dce..159fba800 100644 --- a/kiira.config.ts +++ b/kiira.config.ts @@ -46,6 +46,7 @@ export default defineConfig({ hono: '^4.0.0', '@hono/node-server': '^2.0.0', redis: '^6.0.0', + ioredis: '^5.0.0', pino: '^10.0.0', '@opentelemetry/api': '^1.9.0', // Community adapters (each documented page imports its published package) diff --git a/knip.json b/knip.json index 015ff8f09..fe3c1e189 100644 --- a/knip.json +++ b/knip.json @@ -41,6 +41,12 @@ "packages/ai-openai": { "ignore": ["src/tools/**"] }, + "packages/ai-client": { + "ignoreDependencies": ["@standard-schema/spec"] + }, + "packages/ai-memory": { + "ignoreDependencies": ["ioredis", "redis"] + }, "packages/ai-sandbox": { "ignoreDependencies": ["@ngrok/ngrok"] }, diff --git a/packages/ai-client/src/chat-client.ts b/packages/ai-client/src/chat-client.ts index 920f6cd5d..c55cfd7b4 100644 --- a/packages/ai-client/src/chat-client.ts +++ b/packages/ai-client/src/chat-client.ts @@ -644,6 +644,13 @@ export class ChatClient< data: unknown, context: { toolCallId?: string }, ) => { + // Server-side memory middleware transports its state as a `memory:state` + // CUSTOM event (its own event bus never reaches this browser runtime). + // Route it to the devtools bridge here — the designated custom-event + // path — then still forward to the app's callback. + if (eventType === 'memory:state') { + this.devtoolsBridge.recordMemoryState(data) + } this.callbacksRef.current.onCustomEvent(eventType, data, context) }, }, diff --git a/packages/ai-client/src/devtools-noop.ts b/packages/ai-client/src/devtools-noop.ts index 4f1848eec..6a1f1ce00 100644 --- a/packages/ai-client/src/devtools-noop.ts +++ b/packages/ai-client/src/devtools-noop.ts @@ -86,6 +86,7 @@ export class NoOpChatDevtoolsBridge { return '' } observeChunk(_chunk: StreamChunk): void {} + recordMemoryState(_value: unknown): void {} beginRun(_runId: string, _threadId: string): void {} getCurrentRunEventContext(): ChatClientRunEventContext | undefined { return undefined diff --git a/packages/ai-client/src/devtools.ts b/packages/ai-client/src/devtools.ts index aba6ee5d3..eea282067 100644 --- a/packages/ai-client/src/devtools.ts +++ b/packages/ai-client/src/devtools.ts @@ -25,6 +25,33 @@ export interface AIDevtoolsDisplayOptions { name?: string } +/** + * Structural mirror of `MemoryStateEventValue` from `@tanstack/ai-memory` — the + * payload of the `memory:state` CUSTOM chunk. Kept local so `ai-client` doesn't + * depend on `ai-memory`; the memory middleware is the producer. + */ +interface MemoryStateEventValue { + scope: { sessionId?: string; userId?: string } + adapter: string + query?: string + recall?: { + fragmentCount?: number + hasTools?: boolean + systemPromptChars?: number + durationMs?: number + } + snapshot?: { + takenAt: string + data: unknown + facts?: Array<{ + id: string + text: string + source?: string + createdAt?: string + }> + } +} + export interface AIDevtoolsClientMetadata extends AIDevtoolsDisplayOptions { framework?: string hookName: string @@ -626,8 +653,16 @@ export class ClientDevtoolsBridge { this.emitRegistered() this.emitToolsRegistered() this.emitSnapshot() + this.onReplayState() } + /** + * Extension hook for subclasses to replay any additional cached state on a + * `devtools:request-state` (i.e. when a panel opens). Called only after the + * base guards (disposed/superseded/targetHookId) pass. No-op by default. + */ + protected onReplayState(): void {} + private async handleToolFixtureApply( event: AIDevtoolsEvent, ): Promise { @@ -657,13 +692,16 @@ export class ClientDevtoolsBridge { return true } - private createEnvelope( + protected createEnvelope( eventType: | 'hook:registered' | 'hook:updated' | 'hook:unregistered' | 'hook:state-snapshot' | 'tools:registered' + | 'memory:retrieve:started' + | 'memory:retrieve:completed' + | 'memory:snapshot' | AIDevtoolsRunEventType, visibility: AIDevtoolsEventVisibility = 'client-state', context: { runId?: string } = {}, @@ -740,6 +778,8 @@ export class ChatDevtoolsBridge extends ClientDevtoolsBridge { client.dispose() }) + it('re-emits memory:* devtools events from a transported memory:state CUSTOM chunk', async () => { + const runContexts: Array = [] + const chunks: Array = [ + runStartedChunk({ threadId: 'thread-1', runId: 'run-mem' }), + { + type: EventType.CUSTOM, + model: 'test', + timestamp: Date.now(), + name: 'memory:state', + value: { + scope: { sessionId: 'sess-1' }, + adapter: 'in-memory', + query: 'what is my name?', + recall: { + fragmentCount: 2, + hasTools: false, + systemPromptChars: 96, + durationMs: 4, + }, + snapshot: { + takenAt: '2026-07-22T00:00:00.000Z', + data: { + records: [{ id: 'r1', text: 'name is Jack', kind: 'message' }], + }, + facts: [{ id: 'r1', text: 'name is Jack', source: 'user' }], + }, + }, + }, + textContentChunk({ + messageId: 'msg-mem', + delta: 'Your name is Jack', + content: 'Your name is Jack', + }), + runFinishedChunk({ threadId: 'thread-1', runId: 'run-mem' }), + ] + const client = createClient({ + connection: createRunTrackingAdapter([chunks], runContexts), + }) + vi.clearAllMocks() + + await client.sendMessage('what is my name?') + await waitForCondition( + () => eventClientMock.emitted('memory:snapshot').length > 0, + ) + + expect(eventClientMock.emitted('memory:retrieve:started')).toEqual([ + [ + 'memory:retrieve:started', + expect.objectContaining({ + scope: { sessionId: 'sess-1' }, + adapter: 'in-memory', + query: 'what is my name?', + }), + ], + ]) + expect(eventClientMock.emitted('memory:retrieve:completed')).toEqual([ + [ + 'memory:retrieve:completed', + expect.objectContaining({ + adapter: 'in-memory', + fragmentCount: 2, + hasTools: false, + systemPromptChars: 96, + durationMs: 4, + }), + ], + ]) + expect(eventClientMock.emitted('memory:snapshot')).toEqual([ + [ + 'memory:snapshot', + expect.objectContaining({ + adapter: 'in-memory', + takenAt: '2026-07-22T00:00:00.000Z', + facts: [{ id: 'r1', text: 'name is Jack', source: 'user' }], + }), + ], + ]) + + // Replay: a devtools panel opening AFTER the turn requests state; the bridge + // re-emits the last memory:state so a late-opened panel isn't empty. + vi.clearAllMocks() + eventClientMock.dispatch('devtools:request-state', {}) + await waitForCondition( + () => eventClientMock.emitted('memory:snapshot').length > 0, + ) + expect(eventClientMock.emitted('memory:retrieve:completed')).toEqual([ + [ + 'memory:retrieve:completed', + expect.objectContaining({ adapter: 'in-memory', fragmentCount: 2 }), + ], + ]) + + client.dispose() + }) + it('batches structured output update events while preserving final state', async () => { const runContexts: Array = [] const finalObject = { title: 'Pasta', servings: 2 } diff --git a/packages/ai-devtools/src/components/hooks/HookDetails.tsx b/packages/ai-devtools/src/components/hooks/HookDetails.tsx index 2176711fb..cebade060 100644 --- a/packages/ai-devtools/src/components/hooks/HookDetails.tsx +++ b/packages/ai-devtools/src/components/hooks/HookDetails.tsx @@ -36,6 +36,7 @@ import { visiblePreviewPartsForMessage, } from './preview-messages' import { GenerationPanel, GenerationPreview } from './GenerationPanel' +import { MemoryPanel } from './MemoryPanel' import type { HoverOrigin, HoverTarget, PreviewJsonItem } from './preview-model' import type { HookRecord, @@ -46,7 +47,7 @@ import type { import type { Conversation, Message, ToolCall } from '../../store/ai-store' import type { Component, Setter } from 'solid-js' -type DetailTab = 'conversation' | 'tools' | 'state' +type DetailTab = 'conversation' | 'tools' | 'state' | 'memory' type MessagePart = NonNullable[number] const scrollAnimations = new WeakMap() @@ -144,7 +145,12 @@ export const HookDetails: Component = () => { }) createEffect(() => { - if (isGenerationHook() && activeTab() === 'tools') { + // Tools and Memory are chat-only tabs; if a generation hook becomes active + // while one of them is selected, fall back to the conversation view. + if ( + isGenerationHook() && + (activeTab() === 'tools' || activeTab() === 'memory') + ) { setActiveTab('conversation') } }) @@ -240,6 +246,14 @@ export const HookDetails: Component = () => { activeTab={activeTab()} onSelect={setActiveTab} /> + + +
{ + + + diff --git a/packages/ai-devtools/src/components/hooks/MemoryPanel.tsx b/packages/ai-devtools/src/components/hooks/MemoryPanel.tsx new file mode 100644 index 000000000..86a857078 --- /dev/null +++ b/packages/ai-devtools/src/components/hooks/MemoryPanel.tsx @@ -0,0 +1,290 @@ +import { For, Show, createMemo, createSignal } from 'solid-js' +import { useAIStore } from '../../store/ai-context' +import { useStyles } from '../../styles/use-styles' +import type { Component } from 'solid-js' +import type { + MemoryEventRecord, + MemoryScopeState, +} from '../../store/memory-registry' + +/** + * DevTools "Memory" tab. Memory is per-scope (sessionId), not per-hook, so this + * panel reads the whole `state.memory` registry and lets the user pick a scope + * (defaulting to the most recently active). It renders two things: + * 1. Live contents — the latest `inspect()` records + `listFacts()` facts, + * pushed via `memory:snapshot` (only for adapters that support inspection). + * 2. Operations timeline — the `memory:*` recall/save/error events (always + * available, even when the adapter has no introspection). + */ + +/** Shape of a record inside the built-in adapters' `inspect()` payload. */ +interface MemoryRecordRow { + id: string + text: string + kind: string + role?: string + createdAt?: number + importance?: number +} + +/** Best-effort extraction of `{ records: [...] }` from the opaque snapshot data. */ +function extractRecords(data: unknown): Array { + if (!data || typeof data !== 'object') return [] + const records = (data as { records?: unknown }).records + if (!Array.isArray(records)) return [] + return records.filter( + (r): r is MemoryRecordRow => + Boolean(r) && + typeof r === 'object' && + typeof (r as MemoryRecordRow).id === 'string' && + typeof (r as MemoryRecordRow).text === 'string', + ) +} + +function formatTime(value: number | string | undefined): string { + if (value === undefined) return '' + const date = new Date(value) + if (Number.isNaN(date.getTime())) return '' + return date.toLocaleTimeString() +} + +function eventSummary(event: MemoryEventRecord): string { + switch (event.type) { + case 'retrieve:started': + return `recall "${event.query ?? ''}"` + case 'retrieve:completed': + return `recalled ${event.fragmentCount ?? 0} fragment(s), ${ + event.systemPromptChars ?? 0 + } prompt chars${event.hasTools ? ', +tools' : ''} (${event.durationMs ?? 0}ms)` + case 'persist:started': + return 'save started' + case 'persist:completed': + return `saved ${event.okCount ?? 0}/${event.receiptCount ?? 0} receipt(s) (${ + event.durationMs ?? 0 + }ms)` + case 'error': + return `${event.phase ?? ''} error: ${event.error?.message ?? 'unknown'}` + default: + return event.type + } +} + +export const MemoryPanel: Component = () => { + const { state, clearMemory } = useAIStore() + const styles = useStyles() + const [override, setOverride] = createSignal(null) + + // Scope keys sorted most-recently-active first. + const scopeKeys = createMemo(() => + Object.values(state.memory.scopes) + .slice() + .sort((a, b) => b.lastActivity - a.lastActivity) + .map((s) => s.key), + ) + + const selectedKey = createMemo(() => { + const chosen = override() + if (chosen && state.memory.scopes[chosen]) return chosen + return scopeKeys()[0] ?? null + }) + + const scope = createMemo((): MemoryScopeState | undefined => { + const key = selectedKey() + return key ? state.memory.scopes[key] : undefined + }) + + const records = createMemo(() => extractRecords(scope()?.snapshot?.data)) + const facts = createMemo(() => scope()?.snapshot?.facts ?? []) + // Timeline newest-first. + const events = createMemo(() => + (scope()?.events ?? []).slice().sort((a, b) => b.timestamp - a.timestamp), + ) + + return ( +
+ + No memory activity yet. Send a message through a chat wired with + memoryMiddleware and recall/save events will appear + here. +
+ } + > + {(activeScope) => ( + <> +
+
+ 1} + fallback={ + + {activeScope().adapter ?? 'memory'} + + } + > + + +
+ +
+ + {/* Live contents (inspect + listFacts) */} +
+
+ Stored records ({records().length}) + + {(snap) => ( + + snapshot {formatTime(snap().takenAt)} + + )} + +
+ 0} + fallback={ +
+ {activeScope().snapshot + ? 'Snapshot is empty.' + : 'This adapter does not expose inspect() — see the timeline below for activity.'} +
+ } + > +
+ + {(rec) => ( +
+
+ + {rec.kind} + + + {rec.role} + + + importance {rec.importance?.toFixed(2)} + + + {formatTime(rec.createdAt)} + +
+
+ {rec.text} +
+
+ )} + +
+ +
+ + {/* Facts */} +
+
+ listFacts() ({facts().length}) +
+ 0} + fallback={ +
No facts.
+ } + > +
+ + {(fact) => ( +
+
+ + + {fact.source}:{' '} + + + {fact.text} +
+
+ )} + +
+ +
+ + {/* Operations timeline */} +
+
+ Operations ({events().length}) +
+ 0} + fallback={ +
+ No operations recorded. +
+ } + > +
+ + {(event) => ( +
+
+ + {event.type} + + + {formatTime(event.timestamp)} + +
+
+ {eventSummary(event)} +
+
+ )} + +
+ +
+ + )} + +
+ ) +} diff --git a/packages/ai-devtools/src/components/hooks/index.ts b/packages/ai-devtools/src/components/hooks/index.ts index 2cdc8a2f0..73a7fafb7 100644 --- a/packages/ai-devtools/src/components/hooks/index.ts +++ b/packages/ai-devtools/src/components/hooks/index.ts @@ -1,4 +1,5 @@ export { HookDashboard } from './HookDashboard' export { HookDetails } from './HookDetails' export { GenerationPanel, GenerationPreview } from './GenerationPanel' +export { MemoryPanel } from './MemoryPanel' export { ToolFixtureForm } from './ToolFixtureForm' diff --git a/packages/ai-devtools/src/store/ai-context.tsx b/packages/ai-devtools/src/store/ai-context.tsx index 65397303a..24c72290a 100644 --- a/packages/ai-devtools/src/store/ai-context.tsx +++ b/packages/ai-devtools/src/store/ai-context.tsx @@ -14,12 +14,19 @@ import { createClientToolCallMessage, shouldSkipClientAssistantPlaceholder, } from './message-event-utils' +import { + applyMemoryEvent, + applyMemorySnapshot, + clearMemoryRegistry, + createMemoryRegistryState, +} from './memory-registry' import type { ContentPartSource, TokenUsage } from '@tanstack/ai' import type { DevtoolsToolFixtureApplyEvent, RunLifecycleEvent, } from '@tanstack/ai-event-client' import type { HookRegistryState, ToolFixtureRecord } from './hook-registry' +import type { MemoryRegistryState } from './memory-registry' import type { ParentComponent } from 'solid-js' interface MessagePart { @@ -227,6 +234,7 @@ interface AIStoreState { conversations: Record activeConversationId: string | null hooks: HookRegistryState + memory: MemoryRegistryState } interface AIContextValue { @@ -234,6 +242,7 @@ interface AIContextValue { clearAllConversations: () => void selectConversation: (id: string) => void clearHooks: () => void + clearMemory: () => void selectHook: (id: string | null) => void saveToolFixture: (fixture: ToolFixtureRecord) => void deleteToolFixture: (fixtureId: string) => void @@ -255,6 +264,7 @@ export const AIProvider: ParentComponent = (props) => { conversations: {}, activeConversationId: null, hooks: createHookRegistryState(), + memory: createMemoryRegistryState(), }) const streamToConversation = new Map() @@ -641,6 +651,15 @@ export const AIProvider: ParentComponent = (props) => { ) } + function clearMemory() { + setState( + 'memory', + produce((memory: MemoryRegistryState) => { + clearMemoryRegistry(memory) + }), + ) + } + function selectHook(id: string | null) { setState( 'hooks', @@ -1188,6 +1207,47 @@ export const AIProvider: ParentComponent = (props) => { }), ) + // Memory: the 5 `memory:*` operation events feed the timeline; `memory:snapshot` + // replaces the per-scope stored-state view. Keyed by scope (sessionId). + type MemoryEventInput = Parameters[1] + const recordMemoryEvent = ( + type: MemoryEventInput['type'], + payload: object, + ) => { + setState( + 'memory', + produce((memory: MemoryRegistryState) => { + applyMemoryEvent(memory, { ...payload, type } as MemoryEventInput) + }), + ) + } + + cleanupFns.push( + aiEventClient.on('memory:retrieve:started', (e) => { + recordMemoryEvent('retrieve:started', e.payload) + }), + aiEventClient.on('memory:retrieve:completed', (e) => { + recordMemoryEvent('retrieve:completed', e.payload) + }), + aiEventClient.on('memory:persist:started', (e) => { + recordMemoryEvent('persist:started', e.payload) + }), + aiEventClient.on('memory:persist:completed', (e) => { + recordMemoryEvent('persist:completed', e.payload) + }), + aiEventClient.on('memory:error', (e) => { + recordMemoryEvent('error', e.payload) + }), + aiEventClient.on('memory:snapshot', (e) => { + setState( + 'memory', + produce((memory: MemoryRegistryState) => { + applyMemorySnapshot(memory, e.payload) + }), + ) + }), + ) + const recordRunEvent = ( eventName: | 'run:created' @@ -3402,6 +3462,7 @@ export const AIProvider: ParentComponent = (props) => { clearAllConversations, selectConversation, clearHooks, + clearMemory, selectHook, saveToolFixture, deleteToolFixture, diff --git a/packages/ai-devtools/src/store/memory-registry.ts b/packages/ai-devtools/src/store/memory-registry.ts new file mode 100644 index 000000000..4fd1c9138 --- /dev/null +++ b/packages/ai-devtools/src/store/memory-registry.ts @@ -0,0 +1,190 @@ +import type { + MemoryErrorEvent, + MemoryPersistCompletedEvent, + MemoryPersistStartedEvent, + MemoryRetrieveCompletedEvent, + MemoryRetrieveStartedEvent, + MemoryScopeLite, + MemorySnapshotEvent, +} from '@tanstack/ai-event-client' + +/** + * DevTools-side accumulator for the `memory:*` event stream. Kept as a set of + * pure reducers (mirroring `hook-registry.ts`) so the mapping from events → + * view state is unit-testable in isolation, without a Solid store. + * + * Memory is keyed by scope (sessionId), NOT by hook — several hooks can share + * one session. The `MemoryPanel` reads a single `MemoryScopeState` by key; the + * per-hook tab just resolves which key to show. + */ + +/** One row in a scope's operations timeline. */ +export interface MemoryEventRecord { + id: string + type: + | 'retrieve:started' + | 'retrieve:completed' + | 'persist:started' + | 'persist:completed' + | 'error' + timestamp: number + adapter: string + /** recall:started — the recall query (last user text). */ + query?: string + /** recall:completed. */ + fragmentCount?: number + hasTools?: boolean + systemPromptChars?: number + /** persist:completed. */ + receiptCount?: number + okCount?: number + /** recall:completed / persist:completed. */ + durationMs?: number + /** error. */ + phase?: 'recall' | 'save' + error?: { name: string; message: string } +} + +/** A flat fact row, mirroring `MemoryFact` from `@tanstack/ai-memory`. */ +export interface MemoryFactRecord { + id: string + text: string + source?: string + createdAt?: string +} + +/** Latest `inspect()` + `listFacts()` snapshot pushed via `memory:snapshot`. */ +export interface MemorySnapshotRecord { + takenAt: string + data: unknown + facts: Array +} + +/** Everything known about memory for a single scope (sessionId). */ +export interface MemoryScopeState { + key: string + sessionId: string + userId?: string + /** Most recent adapter id seen for this scope. */ + adapter?: string + events: Array + snapshot?: MemorySnapshotRecord + lastActivity: number +} + +export interface MemoryRegistryState { + scopes: Record +} + +export function createMemoryRegistryState(): MemoryRegistryState { + return { scopes: {} } +} + +/** Stable scope key. Empty/absent sessionId (e.g. error scope) buckets to `(unknown)`. */ +export function memoryScopeKey(scope: MemoryScopeLite | undefined): string { + const sessionId = scope?.sessionId + return sessionId && sessionId.length > 0 ? sessionId : '(unknown)' +} + +const MAX_EVENTS_PER_SCOPE = 200 + +function ensureScope( + state: MemoryRegistryState, + scope: MemoryScopeLite | undefined, +): MemoryScopeState { + const key = memoryScopeKey(scope) + let entry = state.scopes[key] + if (!entry) { + entry = { + key, + sessionId: scope?.sessionId ?? '', + userId: scope?.userId, + events: [], + lastActivity: 0, + } + state.scopes[key] = entry + } + if (scope?.userId) entry.userId = scope.userId + return entry +} + +let fallbackCounter = 0 + +function eventId( + payload: { eventId?: string }, + type: string, + ts: number, +): string { + if (payload.eventId && payload.eventId.length > 0) return payload.eventId + return `${type}:${ts}:${fallbackCounter++}` +} + +type MemoryEventPayload = + | ({ type: 'retrieve:started' } & MemoryRetrieveStartedEvent) + | ({ type: 'retrieve:completed' } & MemoryRetrieveCompletedEvent) + | ({ type: 'persist:started' } & MemoryPersistStartedEvent) + | ({ type: 'persist:completed' } & MemoryPersistCompletedEvent) + | ({ type: 'error' } & MemoryErrorEvent) + +/** Append one `memory:*` operation event to its scope's timeline. */ +export function applyMemoryEvent( + state: MemoryRegistryState, + event: MemoryEventPayload, +): void { + const entry = ensureScope(state, event.scope) + entry.adapter = event.adapter + entry.lastActivity = Math.max(entry.lastActivity, event.timestamp) + + const record: MemoryEventRecord = { + id: eventId(event, event.type, event.timestamp), + type: event.type, + timestamp: event.timestamp, + adapter: event.adapter, + } + switch (event.type) { + case 'retrieve:started': + record.query = event.query + break + case 'retrieve:completed': + record.fragmentCount = event.fragmentCount + record.hasTools = event.hasTools + record.systemPromptChars = event.systemPromptChars + record.durationMs = event.durationMs + break + case 'persist:completed': + record.receiptCount = event.receiptCount + record.okCount = event.okCount + record.durationMs = event.durationMs + break + case 'error': + record.phase = event.phase + record.error = event.error + break + case 'persist:started': + break + } + + entry.events.push(record) + if (entry.events.length > MAX_EVENTS_PER_SCOPE) { + entry.events.splice(0, entry.events.length - MAX_EVENTS_PER_SCOPE) + } +} + +/** Replace a scope's stored-state snapshot from a `memory:snapshot` event. */ +export function applyMemorySnapshot( + state: MemoryRegistryState, + event: MemorySnapshotEvent, +): void { + const entry = ensureScope(state, event.scope) + entry.adapter = event.adapter + entry.lastActivity = Math.max(entry.lastActivity, event.timestamp) + entry.snapshot = { + takenAt: event.takenAt, + data: event.data, + facts: event.facts, + } +} + +export function clearMemoryRegistry(state: MemoryRegistryState): void { + state.scopes = {} +} diff --git a/packages/ai-devtools/src/styles/use-styles.ts b/packages/ai-devtools/src/styles/use-styles.ts index 62d9b6c57..43e003d9a 100644 --- a/packages/ai-devtools/src/styles/use-styles.ts +++ b/packages/ai-devtools/src/styles/use-styles.ts @@ -1898,6 +1898,118 @@ const stylesFactory = (theme: 'light' | 'dark') => { padding: 0 ${size[3]}; `, }, + // MemoryPanel component styles + memoryPanel: { + container: css` + display: flex; + flex: 1; + min-height: 0; + flex-direction: column; + gap: ${size[4]}; + overflow-y: auto; + padding: ${size[4]}; + `, + toolbar: css` + display: flex; + align-items: center; + justify-content: space-between; + gap: ${size[3]}; + `, + scopeControls: css` + display: flex; + align-items: center; + gap: ${size[2]}; + min-width: 0; + `, + scopeSelect: css` + max-width: 220px; + border: 1px solid ${t(colors.gray[200], colors.darkGray[700])}; + border-radius: ${border.radius.sm}; + background: ${t(colors.white, colors.darkGray[800])}; + color: ${t(colors.gray[900], colors.gray[100])}; + font-family: ${fontFamily.mono}; + font-size: ${fontSize.xs}; + padding: ${size[1]} ${size[2]}; + `, + clearButton: css` + border: 1px solid ${t(colors.gray[200], colors.darkGray[700])}; + border-radius: ${border.radius.sm}; + background: ${t(colors.gray[50], colors.darkGray[700])}; + color: ${t(colors.gray[700], colors.gray[200])}; + cursor: pointer; + font-size: ${fontSize.xs}; + padding: ${size[1]} ${size[2]}; + &:hover { + border-color: ${t(colors.blue[300], colors.blue[700])}; + } + `, + empty: css` + color: ${t(colors.gray[500], colors.gray[500])}; + font-size: ${fontSize.sm}; + padding: ${size[4]}; + text-align: center; + `, + section: css` + display: flex; + flex-direction: column; + gap: ${size[2]}; + `, + sectionTitle: css` + color: ${t(colors.gray[600], colors.gray[300])}; + font-size: ${fontSize.xs}; + font-weight: ${font.weight.semibold}; + text-transform: uppercase; + letter-spacing: 0.04em; + `, + sectionEmpty: css` + color: ${t(colors.gray[500], colors.gray[500])}; + font-size: ${fontSize.xs}; + `, + list: css` + display: flex; + flex-direction: column; + gap: ${size[1]}; + `, + row: css` + display: flex; + flex-direction: column; + gap: 2px; + border: 1px solid ${t(colors.gray[200], colors.darkGray[700])}; + border-radius: ${border.radius.sm}; + background: ${t(colors.white, colors.darkGray[800])}; + padding: ${size[2]}; + `, + rowError: css` + border-color: ${t(colors.red[300], colors.red[700])}; + background: ${t(colors.red[50], colors.red[900] + '20')}; + `, + rowHeader: css` + display: flex; + align-items: center; + gap: ${size[2]}; + font-size: ${fontSize.xs}; + color: ${t(colors.gray[500], colors.gray[400])}; + `, + badge: css` + border-radius: ${border.radius.xs}; + background: ${t(colors.blue[50], colors.blue[900] + '30')}; + color: ${t(colors.blue[700], colors.blue[300])}; + font-family: ${fontFamily.mono}; + font-size: 10px; + padding: 1px 6px; + `, + time: css` + margin-left: auto; + font-family: ${fontFamily.mono}; + font-size: 10px; + `, + rowText: css` + color: ${t(colors.gray[900], colors.gray[100])}; + font-size: ${fontSize.sm}; + white-space: pre-wrap; + word-break: break-word; + `, + }, // ConversationDetails component styles conversationDetails: { emptyState: css` diff --git a/packages/ai-devtools/tests/memory-registry.test.ts b/packages/ai-devtools/tests/memory-registry.test.ts new file mode 100644 index 000000000..d9d41c3ac --- /dev/null +++ b/packages/ai-devtools/tests/memory-registry.test.ts @@ -0,0 +1,139 @@ +import { describe, expect, it } from 'vitest' +import { + applyMemoryEvent, + applyMemorySnapshot, + clearMemoryRegistry, + createMemoryRegistryState, + memoryScopeKey, +} from '../src/store/memory-registry' +import type { MemorySnapshotEvent } from '@tanstack/ai-event-client' + +const SCOPE = { sessionId: 'session-1' } + +describe('memory registry', () => { + it('accumulates the operation timeline per scope', () => { + const state = createMemoryRegistryState() + + applyMemoryEvent(state, { + type: 'retrieve:started', + scope: SCOPE, + adapter: 'in-memory', + query: 'What is my name?', + timestamp: 10, + }) + applyMemoryEvent(state, { + type: 'retrieve:completed', + scope: SCOPE, + adapter: 'in-memory', + fragmentCount: 2, + hasTools: false, + systemPromptChars: 128, + durationMs: 5, + timestamp: 12, + }) + applyMemoryEvent(state, { + type: 'persist:completed', + scope: SCOPE, + adapter: 'in-memory', + receiptCount: 2, + okCount: 2, + durationMs: 3, + timestamp: 20, + }) + + const entry = state.scopes[memoryScopeKey(SCOPE)] + expect(entry).toBeDefined() + expect(entry!.adapter).toBe('in-memory') + expect(entry!.lastActivity).toBe(20) + expect(entry!.events).toHaveLength(3) + expect(entry!.events[0]).toMatchObject({ + type: 'retrieve:started', + query: 'What is my name?', + }) + expect(entry!.events[1]).toMatchObject({ + type: 'retrieve:completed', + fragmentCount: 2, + systemPromptChars: 128, + }) + expect(entry!.events[2]).toMatchObject({ + type: 'persist:completed', + okCount: 2, + receiptCount: 2, + }) + }) + + it('records error events with phase and message', () => { + const state = createMemoryRegistryState() + applyMemoryEvent(state, { + type: 'error', + scope: SCOPE, + adapter: 'in-memory', + phase: 'recall', + error: { name: 'Error', message: 'boom' }, + timestamp: 1, + }) + const entry = state.scopes[memoryScopeKey(SCOPE)] + expect(entry!.events[0]).toMatchObject({ + type: 'error', + phase: 'recall', + error: { message: 'boom' }, + }) + }) + + it('replaces the snapshot on memory:snapshot', () => { + const state = createMemoryRegistryState() + const snapshot: MemorySnapshotEvent = { + scope: SCOPE, + adapter: 'in-memory', + takenAt: '2026-07-22T00:00:00.000Z', + data: { + records: [ + { id: 'r1', text: 'My name is Jack', kind: 'message', role: 'user' }, + ], + }, + facts: [{ id: 'r1', text: 'My name is Jack', source: 'user' }], + timestamp: 30, + } + applyMemorySnapshot(state, snapshot) + + const entry = state.scopes[memoryScopeKey(SCOPE)] + expect(entry!.snapshot?.takenAt).toBe('2026-07-22T00:00:00.000Z') + expect(entry!.snapshot?.facts).toHaveLength(1) + + // A newer snapshot fully replaces the prior one. + applyMemorySnapshot(state, { ...snapshot, facts: [], timestamp: 40 }) + expect(state.scopes[memoryScopeKey(SCOPE)]!.snapshot?.facts).toEqual([]) + expect(state.scopes[memoryScopeKey(SCOPE)]!.lastActivity).toBe(40) + }) + + it('isolates scopes and buckets missing sessionId to (unknown)', () => { + const state = createMemoryRegistryState() + applyMemoryEvent(state, { + type: 'persist:started', + scope: { sessionId: 'a' }, + adapter: 'in-memory', + timestamp: 1, + }) + applyMemoryEvent(state, { + type: 'error', + scope: { sessionId: '' }, + adapter: 'in-memory', + phase: 'save', + error: { name: 'Error', message: 'x' }, + timestamp: 2, + }) + expect(Object.keys(state.scopes).sort()).toEqual(['(unknown)', 'a']) + }) + + it('clears the registry', () => { + const state = createMemoryRegistryState() + applyMemoryEvent(state, { + type: 'persist:started', + scope: SCOPE, + adapter: 'in-memory', + timestamp: 1, + }) + clearMemoryRegistry(state) + expect(state.scopes).toEqual({}) + }) +}) diff --git a/packages/ai-event-client/src/index.ts b/packages/ai-event-client/src/index.ts index 0bd1fdc12..fa36b3d24 100644 --- a/packages/ai-event-client/src/index.ts +++ b/packages/ai-event-client/src/index.ts @@ -820,6 +820,93 @@ export interface VideoUsageEvent extends BaseEventContext { usage: TokenUsage } +// --------------------------------------------------------------------------- +// Memory events +// --------------------------------------------------------------------------- + +/** + * Lite scope for devtools payloads. Mirrors the `MemoryScope` contract in + * `@tanstack/ai-memory` (session-centric); kept structurally minimal so the + * event client stays decoupled from the memory package. + */ +export type MemoryScopeLite = { + sessionId?: string + userId?: string +} + +/** Emitted when the middleware begins a `recall` for the current turn. */ +export interface MemoryRetrieveStartedEvent extends BaseEventContext { + scope: MemoryScopeLite + /** Adapter id (e.g. 'in-memory', 'hindsight'). */ + adapter: string + /** Recall query text (typically the last user message). */ + query: string +} + +/** Emitted when `recall` returns, before the result is injected into the prompt. */ +export interface MemoryRetrieveCompletedEvent extends BaseEventContext { + scope: MemoryScopeLite + adapter: string + /** Number of discrete fragments returned (0 when the adapter synthesizes). */ + fragmentCount: number + /** Whether the recall result injected any tools this turn. */ + hasTools: boolean + /** Length (chars) of the rendered system-prompt block. */ + systemPromptChars: number + durationMs: number +} + +/** Emitted when the middleware begins a deferred `save` for the finished turn. */ +export interface MemoryPersistStartedEvent extends BaseEventContext { + scope: MemoryScopeLite + adapter: string +} + +/** Emitted when a deferred `save` completes. */ +export interface MemoryPersistCompletedEvent extends BaseEventContext { + scope: MemoryScopeLite + adapter: string + /** Total receipts returned by `save`. */ + receiptCount: number + /** Receipts with `ok: true`. */ + okCount: number + durationMs: number +} + +/** Emitted when a `recall` or `save` throws. Memory failures are non-fatal. */ +export interface MemoryErrorEvent extends BaseEventContext { + scope: MemoryScopeLite + adapter: string + phase: 'recall' | 'save' + error: { name: string; message: string } +} + +/** A flat fact row, mirroring `MemoryFact` from `@tanstack/ai-memory`. */ +export interface MemoryFactLite { + id: string + text: string + source?: string + createdAt?: string +} + +/** + * Emitted after a successful `save` when the adapter supports introspection + * (`inspect`/`listFacts`), carrying the current stored state for the scope so + * DevTools can render "what's in memory". Adapters without introspection never + * emit this — DevTools then falls back to the metrics-only timeline. Structurally + * decoupled from `@tanstack/ai-memory` (mirrors `MemorySnapshot` + `MemoryFact`). + */ +export interface MemorySnapshotEvent extends BaseEventContext { + scope: MemoryScopeLite + adapter: string + /** ISO timestamp the snapshot was taken (from `MemorySnapshot.takenAt`). */ + takenAt: string + /** Adapter-defined `inspect()` payload (e.g. `{ records: [...] }`). */ + data: unknown + /** Flat fact list from `listFacts()`; `[]` when the adapter lacks it. */ + facts: Array +} + // =========================== // Client Events // =========================== @@ -1050,6 +1137,14 @@ export interface AIDevtoolsEventMap { 'client:messages:cleared': ClientMessagesClearedEvent 'client:reloaded': ClientReloadedEvent 'client:stopped': ClientStoppedEvent + + // Memory events + 'memory:retrieve:started': MemoryRetrieveStartedEvent + 'memory:retrieve:completed': MemoryRetrieveCompletedEvent + 'memory:persist:started': MemoryPersistStartedEvent + 'memory:persist:completed': MemoryPersistCompletedEvent + 'memory:error': MemoryErrorEvent + 'memory:snapshot': MemorySnapshotEvent } class AiEventClient extends EventClient { diff --git a/packages/ai-memory/package.json b/packages/ai-memory/package.json new file mode 100644 index 000000000..c96b8256b --- /dev/null +++ b/packages/ai-memory/package.json @@ -0,0 +1,96 @@ +{ + "name": "@tanstack/ai-memory", + "version": "0.0.0", + "description": "Pluggable memory adapters for TanStack AI memoryMiddleware", + "author": "", + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/TanStack/ai.git", + "directory": "packages/ai-memory" + }, + "type": "module", + "module": "./dist/esm/index.js", + "types": "./dist/esm/index.d.ts", + "exports": { + ".": { + "types": "./dist/esm/index.d.ts", + "import": "./dist/esm/index.js" + }, + "./in-memory": { + "types": "./dist/esm/providers/in-memory/index.d.ts", + "import": "./dist/esm/providers/in-memory/index.js" + }, + "./redis": { + "types": "./dist/esm/providers/redis/index.d.ts", + "import": "./dist/esm/providers/redis/index.js" + }, + "./hindsight": { + "types": "./dist/esm/providers/hindsight/index.d.ts", + "import": "./dist/esm/providers/hindsight/index.js" + }, + "./mem0": { + "types": "./dist/esm/providers/mem0/index.d.ts", + "import": "./dist/esm/providers/mem0/index.js" + }, + "./honcho": { + "types": "./dist/esm/providers/honcho/index.d.ts", + "import": "./dist/esm/providers/honcho/index.js" + } + }, + "sideEffects": false, + "files": [ + "dist", + "src", + "skills" + ], + "scripts": { + "build": "vite build", + "clean": "premove ./build ./dist", + "lint:fix": "oxlint src --type-aware --fix", + "test:build": "publint --strict", + "test:oxlint": "oxlint src --type-aware", + "test:lib": "vitest --passWithNoTests", + "test:lib:dev": "pnpm test:lib --watch", + "test:types": "tsc" + }, + "keywords": [ + "ai", + "tanstack", + "memory", + "redis", + "rag" + ], + "dependencies": { + "@tanstack/ai-event-client": "workspace:*" + }, + "peerDependencies": { + "@honcho-ai/sdk": ">=2.0.0", + "@tanstack/ai": "workspace:^", + "@vectorize-io/hindsight-client": ">=0.6.0", + "ioredis": ">=5.0.0", + "redis": ">=4.0.0" + }, + "peerDependenciesMeta": { + "ioredis": { + "optional": true + }, + "redis": { + "optional": true + }, + "@vectorize-io/hindsight-client": { + "optional": true + }, + "@honcho-ai/sdk": { + "optional": true + } + }, + "devDependencies": { + "@honcho-ai/sdk": "^2.1.1", + "@tanstack/ai": "workspace:*", + "@vectorize-io/hindsight-client": "^0.6.1", + "@vitest/coverage-v8": "4.0.14", + "ioredis-mock": "^8.9.0", + "redis": "^4.7.0" + } +} diff --git a/packages/ai-memory/project.json b/packages/ai-memory/project.json new file mode 100644 index 000000000..242f783af --- /dev/null +++ b/packages/ai-memory/project.json @@ -0,0 +1,3 @@ +{ + "name": "@tanstack/ai-memory" +} diff --git a/packages/ai-memory/skills/tanstack-ai-memory-hindsight/SKILL.md b/packages/ai-memory/skills/tanstack-ai-memory-hindsight/SKILL.md new file mode 100644 index 000000000..fa12efaa9 --- /dev/null +++ b/packages/ai-memory/skills/tanstack-ai-memory-hindsight/SKILL.md @@ -0,0 +1,38 @@ +--- +name: tanstack-ai-memory-hindsight +description: Use when wiring hindsight() from @tanstack/ai-memory/hindsight — a hosted memory adapter that buckets memory per conversation and exposes retain/recall/reflect tools to the model. Requires the optional @vectorize-io/hindsight-client peer. +--- + +# Hindsight Memory Adapter + +Hosted `recall`/`save` adapter backed by Hindsight. Hindsight owns extraction and +ranking server-side, buckets memory into per-conversation "banks" +(`{userId}__{sessionId}`), and — uniquely — exposes LLM **tools** through `recall` so the +model can retain/recall/reflect directly. + +## Setup + +```ts +import { memoryMiddleware } from '@tanstack/ai-memory' +import { hindsight } from '@tanstack/ai-memory/hindsight' + +const memory = hindsight({ user: currentUserId }) // baseUrl defaults to HINDSIGHT_URL + +memoryMiddleware({ adapter: memory, scope }) +``` + +`@vectorize-io/hindsight-client` is an **optional peer dependency**, loaded lazily on +first use — install it where you use `hindsight()`. + +## Options + +- `user` — durable user id for the bank key (falls back to `scope.userId`). +- `baseUrl` — Hindsight server URL (default `HINDSIGHT_URL` or `http://localhost:8888`). +- `budget` — recall budget: `'low' | 'mid' | 'high'` (default `'mid'`). +- `onToolRetain` / `onToolRecall` — callbacks fired when the model uses the memory tools. + +## Tools + +`recall` returns `hindsight_retain`, `hindsight_recall`, and `hindsight_reflect` in its +`tools` plus a `toolGuidance` block. `memoryMiddleware` merges them into the run so the +model can manage long-term memory itself. diff --git a/packages/ai-memory/skills/tanstack-ai-memory-honcho/SKILL.md b/packages/ai-memory/skills/tanstack-ai-memory-honcho/SKILL.md new file mode 100644 index 000000000..a41b59790 --- /dev/null +++ b/packages/ai-memory/skills/tanstack-ai-memory-honcho/SKILL.md @@ -0,0 +1,36 @@ +--- +name: tanstack-ai-memory-honcho +description: Use when wiring honcho() from @tanstack/ai-memory/honcho — a hosted memory adapter where recall is a dialectic answer over the user's representation (no discrete fragments). Requires the optional @honcho-ai/sdk peer. +--- + +# Honcho Memory Adapter + +Hosted `recall`/`save` adapter backed by Honcho. Honcho models memory as peers +exchanging messages in a session; `recall` returns a **synthesized dialectic answer** +over the user peer's representation (so there are no discrete fragments), and `save` +appends the turn's messages to the session. + +## Setup + +```ts +import { memoryMiddleware } from '@tanstack/ai-memory' +import { honcho } from '@tanstack/ai-memory/honcho' + +const memory = honcho({ user: currentUserId }) // baseURL defaults to HONCHO_URL + +memoryMiddleware({ adapter: memory, scope }) +``` + +`@honcho-ai/sdk` is an **optional peer dependency**, loaded lazily on first use — install +it where you use `honcho()`. + +## Options + +- `user` — user peer id (falls back to `scope.userId`, then `'demo-user'`). +- `baseURL` — Honcho server URL (default `HONCHO_URL` or `http://localhost:8001`). +- `workspaceId` — default `HONCHO_APP_NAME` or `'ai-memory'`. +- `apiKey` — default `HONCHO_API_KEY`. +- `assistantId` — assistant peer id (default `'assistant'`). + +`recall` calls the user peer's dialectic `chat()` and injects the answer as the system +prompt; Honcho exposes no LLM tools. diff --git a/packages/ai-memory/skills/tanstack-ai-memory-in-memory/SKILL.md b/packages/ai-memory/skills/tanstack-ai-memory-in-memory/SKILL.md new file mode 100644 index 000000000..03f62719e --- /dev/null +++ b/packages/ai-memory/skills/tanstack-ai-memory-in-memory/SKILL.md @@ -0,0 +1,51 @@ +--- +name: tanstack-ai-memory-in-memory +description: Use when wiring inMemory() from @tanstack/ai-memory/in-memory — explains setup, options (embedder, extract, topK/minScore), when to pick it (dev/tests/single-process demos), and what NOT to use it for (multi-process or persistent). +--- + +# In-Memory Memory Adapter + +Zero-dependency `recall`/`save` adapter backed by a `Map`. Records vanish on process +restart. + +## When to use it + +- Local development. +- Vitest / Playwright tests. +- Single-process demos where users don't need persistence. + +## When NOT to use it + +- Production multi-process deployments — every worker has its own `Map`; users get + inconsistent memory. +- Anything that needs survival across restarts. + +For production, use `redis()` (see the `tanstack-ai-memory-redis` skill). + +## Setup + +```ts +import { memoryMiddleware } from '@tanstack/ai-memory' +import { inMemory } from '@tanstack/ai-memory/in-memory' + +const memory = inMemory() + +memoryMiddleware({ adapter: memory, scope }) +``` + +## Options + +`inMemory(options?)` accepts: + +- `topK` (default 6), `minScore` (default 0.15), `kinds` — recall tuning. +- `embedder: { embed(text): Promise }` — enable semantic scoring (both + `recall` and `save` embed through it). +- `extract(turn, scope)` — return derived facts to persist alongside the raw turn + (e.g. call an LLM to pull out preferences). Without it, `save` stores the raw + user/assistant messages and `recall` scores them lexically + by recency. +- `render(hits)` — replace the built-in prompt renderer. + +## Capacity + +The adapter scans every record in a scope per `recall`. Fine up to ~100k records; beyond +that, switch to Redis. diff --git a/packages/ai-memory/skills/tanstack-ai-memory-mem0/SKILL.md b/packages/ai-memory/skills/tanstack-ai-memory-mem0/SKILL.md new file mode 100644 index 000000000..7f442f340 --- /dev/null +++ b/packages/ai-memory/skills/tanstack-ai-memory-mem0/SKILL.md @@ -0,0 +1,33 @@ +--- +name: tanstack-ai-memory-mem0 +description: Use when wiring mem0() from @tanstack/ai-memory/mem0 — a hosted memory adapter that talks to a mem0 server over plain HTTP (no SDK peer). Requires a running mem0 server. +--- + +# mem0 Memory Adapter + +Hosted `recall`/`save` adapter backed by a mem0 server. mem0 owns extraction and ranking +server-side. Talks to the server over plain HTTP — **no SDK peer dependency**. + +## Setup + +```ts +import { memoryMiddleware } from '@tanstack/ai-memory' +import { mem0 } from '@tanstack/ai-memory/mem0' + +const memory = mem0({ user: currentUserId }) // baseUrl defaults to MEM0_URL + +memoryMiddleware({ adapter: memory, scope }) +``` + +Requires a running mem0 server (self-hosted or hosted). Point it via `baseUrl` (or +`MEM0_URL`); pass `apiKey` (or `MEM0_ADMIN_API_KEY`) when secured. + +## Options + +- `user` — mem0 `user_id` (falls back to `scope.userId`, then `'demo-user'`). +- `baseUrl` — mem0 server URL (default `MEM0_URL` or `http://localhost:8000`). +- `apiKey` — bearer token (default `MEM0_ADMIN_API_KEY`). +- `rerank` (default `true`), `threshold` (default `0.1`) — search tuning. + +`save` posts the `{ user, assistant }` turn to `/memories`; `recall` queries `/search` +and renders the results into the system prompt. mem0 exposes no LLM tools. diff --git a/packages/ai-memory/skills/tanstack-ai-memory-redis/SKILL.md b/packages/ai-memory/skills/tanstack-ai-memory-redis/SKILL.md new file mode 100644 index 000000000..09c20eb0e --- /dev/null +++ b/packages/ai-memory/skills/tanstack-ai-memory-redis/SKILL.md @@ -0,0 +1,77 @@ +--- +name: tanstack-ai-memory-redis +description: Use when wiring redis() from @tanstack/ai-memory/redis in production — covers client setup (ioredis or node-redis via fromNodeRedis), the storage model, client-side ranking limits, and troubleshooting. +--- + +# Redis Memory Adapter + +Production-grade `recall`/`save` adapter backed by plain Redis (no vector index +required). Ranks client-side (lexical + optional cosine + recency + importance). + +## Setup + +Bring your own Redis client. `ioredis` wires in directly; `redis` (node-redis v4+) needs +a small wrapper. + +### Option A: `ioredis` + +```ts +import Redis from 'ioredis' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { redis } from '@tanstack/ai-memory/redis' + +const client = new Redis(process.env.REDIS_URL) +const memory = redis({ redis: client, prefix: 'myapp:memory' }) + +memoryMiddleware({ adapter: memory, scope }) +``` + +### Option B: `redis` (node-redis v4+) + +```ts +import { createClient } from 'redis' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { redis, fromNodeRedis } from '@tanstack/ai-memory/redis' + +const client = createClient({ url: process.env.REDIS_URL }) +await client.connect() + +const memory = redis({ + redis: fromNodeRedis(client), + prefix: 'myapp:memory', +}) + +memoryMiddleware({ adapter: memory, scope }) +``` + +node-redis exposes a camelCase API (`sAdd`, `mGet`); `fromNodeRedis` translates it +to the lowercase `RedisLike` shape. Passing a raw node-redis client without the wrapper +throws `client.sadd is not a function`. + +`redis()` accepts the same `topK` / `minScore` / `kinds` / `embedder` / `extract` options +as `inMemory()`. + +## Storage model + +```text +{prefix}:record:{id} -> JSON record +{prefix}:index:{userId or _}:{sessionId} -> Set +``` + +`save` writes the record and adds it to the scope's index set; `recall` loads the set, +scores, and renders. Scope values are escaped so a `:` in a `userId`/`sessionId` can't +collide two scopes. + +## Ranking limits + +Ranking is client-side: `recall` loads every record for the scope into Node and scores +it. Fine up to ~10k records per scope. Beyond that, write a vector-index-aware adapter +against the same `recall`/`save` contract. + +## Troubleshooting + +- **Records not visible across processes:** ensure every process uses the same + `REDIS_URL` and `prefix`. +- **Malformed JSON rows:** a row whose JSON won't parse is skipped on read and **left in + place** (never deleted) — the signal is a one-time `console.warn` per bad id. Fix or + delete the offending `{prefix}:record:{id}` key to remediate. diff --git a/packages/ai-memory/skills/tanstack-ai-memory/SKILL.md b/packages/ai-memory/skills/tanstack-ai-memory/SKILL.md new file mode 100644 index 000000000..9721e3e09 --- /dev/null +++ b/packages/ai-memory/skills/tanstack-ai-memory/SKILL.md @@ -0,0 +1,94 @@ +--- +name: tanstack-ai-memory +description: Use when wiring memoryMiddleware from @tanstack/ai-memory into a chat() call — covers the recall/save adapter contract, scope shape and server-side scope security, the recall-inject / deferred-save lifecycle, choosing an adapter (inMemory, redis, hindsight, mem0, honcho), and devtools events. +--- + +# TanStack AI Memory Middleware + +Use this when adding **server-side memory** to a `chat()` call. Everything lives in +`@tanstack/ai-memory`. A memory adapter is a single contract with two verbs — `recall` +and `save` — and the middleware is thin: it recalls into the system prompt before the +model runs and defers `save` after the turn finishes. + +## When to reach for it + +- A user expects "remember what I told you last time." +- Per-user or per-thread context that must survive across sessions. +- A hosted memory service (mem0, Honcho, Hindsight). + +Do NOT use this just to keep recent messages — that's the `messages` array on `chat()`. +Memory is for cross-turn / cross-session recall, not within-turn history. + +## Wire it up + +```ts +import { chat } from '@tanstack/ai' +import { openaiText } from '@tanstack/ai-openai' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { inMemory } from '@tanstack/ai-memory/in-memory' + +const memory = inMemory() // dev/tests only — see the in-memory skill + +const stream = chat({ + adapter: openaiText('gpt-5.5'), + messages, + context: { session }, // attached by your auth middleware + middleware: [ + memoryMiddleware({ + adapter: memory, + // Derive scope server-side from trusted session state. + scope: (ctx) => { + const session = getSession(ctx) + return { sessionId: session.threadId, userId: session.userId } + }, + }), + ], +}) +``` + +`memoryMiddleware` options: `adapter`, `scope` (static or a function of `ctx`), +`role` (`'recall+save'` default, or `'save-only'`), and `onRecall` / `onSave` telemetry +callbacks. + +## The contract + +```ts +interface MemoryAdapter { + id: string + recall(scope, query): Promise // { systemPrompt, fragments?, tools?, toolGuidance? } + save(scope, turn): Promise> // turn = { user, assistant }; extraction lives HERE + inspect?(scope): Promise // optional (devtools) + listFacts?(scope): Promise> // optional (devtools) +} +``` + +- `recall` decides relevance and renders a `systemPrompt`; it may also return `tools` + + `toolGuidance` to hand the model direct control of memory (hindsight does this). +- `save` owns extraction — turning the raw turn into whatever gets persisted. + +## Scope security + +`MemoryScope` is `{ sessionId, userId? }` and is the isolation boundary. **Never trust a +client-supplied `userId`/`sessionId`.** Resolve scope server-side from session/auth and +pass the validated session through `chat({ context: { session } })`. If you accept a +thread id from the request body, validate it belongs to the session user BEFORE using it. + +## Adapters + +- `inMemory()` from `@tanstack/ai-memory/in-memory` — dev, tests, single-process demos. +- `redis({ redis })` from `@tanstack/ai-memory/redis` — production, plain Redis. +- `hindsight()` / `mem0()` / `honcho()` — hosted memory services (optional peer SDKs). +- Custom — implement `recall`/`save` and run `@tanstack/ai-memory/tests/contract`. + +## Failure modes + +Memory failures are non-fatal: a throwing `recall` or `save` emits `memory:error` and +the run continues with degraded memory. Streaming is never blocked; a failed save never +fails the turn. + +## Devtools + +Five events on `aiEventClient` (from `@tanstack/ai-event-client`): +`memory:retrieve:started` / `:completed`, `memory:persist:started` / `:completed`, +`memory:error` (`phase: 'recall' | 'save'`). Payloads carry the adapter id and +fragment/receipt counts, not full memory text. diff --git a/packages/ai-memory/src/index.ts b/packages/ai-memory/src/index.ts new file mode 100644 index 000000000..992b73f17 --- /dev/null +++ b/packages/ai-memory/src/index.ts @@ -0,0 +1,20 @@ +export { + memoryMiddleware, + MEMORY_STATE_EVENT, + type MemoryMiddlewareOptions, + type MemoryMiddlewareRole, + type MemoryRecallInfo, + type MemorySaveInfo, + type MemoryStateEventValue, +} from './middleware' + +export type { + MemoryAdapter, + MemoryScope, + MemoryTurn, + MemoryFragment, + RecallResult, + SaveReceipt, + MemorySnapshot, + MemoryFact, +} from './types' diff --git a/packages/ai-memory/src/internal/store.ts b/packages/ai-memory/src/internal/store.ts new file mode 100644 index 000000000..013e4cb06 --- /dev/null +++ b/packages/ai-memory/src/internal/store.ts @@ -0,0 +1,368 @@ +/** + * Shared internals for the built-in `inMemory()` and `redis()` adapters. + * + * NOT part of the public contract — nothing here is exported from the package + * root. Both built-in adapters keep a set of scored, optionally-embedded + * `MemoryRecord`s and expose only `recall`/`save`; this module holds the record + * model, the scoring/rendering helpers, and the extract→store→score→render + * pipeline they share. The only thing an adapter supplies is a {@link RecordStore} + * (a Map for in-memory, Redis keys for redis). + */ + +import type { + MemoryFact, + MemoryFragment, + MemoryScope, + MemorySnapshot, + MemoryTurn, + RecallResult, + SaveReceipt, +} from '../types' + +export type MemoryKind = 'message' | 'summary' | 'fact' | 'preference' +export type MemoryRole = 'user' | 'assistant' + +/** Internal stored record. Never crosses the public boundary. */ +export interface MemoryRecord { + id: string + scope: MemoryScope + text: string + kind: MemoryKind + role?: MemoryRole + createdAt: number + updatedAt?: number + expiresAt?: number + importance?: number + embedding?: Array + metadata?: Record +} + +/** Pluggable extractor: turn a completed turn into extra records to persist. */ +export type ExtractFn = ( + turn: MemoryTurn, + scope: MemoryScope, +) => + | Promise | undefined> + | Array + | undefined + +export interface ExtractedFact { + text: string + kind?: MemoryKind + importance?: number + metadata?: Record +} + +export interface Embedder { + embed: (text: string) => Promise> +} + +/** Options common to the built-in adapters. */ +export interface BuiltinOptions { + /** Max hits returned by recall. Defaults to 6. */ + topK?: number + /** Drop hits scoring below this. Defaults to 0.15. */ + minScore?: number + /** Restrict recall to these kinds. Defaults to all. */ + kinds?: Array + /** Optional embedder for semantic scoring on both save and recall. */ + embedder?: Embedder + /** Optional extractor run on `save` to persist derived facts/preferences. */ + extract?: ExtractFn + /** Replace the built-in prompt renderer. */ + render?: (hits: Array) => string +} + +export interface MemoryHit { + record: MemoryRecord + score: number +} + +/** + * Minimal storage backend the built-in adapters run on. `add` upserts by id; + * `loadScope` returns the live (non-expired) records for exactly this scope. + */ +export interface RecordStore { + add: (records: Array) => Promise + loadScope: (scope: MemoryScope) => Promise> +} + +// =========================== +// Scope +// =========================== + +/** + * Exact scope match. In the recall/save model the scope is always fully + * specified at both write and read (same middleware, same resolver), so a + * record is in-scope iff its `sessionId` matches and — when the query carries a + * `userId` — its `userId` matches too. + */ +export function sameScope(record: MemoryScope, query: MemoryScope): boolean { + if (record.sessionId !== query.sessionId) return false + if (query.userId != null && query.userId !== '') { + return record.userId === query.userId + } + return true +} + +// =========================== +// Scoring helpers +// =========================== + +const DEFAULT_HALF_LIFE_MS = 1000 * 60 * 60 * 24 * 30 // 30 days + +export function cosine(a?: Array, b?: Array): number { + if (!a || !b || a.length !== b.length || a.length === 0) return 0 + let dot = 0 + let aMag = 0 + let bMag = 0 + for (let i = 0; i < a.length; i++) { + const av = a[i] as number + const bv = b[i] as number + dot += av * bv + aMag += av ** 2 + bMag += bv ** 2 + } + if (aMag === 0 || bMag === 0) return 0 + return dot / (Math.sqrt(aMag) * Math.sqrt(bMag)) +} + +export function lexicalOverlap(query: string, text: string): number { + const queryTokens = new Set(query.toLowerCase().split(/\W+/).filter(Boolean)) + if (queryTokens.size === 0) return 0 + const textTokens = new Set(text.toLowerCase().split(/\W+/).filter(Boolean)) + let overlap = 0 + for (const token of queryTokens) { + if (textTokens.has(token)) overlap++ + } + return overlap / queryTokens.size +} + +export function recencyScore( + createdAt: number, + halfLifeMs: number = DEFAULT_HALF_LIFE_MS, + now: number = Date.now(), +): number { + const age = Math.max(0, now - createdAt) + return Math.pow(0.5, age / halfLifeMs) +} + +export function isExpired( + record: MemoryRecord, + now: number = Date.now(), +): boolean { + return record.expiresAt !== undefined && record.expiresAt < now +} + +/** + * Reference ranking: weighted sum of semantic (0.55), lexical (0.20), recency + * (0.15), and importance (0.10). Unset importance contributes 0 — no mid-range + * fallback, so recent records don't automatically clear the `minScore` floor. + */ +export function defaultScoreHit(args: { + record: MemoryRecord + queryText: string + queryEmbedding?: Array + now?: number +}): number { + const { record, queryText, queryEmbedding, now } = args + const semantic = cosine(queryEmbedding, record.embedding) + const lexical = lexicalOverlap(queryText, record.text) + const recency = recencyScore(record.createdAt, undefined, now) + const importance = record.importance ?? 0 + return semantic * 0.55 + lexical * 0.2 + recency * 0.15 + importance * 0.1 +} + +export function defaultRenderMemory(hits: Array): string { + if (hits.length === 0) return '' + return [ + 'Relevant memory:', + 'Use this information only when it is relevant to the current user request.', + 'Do not mention memory directly unless the user asks about it.', + 'If current conversation context contradicts memory, prefer the current conversation.', + '', + // JSON.stringify the text so persisted content with newlines or + // instruction-shaped text can't break out of the list and steer the turn. + ...hits.map( + (hit, index) => + `${index + 1}. [${hit.record.kind}] ${JSON.stringify(hit.record.text)}`, + ), + ].join('\n') +} + +// =========================== +// Shared recall / save pipeline +// =========================== + +/** Portable record id — real UUID where available, deterministic fallback otherwise. */ +export function newRecordId(): string { + try { + return crypto.randomUUID() + } catch { + return `mem-${Date.now()}-${Math.random().toString(36).slice(2, 10)}` + } +} + +/** + * Build the records for a completed turn: the raw user/assistant messages + * (importance 0.4) plus anything the optional extractor returns, embedding each + * when an embedder is configured. + */ +export async function buildTurnRecords( + scope: MemoryScope, + turn: MemoryTurn, + options: BuiltinOptions, +): Promise> { + const now = Date.now() + const records: Array = [] + + async function embed(text: string): Promise | undefined> { + if (!options.embedder) return undefined + return options.embedder.embed(text) + } + + if (turn.user) { + records.push({ + id: newRecordId(), + scope, + text: turn.user, + kind: 'message', + role: 'user', + createdAt: now, + importance: 0.4, + embedding: await embed(turn.user), + }) + } + if (turn.assistant) { + records.push({ + id: newRecordId(), + scope, + text: turn.assistant, + kind: 'message', + role: 'assistant', + createdAt: now, + importance: 0.4, + embedding: await embed(turn.assistant), + }) + } + + const extracted = await options.extract?.(turn, scope) + if (extracted) { + for (const fact of extracted) { + records.push({ + id: newRecordId(), + scope, + text: fact.text, + kind: fact.kind ?? 'fact', + createdAt: now, + importance: fact.importance, + embedding: await embed(fact.text), + metadata: fact.metadata, + }) + } + } + return records +} + +/** Persist a turn to the store and return one receipt for the batch. */ +export async function saveTurn( + store: RecordStore, + scope: MemoryScope, + turn: MemoryTurn, + options: BuiltinOptions, +): Promise> { + const startedAt = Date.now() + try { + const records = await buildTurnRecords(scope, turn, options) + if (records.length > 0) await store.add(records) + return [ + { + ok: true, + latencyMs: Date.now() - startedAt, + raw: { addedIds: records.map((r) => r.id) }, + }, + ] + } catch (error) { + return [ + { + ok: false, + latencyMs: Date.now() - startedAt, + error: error instanceof Error ? error.message : String(error), + }, + ] + } +} + +/** Score the scoped records against the query and render a recall result. */ +export async function recallRecords( + store: RecordStore, + scope: MemoryScope, + query: string, + options: BuiltinOptions, +): Promise { + const topK = options.topK ?? 6 + const minScore = options.minScore ?? 0.15 + const now = Date.now() + + const queryEmbedding = options.embedder + ? await options.embedder.embed(query) + : undefined + + const records = await store.loadScope(scope) + const kinds = options.kinds + const candidates = + kinds && kinds.length > 0 + ? records.filter((r) => kinds.includes(r.kind)) + : records + + const hits = candidates + .map((record) => ({ + record, + score: defaultScoreHit({ record, queryText: query, queryEmbedding, now }), + })) + .filter((h) => h.score >= minScore) + .sort((a, b) => b.score - a.score) + .slice(0, topK) + + const systemPrompt = (options.render ?? defaultRenderMemory)(hits) + const fragments: Array = hits.map((h) => ({ + text: h.record.text, + source: h.record.id, + })) + return { systemPrompt, fragments } +} + +/** Devtools inspect over a scope's live records. */ +export async function inspectRecords( + store: RecordStore, + scope: MemoryScope, +): Promise { + const records = await store.loadScope(scope) + return { + takenAt: new Date().toISOString(), + data: { + records: records.map((r) => ({ + id: r.id, + text: r.text, + kind: r.kind, + role: r.role, + createdAt: r.createdAt, + importance: r.importance, + })), + }, + } +} + +/** Devtools flat fact list over a scope's live records. */ +export async function listRecordFacts( + store: RecordStore, + scope: MemoryScope, +): Promise> { + const records = await store.loadScope(scope) + return records.map((r) => ({ + id: r.id, + text: r.text, + source: r.role ?? r.kind, + createdAt: new Date(r.createdAt).toISOString(), + })) +} diff --git a/packages/ai-memory/src/middleware.ts b/packages/ai-memory/src/middleware.ts new file mode 100644 index 000000000..ea6cb12eb --- /dev/null +++ b/packages/ai-memory/src/middleware.ts @@ -0,0 +1,390 @@ +import { aiEventClient } from '@tanstack/ai-event-client' +import type { + ChatMiddleware, + ChatMiddlewareConfig, + ChatMiddlewareContext, + ModelMessage, + StreamChunk, +} from '@tanstack/ai' +import type { + MemoryAdapter, + MemoryFact, + MemoryScope, + MemoryTurn, + RecallResult, + SaveReceipt, +} from './types' + +/** + * CUSTOM stream-event name carrying server-side memory state to the browser. + * The middleware injects one of these per turn (via `onChunk`); the client + * devtools bridge (`@tanstack/ai-client`) recognizes it and re-emits `memory:*` + * on the browser event bus. This is how server-side memory reaches the browser + * DevTools panel — server-emitted `aiEventClient` events never cross runtimes; + * everything the panel shows is re-derived client-side from the chat stream + * (mirrors how generation results ride `CUSTOM` events — see `GENERATION_EVENTS`). + */ +export const MEMORY_STATE_EVENT = 'memory:state' + +/** Payload of the {@link MEMORY_STATE_EVENT} CUSTOM chunk. Captures memory state + * as of the turn's START — the snapshot reflects every prior turn's save; this + * turn's own save (deferred) surfaces in the next turn's snapshot. */ +export interface MemoryStateEventValue { + scope: MemoryScope + adapter: string + /** The recall query (last user text). */ + query: string + /** Recall metrics for the operations timeline. */ + recall: { + fragmentCount: number + hasTools: boolean + systemPromptChars: number + durationMs: number + } + /** Live store snapshot, when the adapter supports `inspect`/`listFacts`. */ + snapshot?: { + takenAt: string + data: unknown + facts: Array + } +} + +/** + * How the middleware participates in the run: + * - `'recall+save'` (default): recall on init (inject prompt + tools), save on finish. + * - `'save-only'`: skip recall entirely — persist the turn but never read/inject. + */ +export type MemoryMiddlewareRole = 'recall+save' | 'save-only' + +export interface MemoryRecallInfo { + scope: MemoryScope + query: string + result: RecallResult +} + +export interface MemorySaveInfo { + scope: MemoryScope + turn: MemoryTurn + receipts: Array +} + +export interface MemoryMiddlewareOptions { + /** The memory backend to recall from / save to. */ + adapter: MemoryAdapter + /** + * Scope for every adapter call. The function form is the safer default for + * multi-tenant apps: derive scope per request from trusted, server-validated + * chat context — never from client input. + */ + scope: + | MemoryScope + | ((ctx: ChatMiddlewareContext) => MemoryScope | Promise) + /** Participation role. Defaults to `'recall+save'`. */ + role?: MemoryMiddlewareRole + /** Fired after `recall` completes (post-injection), for app telemetry. */ + onRecall?: (info: MemoryRecallInfo) => void | Promise + /** Fired after the deferred `save` completes, for app telemetry. */ + onSave?: (info: MemorySaveInfo) => void | Promise +} + +/** Per-request scratch state, keyed by context in a module-level WeakMap so the + * same middleware instance is safe across concurrent `chat()` calls. */ +interface MemoryRequestState { + resolvedScope?: MemoryScope + lastUserText: string + /** Pending devtools transport chunk, injected once by the first `onChunk`. */ + stateChunk?: { emitted: boolean; value: MemoryStateEventValue } +} + +const stateByCtx = new WeakMap() + +/** + * Server-side memory middleware. Recalls relevant memory into the prompt before + * the model runs, then defers `save` of the completed turn after it finishes. + * All extraction/ranking/rendering lives in the adapter — this middleware only + * wires `recall`/`save` into the chat lifecycle and emits devtools events. + */ +export function memoryMiddleware( + options: MemoryMiddlewareOptions, +): ChatMiddleware { + const role = options.role ?? 'recall+save' + + async function resolveScope( + ctx: ChatMiddlewareContext, + state: MemoryRequestState, + ): Promise { + if (state.resolvedScope) return state.resolvedScope + state.resolvedScope = + typeof options.scope === 'function' + ? await options.scope(ctx) + : options.scope + return state.resolvedScope + } + + return { + name: `memory:${options.adapter.id}`, + + async onConfig(ctx, config) { + if (ctx.phase !== 'init') return + + const state: MemoryRequestState = { lastUserText: '' } + stateByCtx.set(ctx, state) + + state.lastUserText = getMessageText(findLastUserMessage(config.messages)) + if (!state.lastUserText || role === 'save-only') return + + const startedAt = Date.now() + let scope: MemoryScope + let result: RecallResult + try { + scope = await resolveScope(ctx, state) + safeEmit('memory:retrieve:started', { + scope, + adapter: options.adapter.id, + query: state.lastUserText, + timestamp: startedAt, + }) + result = await options.adapter.recall(scope, state.lastUserText) + } catch (error) { + const errScope = state.resolvedScope ?? emptyScope() + safeEmit('memory:error', { + scope: errScope, + adapter: options.adapter.id, + phase: 'recall', + error: errorInfo(error), + timestamp: Date.now(), + }) + return + } + + const tools = result.tools ?? [] + const recallMetrics = { + fragmentCount: result.fragments?.length ?? 0, + hasTools: tools.length > 0, + systemPromptChars: result.systemPrompt.length, + durationMs: Date.now() - startedAt, + } + safeEmit('memory:retrieve:completed', { + scope, + adapter: options.adapter.id, + ...recallMetrics, + timestamp: Date.now(), + }) + await options.onRecall?.({ scope, query: state.lastUserText, result }) + + // Stage the devtools transport chunk (recall metrics + current store + // snapshot). Injected into the stream by `onChunk` so it reaches the + // browser panel; see MEMORY_STATE_EVENT. + const snapshot = await gatherSnapshot(options.adapter, scope) + state.stateChunk = { + emitted: false, + value: { + scope, + adapter: options.adapter.id, + query: state.lastUserText, + recall: recallMetrics, + ...(snapshot ? { snapshot } : {}), + }, + } + + const memoryPrompts = [result.toolGuidance ?? '', result.systemPrompt] + const additions = memoryPrompts.filter((p) => p.length > 0) + if (additions.length === 0 && tools.length === 0) return + + return { + systemPrompts: [...config.systemPrompts, ...additions], + tools: [...config.tools, ...tools], + } satisfies Partial + }, + + onChunk(ctx, chunk) { + // Inject the staged memory-state chunk exactly once, riding alongside the + // first stream chunk (typically RUN_STARTED) so the browser devtools sees + // it. Returning an array expands the stream; see ChatMiddleware.onChunk. + const state = stateByCtx.get(ctx) + if (!state?.stateChunk || state.stateChunk.emitted) return + state.stateChunk.emitted = true + const custom: StreamChunk = { + type: 'CUSTOM', + name: MEMORY_STATE_EVENT, + value: state.stateChunk.value, + timestamp: Date.now(), + } + return [chunk, custom] + }, + + onFinish(ctx, info) { + const state = stateByCtx.get(ctx) + stateByCtx.delete(ctx) + const userText = + state?.lastUserText || getMessageText(findLastUserMessage(ctx.messages)) + const assistant = info.content + if (!userText || !assistant) return + const scope = state?.resolvedScope + + ctx.defer( + (async () => { + // Resolve scope defensively — a throwing resolver must not escape the + // terminal hook. Memory failures are always non-fatal + observable. + let resolved: MemoryScope + try { + resolved = + scope ?? (await resolveScope(ctx, { lastUserText: userText })) + } catch (error) { + safeEmit('memory:error', { + scope: emptyScope(), + adapter: options.adapter.id, + phase: 'save', + error: errorInfo(error), + timestamp: Date.now(), + }) + return + } + + const turn: MemoryTurn = { user: userText, assistant } + const startedAt = Date.now() + safeEmit('memory:persist:started', { + scope: resolved, + adapter: options.adapter.id, + timestamp: startedAt, + }) + let receipts: Array + try { + receipts = await options.adapter.save(resolved, turn) + } catch (error) { + receipts = [{ ok: false, error: String(error) }] + safeEmit('memory:error', { + scope: resolved, + adapter: options.adapter.id, + phase: 'save', + error: errorInfo(error), + timestamp: Date.now(), + }) + } + safeEmit('memory:persist:completed', { + scope: resolved, + adapter: options.adapter.id, + receiptCount: receipts.length, + okCount: receipts.filter((r) => r.ok).length, + durationMs: Date.now() - startedAt, + timestamp: Date.now(), + }) + await emitSnapshot(options.adapter, resolved) + await options.onSave?.({ scope: resolved, turn, receipts }) + })(), + ) + }, + } +} + +// =========================== +// Internals +// =========================== + +function emptyScope(): MemoryScope { + return { sessionId: '' } +} + +/** + * Read the adapter's current stored state via the optional `inspect`/`listFacts` + * introspection methods. Returns `undefined` for adapters that don't implement + * `inspect` (they degrade to the metrics-only timeline). Fully guarded: + * introspection must never affect chat. + */ +async function gatherSnapshot( + adapter: MemoryAdapter, + scope: MemoryScope, +): Promise< + { takenAt: string; data: unknown; facts: Array } | undefined +> { + if (!adapter.inspect) return undefined + try { + const snapshot = await adapter.inspect(scope) + const facts = (await adapter.listFacts?.(scope)) ?? [] + return { takenAt: snapshot.takenAt, data: snapshot.data, facts } + } catch { + // ignored — introspection is best-effort telemetry. + return undefined + } +} + +/** + * DevTools-only: after a save, emit the adapter's current stored state on the + * (in-process) event bus, so a devtools consumer running in the SAME runtime as + * the chat (client-side execution / server-side listener) sees "what's in + * memory". For the standard server-side topology, the browser panel instead + * gets state via the {@link MEMORY_STATE_EVENT} stream chunk (see `onChunk`). + */ +async function emitSnapshot( + adapter: MemoryAdapter, + scope: MemoryScope, +): Promise { + const snapshot = await gatherSnapshot(adapter, scope) + if (!snapshot) return + safeEmit('memory:snapshot', { + scope, + adapter: adapter.id, + ...snapshot, + timestamp: Date.now(), + }) +} + +function findLastUserMessage( + messages: ReadonlyArray, +): ModelMessage | undefined { + for (let i = messages.length - 1; i >= 0; i--) { + const message = messages[i] + if (message && message.role === 'user') return message + } + return undefined +} + +/** + * Extract plain text from a `ModelMessage`. Text lives on `part.content` for + * `TextPart`; bare strings in the content array are tolerated. All other + * content kinds (tool-call, image, …) yield '' so they don't pollute the + * recall query. + */ +function getMessageText(message?: ModelMessage): string { + if (!message) return '' + if (typeof message.content === 'string') return message.content + if (Array.isArray(message.content)) { + return message.content + .map((part) => { + if (typeof part === 'string') return part + if (part.type === 'text' && typeof part.content === 'string') { + return part.content + } + return '' + }) + .filter(Boolean) + .join('\n') + } + return '' +} + +function errorInfo(error: unknown): { name: string; message: string } { + if (error instanceof Error) + return { name: error.name, message: error.message } + if ( + error && + typeof error === 'object' && + 'name' in error && + typeof error.name === 'string' + ) { + return { + name: error.name, + message: String((error as { message?: unknown }).message ?? error), + } + } + return { name: 'Error', message: String(error) } +} + +/** Fire-and-forget devtools emit — telemetry failures must never affect chat. */ +function safeEmit(...args: Parameters): void { + try { + aiEventClient.emit(...args) + } catch { + // ignored — telemetry must not affect chat behaviour + } +} diff --git a/packages/ai-memory/src/providers/hindsight/index.ts b/packages/ai-memory/src/providers/hindsight/index.ts new file mode 100644 index 000000000..91b9d466f --- /dev/null +++ b/packages/ai-memory/src/providers/hindsight/index.ts @@ -0,0 +1,232 @@ +/** + * Hindsight memory adapter. Hindsight owns extraction/ranking server-side and + * buckets memory into per-conversation "banks" (`{userId}__{sessionId}`). Recall + * returns a rendered prompt block AND a set of LLM tools (retain/recall/reflect) + * that let the model take direct control of memory. + * + * `@vectorize-io/hindsight-client` is an OPTIONAL peer dependency, loaded lazily. + */ + +import { makeHindsightTools } from './tools' +import type { + MemoryAdapter, + MemoryFact, + MemoryFragment, + MemoryScope, + MemorySnapshot, + MemoryTurn, + RecallResult, + SaveReceipt, +} from '../../types' + +/** Recall payload shape (the subset this adapter reads). */ +export interface HindsightRecallResponse { + results?: Array<{ text: string; type?: string; id: string }> +} + +/** + * Structural view of the hindsight client — only the methods this adapter uses. + * Decouples the adapter from the SDK's exact type surface. + */ +export interface HindsightClientLike { + retain: ( + bankId: string, + text: string, + opts: { context: string; timestamp: Date }, + ) => Promise + recall: ( + bankId: string, + query: string, + opts: { budget: string }, + ) => Promise + reflect: (bankId: string, query: string) => Promise<{ text?: string }> + listMemories: ( + bankId: string, + opts: { limit: number }, + ) => Promise<{ items?: Array> }> + getBankProfile: (bankId: string) => Promise + deleteBank: (bankId: string) => Promise +} + +export interface HindsightRuntime { + client: HindsightClientLike + recallToPrompt: (data: unknown) => string +} + +export interface HindsightOptions { + /** Durable user id used in the bank key. Falls back to `scope.userId`, then `'demo-user'`. */ + user?: string + /** Hindsight server URL. Defaults to `HINDSIGHT_URL` or `http://localhost:8888`. */ + baseUrl?: string + /** Recall budget. Defaults to `'mid'`. */ + budget?: 'low' | 'mid' | 'high' + /** Fired when a `hindsight_retain` tool call completes. */ + onToolRetain?: (receipt: SaveReceipt) => void + /** Fired when a `hindsight_recall` tool call completes. */ + onToolRecall?: (query: string, result: RecallResult) => void +} + +const TOOL_GUIDANCE = `You have access to persistent long-term memory that survives across sessions. + +Relevant memories for this turn have already been recalled and included in +your context. You also have three tools for direct control over memory: + +- hindsight_retain(content): explicitly store a fact, decision, or piece of + context you want to ensure is remembered in future sessions. + +- hindsight_recall(query): query memory directly with a specific question, + to look up a different topic than the user's last message. + +- hindsight_reflect(question): synthesize across many memories to answer + questions that require reasoning over accumulated knowledge. + +Prefer to use these tools when they would meaningfully improve your response. +You do not need to call them on every turn.` + +export function hindsight(options: HindsightOptions = {}): MemoryAdapter { + const budget = options.budget ?? 'mid' + let runtimePromise: Promise | null = null + + function getRuntime(): Promise { + if (!runtimePromise) { + runtimePromise = (async () => { + const mod = await import('@vectorize-io/hindsight-client') + const baseUrl = + options.baseUrl ?? + process.env.HINDSIGHT_URL ?? + 'http://localhost:8888' + // oxlint-disable-next-line eslint-js/no-restricted-syntax -- intentionally decoupled from the SDK's exact client type; the adapter only uses the HindsightClientLike subset + const client = new mod.HindsightClient({ + baseUrl, + }) as unknown as HindsightClientLike + const recallToPrompt = mod.recallResponseToPromptString as ( + data: unknown, + ) => string + return { client, recallToPrompt } + })().catch((err) => { + runtimePromise = null + throw err + }) + } + return runtimePromise + } + + function bankId(scope: MemoryScope): string { + const user = options.user ?? scope.userId ?? 'demo-user' + return `${user}__${scope.sessionId}` + } + + return { + id: 'hindsight', + + async save(scope, turn: MemoryTurn): Promise> { + const bank = bankId(scope) + const timestamp = new Date() + async function retain( + text: string, + context: string, + ): Promise { + const start = Date.now() + try { + const { client } = await getRuntime() + const data = await client.retain(bank, text, { context, timestamp }) + return { ok: true, latencyMs: Date.now() - start, raw: data } + } catch (err) { + return { + ok: false, + latencyMs: Date.now() - start, + error: err instanceof Error ? err.message : String(err), + } + } + } + return Promise.all([ + retain(turn.user, 'chat:user'), + retain(turn.assistant, 'chat:assistant'), + ]) + }, + + async recall(scope, query): Promise { + const bank = bankId(scope) + const tools = makeHindsightTools({ + getRuntime, + bankId: bank, + budget, + onToolRetain: options.onToolRetain, + onToolRecall: options.onToolRecall, + }) + try { + const { client, recallToPrompt } = await getRuntime() + const data = await client.recall(bank, query, { budget }) + const fragments: Array = (data.results ?? []).map( + (r) => ({ + text: r.text, + source: r.type ?? r.id, + }), + ) + return { + systemPrompt: recallToPrompt(data), + fragments, + tools, + toolGuidance: TOOL_GUIDANCE, + raw: data, + } + } catch (err) { + return { + systemPrompt: '', + fragments: [], + tools, + toolGuidance: TOOL_GUIDANCE, + raw: { error: err instanceof Error ? err.message : String(err) }, + } + } + }, + + async inspect(scope): Promise { + const bank = bankId(scope) + try { + const { client } = await getRuntime() + const [memories, profile] = await Promise.all([ + client.listMemories(bank, { limit: 200 }), + client.getBankProfile(bank), + ]) + return { + takenAt: new Date().toISOString(), + data: { memories, profile }, + } + } catch (err) { + return { + takenAt: new Date().toISOString(), + data: { error: err instanceof Error ? err.message : String(err) }, + } + } + }, + + async listFacts(scope): Promise> { + const bank = bankId(scope) + try { + const { client } = await getRuntime() + const res = await client.listMemories(bank, { limit: 200 }) + const items = res.items ?? [] + return items + .map((m, i): MemoryFact | null => { + const text = + (typeof m.text === 'string' ? m.text : undefined) ?? + (typeof m.content === 'string' ? m.content : undefined) + if (!text) return null + return { + id: typeof m.id === 'string' ? m.id : `hindsight-${i}`, + text, + source: typeof m.context === 'string' ? m.context : 'memory', + createdAt: + typeof m.created_at === 'string' ? m.created_at : undefined, + } + }) + .filter((f): f is MemoryFact => f !== null) + } catch { + return [] + } + }, + } +} + +export { makeHindsightTools } from './tools' diff --git a/packages/ai-memory/src/providers/hindsight/tools.ts b/packages/ai-memory/src/providers/hindsight/tools.ts new file mode 100644 index 000000000..bac1179e4 --- /dev/null +++ b/packages/ai-memory/src/providers/hindsight/tools.ts @@ -0,0 +1,139 @@ +import type { Tool } from '@tanstack/ai' +import type { MemoryFragment, RecallResult, SaveReceipt } from '../../types' +import type { HindsightRuntime } from './index' + +export interface HindsightToolDeps { + getRuntime: () => Promise + bankId: string + budget: string + onToolRetain?: (receipt: SaveReceipt) => void + onToolRecall?: (query: string, result: RecallResult) => void +} + +function stringField(args: unknown, key: string): string { + if (args && typeof args === 'object' && key in args) { + const value = (args as Record)[key] + if (typeof value === 'string') return value + } + return '' +} + +/** + * Build the hindsight LLM tools (retain / recall / reflect). These let the model + * take direct control of long-term memory beyond the automatic recall/save the + * middleware performs. Returned in `RecallResult.tools` and merged into the run. + */ +export function makeHindsightTools(deps: HindsightToolDeps): Array { + const retainTool: Tool = { + name: 'hindsight_retain', + description: + 'Explicitly store a fact, decision, or piece of context to remember in future sessions. Call this when the user shares something important about themselves, their preferences, their work, or any detail that should persist beyond this conversation.', + inputSchema: { + type: 'object', + properties: { + content: { + type: 'string', + description: + 'The exact fact, decision, or piece of context to store. Write it as a self-contained statement that will still make sense out of conversation context.', + }, + }, + required: ['content'], + additionalProperties: false, + }, + async execute(args) { + const content = stringField(args, 'content') + const start = Date.now() + try { + const { client } = await deps.getRuntime() + const data = await client.retain(deps.bankId, content, { + context: 'chat:tool', + timestamp: new Date(), + }) + deps.onToolRetain?.({ + ok: true, + latencyMs: Date.now() - start, + raw: data, + }) + return { ok: true } + } catch (err) { + const error = err instanceof Error ? err.message : String(err) + deps.onToolRetain?.({ ok: false, latencyMs: Date.now() - start, error }) + return { ok: false, error } + } + }, + } + + const recallTool: Tool = { + name: 'hindsight_recall', + description: + "Query memory directly with a specific question. Use this when you need context that may not have surfaced in the automatic recall — for example, to look up a different topic than the user's last message, or to find facts about an entity mentioned in passing.", + inputSchema: { + type: 'object', + properties: { + query: { + type: 'string', + description: 'Natural-language question or topic to look up.', + }, + }, + required: ['query'], + additionalProperties: false, + }, + async execute(args) { + const query = stringField(args, 'query') + const start = Date.now() + try { + const { client, recallToPrompt } = await deps.getRuntime() + const data = await client.recall(deps.bankId, query, { + budget: deps.budget, + }) + const systemPrompt = recallToPrompt(data) + const fragments: Array = (data.results ?? []).map( + (r) => ({ + text: r.text, + source: r.type ?? r.id, + }), + ) + deps.onToolRecall?.(query, { + systemPrompt, + fragments, + latencyMs: Date.now() - start, + raw: data, + } as RecallResult) + return systemPrompt || '(no relevant memories found)' + } catch (err) { + const error = err instanceof Error ? err.message : String(err) + return `(no memory available: ${error})` + } + }, + } + + const reflectTool: Tool = { + name: 'hindsight_reflect', + description: + 'Synthesize across many memories to answer questions that require reasoning over accumulated knowledge, rather than retrieving specific facts. Use this for questions like "what do I know about this user\'s stack?" or "what has the user been working on lately?"', + inputSchema: { + type: 'object', + properties: { + query: { + type: 'string', + description: + 'The synthesis question to reflect on, e.g. "what do I know about the user\'s preferences?"', + }, + }, + required: ['query'], + additionalProperties: false, + }, + async execute(args) { + const query = stringField(args, 'query') + try { + const { client } = await deps.getRuntime() + const data = await client.reflect(deps.bankId, query) + return data.text ?? '(no reflection)' + } catch (err) { + return `(reflection failed: ${err instanceof Error ? err.message : String(err)})` + } + }, + } + + return [retainTool, recallTool, reflectTool] +} diff --git a/packages/ai-memory/src/providers/honcho/index.ts b/packages/ai-memory/src/providers/honcho/index.ts new file mode 100644 index 000000000..88cb1fd0e --- /dev/null +++ b/packages/ai-memory/src/providers/honcho/index.ts @@ -0,0 +1,223 @@ +/** + * Honcho memory adapter. Honcho models memory as peers exchanging messages in a + * session and answers recall via a "dialectic" query over the user peer's + * representation — so `recall` returns a synthesized answer (no discrete + * fragments) and `save` appends the turn's messages to the session. + * + * `@honcho-ai/sdk` is an OPTIONAL peer dependency, loaded lazily on first use. + */ + +import type { Peer, Session } from '@honcho-ai/sdk' +import type { + MemoryAdapter, + MemoryFact, + MemoryScope, + MemorySnapshot, + MemoryTurn, + RecallResult, + SaveReceipt, +} from '../../types' + +export interface HonchoOptions { + /** Durable user id. Falls back to `scope.userId`, then `'demo-user'`. */ + user?: string + /** Honcho server URL. Defaults to `HONCHO_URL` or `http://localhost:8001`. */ + baseURL?: string + /** Workspace id. Defaults to `HONCHO_APP_NAME` or `'ai-memory'`. */ + workspaceId?: string + /** API key. Defaults to `HONCHO_API_KEY` (or `'dev-no-auth'`). */ + apiKey?: string + /** Assistant peer id. Defaults to `'assistant'`. */ + assistantId?: string +} + +type Timed = + | { ok: true; latencyMs: number; data: T } + | { ok: false; latencyMs: number; error: string } + +async function timed(fn: () => Promise): Promise> { + const start = Date.now() + try { + const data = await fn() + return { ok: true, latencyMs: Date.now() - start, data } + } catch (err) { + return { + ok: false, + latencyMs: Date.now() - start, + error: err instanceof Error ? err.message : String(err), + } + } +} + +const HONCHO_LINE_RE = /^\[(?[^\]]+)\]\s+(?.+)$/ + +/** Parse a Honcho `peer.representation()` text blob into flat fact rows. */ +export function parseHonchoRepresentation(raw: string): Array { + return raw + .split('\n') + .map((line) => line.trim()) + .filter( + (line) => + line.length > 0 && + !line.startsWith('##') && + !line.startsWith('Explicit Observations'), + ) + .map((line, i): MemoryFact => { + const m = line.match(HONCHO_LINE_RE) + if (m?.groups?.ts && m.groups.text) { + return { + id: `honcho-${m.groups.ts}-${i}`, + text: m.groups.text, + source: 'observation', + createdAt: m.groups.ts, + } + } + return { id: `honcho-${i}`, text: line, source: 'representation' } + }) +} + +export function honcho(options: HonchoOptions = {}): MemoryAdapter { + const assistantId = options.assistantId ?? 'assistant' + + // Client + entity caches live in this factory's closure — each honcho() + // instance owns its own. + type Client = Awaited> + let clientPromise: Promise | null = null + const sessionCache = new Map>() + const userPeerCache = new Map>() + let assistantPeerPromise: Promise | null = null + + async function loadClient() { + const mod = await import('@honcho-ai/sdk') + return new mod.Honcho({ + baseURL: + options.baseURL ?? process.env.HONCHO_URL ?? 'http://localhost:8001', + workspaceId: + options.workspaceId ?? process.env.HONCHO_APP_NAME ?? 'ai-memory', + apiKey: options.apiKey ?? process.env.HONCHO_API_KEY ?? 'dev-no-auth', + }) + } + + function getClient(): Promise { + if (!clientPromise) clientPromise = loadClient() + return clientPromise + } + + function cached( + cache: Map>, + key: string, + create: () => Promise, + ): Promise { + const existing = cache.get(key) + if (existing) return existing + const created = create().catch((err) => { + if (cache.get(key) === created) cache.delete(key) + throw err + }) + cache.set(key, created) + return created + } + + function getUserPeer(userId: string): Promise { + return cached(userPeerCache, userId, async () => + (await getClient()).peer(userId), + ) + } + function getAssistantPeer(): Promise { + if (!assistantPeerPromise) { + assistantPeerPromise = (async () => + (await getClient()).peer(assistantId))().catch((err) => { + assistantPeerPromise = null + throw err + }) + } + return assistantPeerPromise + } + function getSession(sessionId: string): Promise { + return cached(sessionCache, sessionId, async () => + (await getClient()).session(sessionId), + ) + } + + function userIdFor(scope: MemoryScope): string { + return options.user ?? scope.userId ?? 'demo-user' + } + + return { + id: 'honcho', + + async save(scope, turn: MemoryTurn): Promise> { + const result = await timed(async () => { + const [userPeer, assistantPeer, session] = await Promise.all([ + getUserPeer(userIdFor(scope)), + getAssistantPeer(), + getSession(scope.sessionId), + ]) + return session.addMessages([ + userPeer.message(turn.user), + assistantPeer.message(turn.assistant), + ]) + }) + return [ + { + ok: result.ok, + latencyMs: result.latencyMs, + raw: result.ok ? result.data : undefined, + error: result.ok ? undefined : result.error, + }, + ] + }, + + async recall(scope, query): Promise { + const result = await timed(async () => { + const [userPeer, session] = await Promise.all([ + getUserPeer(userIdFor(scope)), + getSession(scope.sessionId), + ]) + return userPeer.chat(query, { session }) + }) + if (!result.ok) { + return { systemPrompt: '', raw: { error: result.error } } + } + const text = typeof result.data === 'string' ? result.data : '' + return { systemPrompt: text, raw: { dialectic: text } } + }, + + async inspect(scope): Promise { + const session = await getSession(scope.sessionId).catch(() => null) + if (!session) { + return { + takenAt: new Date().toISOString(), + data: { error: 'failed to get session' }, + } + } + const [messages, summaries] = await Promise.all([ + timed(() => session.messages({ size: 50 })), + timed(() => session.summaries()), + ]) + return { + takenAt: new Date().toISOString(), + data: { + messages: messages.ok ? messages.data : { error: messages.error }, + summaries: summaries.ok ? summaries.data : { error: summaries.error }, + }, + } + }, + + async listFacts(scope): Promise> { + const result = await timed(async () => { + const userPeer = await getUserPeer(userIdFor(scope)) + return userPeer.representation() + }) + if (!result.ok) return [] + const raw = + typeof result.data === 'string' + ? result.data + : String( + (result.data as { representation?: unknown }).representation ?? + '', + ) + return parseHonchoRepresentation(raw) + }, + } +} diff --git a/packages/ai-memory/src/providers/in-memory/index.ts b/packages/ai-memory/src/providers/in-memory/index.ts new file mode 100644 index 000000000..e9bb743a5 --- /dev/null +++ b/packages/ai-memory/src/providers/in-memory/index.ts @@ -0,0 +1,63 @@ +import { + inspectRecords, + isExpired, + listRecordFacts, + recallRecords, + sameScope, + saveTurn, +} from '../../internal/store' +import type { + BuiltinOptions, + MemoryRecord, + RecordStore, +} from '../../internal/store' +import type { MemoryAdapter, MemoryScope } from '../../types' + +/** + * Options for {@link inMemory}. Retrieval/extraction knobs that used to live on + * the middleware are adapter options here. + */ +export interface InMemoryOptions extends BuiltinOptions {} + +/** + * Zero-dependency memory adapter backed by a `Map`. Records vanish on process + * restart, so this is for local development, tests, and single-process demos — + * not multi-process production (each worker gets its own Map). For production, + * use {@link redis} from `@tanstack/ai-memory/redis`. + * + * By default `save` stores the raw user/assistant turn and `recall` scores it + * lexically + by recency. Pass an `embedder` for semantic scoring and/or an + * `extract` function to persist derived facts. + */ +export function inMemory(options: InMemoryOptions = {}): MemoryAdapter { + const records = new Map() + + function sweep(): Array { + const now = Date.now() + const live: Array = [] + for (const r of records.values()) { + if (isExpired(r, now)) records.delete(r.id) + else live.push(r) + } + return live + } + + const store: RecordStore = { + async add(batch) { + const now = Date.now() + for (const r of batch) records.set(r.id, { ...r, updatedAt: now }) + sweep() + }, + async loadScope(scope: MemoryScope) { + return sweep().filter((r) => sameScope(r.scope, scope)) + }, + } + + return { + id: 'in-memory', + recall: (scope, query) => recallRecords(store, scope, query, options), + save: (scope, turn) => saveTurn(store, scope, turn, options), + inspect: (scope) => inspectRecords(store, scope), + listFacts: (scope) => listRecordFacts(store, scope), + } +} diff --git a/packages/ai-memory/src/providers/mem0/index.ts b/packages/ai-memory/src/providers/mem0/index.ts new file mode 100644 index 000000000..f6aa3f1f2 --- /dev/null +++ b/packages/ai-memory/src/providers/mem0/index.ts @@ -0,0 +1,185 @@ +/** + * mem0 memory adapter — talks to a mem0 server over plain HTTP (no SDK, so no + * peer dependency). mem0 owns extraction and ranking server-side; this adapter + * maps the `recall`/`save` contract onto its `/memories` and `/search` endpoints. + * + * Requires a running mem0 server. Point it at one via `baseUrl` (or the + * `MEM0_URL` env var); pass `apiKey` (or `MEM0_ADMIN_API_KEY`) when it's secured. + */ + +import type { + MemoryAdapter, + MemoryFact, + MemoryFragment, + MemoryScope, + MemorySnapshot, + MemoryTurn, + RecallResult, + SaveReceipt, +} from '../../types' + +export interface Mem0Options { + /** Durable user id. Falls back to `scope.userId`, then `'demo-user'`. */ + user?: string + /** mem0 server URL. Defaults to `MEM0_URL` or `http://localhost:8000`. */ + baseUrl?: string + /** Bearer token. Defaults to `MEM0_ADMIN_API_KEY`. */ + apiKey?: string + /** Ask mem0 to rerank search results. Defaults to `true`. */ + rerank?: boolean + /** Minimum search score. Defaults to `0.1`. */ + threshold?: number +} + +type JsonResult = + | { ok: true; latencyMs: number; data: unknown } + | { ok: false; latencyMs: number; error: string } + +function asRecord(value: unknown): Record | undefined { + return value && typeof value === 'object' + ? (value as Record) + : undefined +} + +function asString(value: unknown): string | undefined { + return typeof value === 'string' ? value : undefined +} + +/** Pull the array of items out of a mem0 response (`{results: []}` or a bare array). */ +function itemsOf(data: unknown): Array> { + const rec = asRecord(data) + const candidate = rec && 'results' in rec ? rec.results : data + if (!Array.isArray(candidate)) return [] + return candidate.filter( + (m): m is Record => !!m && typeof m === 'object', + ) +} + +export function mem0(options: Mem0Options = {}): MemoryAdapter { + const baseUrl = + options.baseUrl ?? process.env.MEM0_URL ?? 'http://localhost:8000' + const apiKey = options.apiKey ?? process.env.MEM0_ADMIN_API_KEY ?? '' + const rerank = options.rerank ?? true + const threshold = options.threshold ?? 0.1 + + function headers(): Record { + const h: Record = { 'Content-Type': 'application/json' } + if (apiKey) h.Authorization = `Bearer ${apiKey}` + return h + } + + function userId(scope: MemoryScope): string { + return options.user ?? scope.userId ?? 'demo-user' + } + + async function safeJson(fn: () => Promise): Promise { + const start = Date.now() + try { + const res = await fn() + const latencyMs = Date.now() - start + if (!res.ok) { + const text = await res.text().catch(() => '') + return { + ok: false, + latencyMs, + error: `HTTP ${res.status}: ${text.slice(0, 300)}`, + } + } + const data = await res.json().catch(() => null) + return { ok: true, latencyMs, data } + } catch (err) { + return { + ok: false, + latencyMs: Date.now() - start, + error: err instanceof Error ? err.message : String(err), + } + } + } + + async function loadMemories(scope: MemoryScope): Promise { + const url = `${baseUrl}/memories?user_id=${encodeURIComponent(userId(scope))}` + return safeJson(() => fetch(url, { method: 'GET', headers: headers() })) + } + + return { + id: 'mem0', + + async save(scope, turn: MemoryTurn): Promise> { + const result = await safeJson(() => + fetch(`${baseUrl}/memories`, { + method: 'POST', + headers: headers(), + body: JSON.stringify({ + messages: [ + { role: 'user', content: turn.user }, + { role: 'assistant', content: turn.assistant }, + ], + user_id: userId(scope), + }), + }), + ) + return [ + { + ok: result.ok, + latencyMs: result.latencyMs, + raw: result.ok ? result.data : undefined, + error: result.ok ? undefined : result.error, + }, + ] + }, + + async recall(scope, query): Promise { + const result = await safeJson(() => + fetch(`${baseUrl}/search`, { + method: 'POST', + headers: headers(), + body: JSON.stringify({ + query, + user_id: userId(scope), + rerank, + threshold, + }), + }), + ) + if (!result.ok) { + return { systemPrompt: '', fragments: [], raw: { error: result.error } } + } + const fragments: Array = itemsOf(result.data).map( + (m) => ({ + text: asString(m.memory) ?? asString(m.text) ?? JSON.stringify(m), + source: asString(m.id) ?? 'mem0', + }), + ) + const systemPrompt = + fragments.length === 0 + ? '' + : `Recalled memory:\n${fragments.map((f) => `- (${f.source}) ${f.text}`).join('\n')}` + return { systemPrompt, fragments, raw: result.data } + }, + + async inspect(scope): Promise { + const result = await loadMemories(scope) + return { + takenAt: new Date().toISOString(), + data: result.ok ? result.data : { error: result.error }, + } + }, + + async listFacts(scope): Promise> { + const result = await loadMemories(scope) + if (!result.ok) return [] + return itemsOf(result.data) + .map((m): MemoryFact | null => { + const text = asString(m.memory) + if (!text) return null + return { + id: asString(m.id) ?? 'mem0', + text, + source: 'memory', + createdAt: asString(m.updated_at) ?? asString(m.created_at), + } + }) + .filter((f): f is MemoryFact => f !== null) + }, + } +} diff --git a/packages/ai-memory/src/providers/redis/index.ts b/packages/ai-memory/src/providers/redis/index.ts new file mode 100644 index 000000000..a3b0961dd --- /dev/null +++ b/packages/ai-memory/src/providers/redis/index.ts @@ -0,0 +1,171 @@ +import { + inspectRecords, + isExpired, + listRecordFacts, + recallRecords, + saveTurn, +} from '../../internal/store' +import type { + BuiltinOptions, + MemoryRecord, + RecordStore, +} from '../../internal/store' +import type { MemoryAdapter, MemoryScope } from '../../types' + +/** + * Minimal subset of the Redis client API the adapter uses. Shaped to match + * `ioredis` directly (lowercase method names). For node-redis v4+'s camelCase + * API, wrap the client with {@link fromNodeRedis}. + */ +export interface RedisLike { + set: (key: string, value: string) => Promise + get: (key: string) => Promise + del: (...keys: Array) => Promise + sadd: (key: string, ...members: Array) => Promise + srem: (key: string, ...members: Array) => Promise + smembers: (key: string) => Promise> + mget: (...keys: Array) => Promise> +} + +/** node-redis v4+ default-mode (camelCase) surface used by {@link fromNodeRedis}. */ +export interface NodeRedisLike { + get: (key: string) => Promise + set: (key: string, value: string) => Promise + del: (keys: Array | string) => Promise + sAdd: (key: string, members: string | Array) => Promise + sRem: (key: string, members: string | Array) => Promise + sMembers: (key: string) => Promise> + mGet: (keys: Array) => Promise> +} + +/** + * Wrap a node-redis v4+ default-mode client (camelCase API) into the lowercase + * {@link RedisLike} shape this adapter expects. For `ioredis`, no wrapper is + * needed — pass the client directly. + */ +export function fromNodeRedis(client: NodeRedisLike): RedisLike { + return { + get: (key) => client.get(key), + set: (key, value) => client.set(key, value), + del: (...keys) => client.del(keys), + sadd: (key, ...members) => client.sAdd(key, members), + srem: (key, ...members) => client.sRem(key, members), + smembers: (key) => client.sMembers(key), + mget: (...keys) => client.mGet(keys), + } +} + +export interface RedisOptions extends BuiltinOptions { + /** A Redis client implementing {@link RedisLike} (ioredis, or wrapped node-redis). */ + redis: RedisLike + /** Key prefix. Defaults to `'tanstack-ai:memory'`. */ + prefix?: string +} + +/** + * Escape the `:` scope-key delimiter (and the `\` escape character itself) in a + * scope value before composing the colon-joined key. Without this, a scope + * value containing `:` could shift segment positions and collide two different + * scopes' index buckets. `_` is escaped too so a literal `_` value can't collide + * with the unset-key placeholder. + */ +function escapeScopeValue(value: string): string { + return value.replace(/[\\:_]/g, '\\$&') +} + +// Track ids we've warned about so ongoing corruption of DIFFERENT ids keeps +// surfacing, bounded so a pathological store can't spam the console forever. +const warnedMalformedIds = new Set() +const MALFORMED_WARN_CAP = 100 +function warnMalformedRow(id: string, err: unknown): void { + if ( + warnedMalformedIds.has(id) || + warnedMalformedIds.size >= MALFORMED_WARN_CAP + ) { + return + } + warnedMalformedIds.add(id) + console.warn( + `[tanstack-ai-memory] redis: skipped malformed record JSON (id=${id}). ` + + `The row is left in place (not deleted) in case it is recoverable. ` + + `Reason: ${String(err)}`, + ) +} + +/** + * Production memory adapter backed by plain Redis (no vector index required). + * Ranks client-side (lexical + optional cosine + recency + importance), so it's + * suited to up to ~10k records per scope. Bring your own client (`ioredis`, or + * node-redis wrapped with {@link fromNodeRedis}). + * + * Storage model: + * ```text + * {prefix}:record:{id} -> JSON MemoryRecord + * {prefix}:index:{userId or _}:{sessionId} -> Set + * ``` + */ +export function redis(options: RedisOptions): MemoryAdapter { + const client = options.redis + const prefix = options.prefix ?? 'tanstack-ai:memory' + + const scopeKey = (scope: MemoryScope): string => + `${escapeScopeValue(scope.userId != null && scope.userId !== '' ? scope.userId : '_')}:${escapeScopeValue(scope.sessionId)}` + const indexKey = (scope: MemoryScope): string => + `${prefix}:index:${scopeKey(scope)}` + const recordKey = (id: string): string => `${prefix}:record:${id}` + + const store: RecordStore = { + async add(batch) { + const now = Date.now() + for (const r of batch) { + const next: MemoryRecord = { ...r, updatedAt: now } + await client.set(recordKey(r.id), JSON.stringify(next)) + await client.sadd(indexKey(r.scope), r.id) + } + }, + + async loadScope(scope: MemoryScope) { + const idx = indexKey(scope) + const ids = await client.smembers(idx) + if (ids.length === 0) return [] + const raws = await client.mget(...ids.map(recordKey)) + const out: Array = [] + const stale: Array = [] + for (let i = 0; i < raws.length; i++) { + const raw = raws[i] as string | null + const id = ids[i] as string + if (!raw) { + stale.push(id) + continue + } + let record: MemoryRecord + try { + record = JSON.parse(raw) as MemoryRecord + } catch (err) { + // Malformed JSON is skipped, NOT swept — a parse failure isn't proof + // the data is unrecoverable (truncated read, older schema, etc.). + warnMalformedRow(id, err) + continue + } + if (isExpired(record)) { + stale.push(id) + continue + } + out.push(record) + } + if (stale.length > 0) { + await client.srem(idx, ...stale) + await client.del(...stale.map(recordKey)) + } + return out + }, + } + + return { + id: 'redis', + recall: (scope, query) => recallRecords(store, scope, query, options), + save: (scope, turn) => saveTurn(store, scope, turn, options), + inspect: (scope) => inspectRecords(store, scope), + listFacts: (scope) => listRecordFacts(store, scope), + } +} diff --git a/packages/ai-memory/src/types.ts b/packages/ai-memory/src/types.ts new file mode 100644 index 000000000..39afd4002 --- /dev/null +++ b/packages/ai-memory/src/types.ts @@ -0,0 +1,160 @@ +/** + * Public contract for the TanStack AI memory subsystem. + * + * A memory backend implements ONE contract with two verbs: {@link MemoryAdapter.recall} + * and {@link MemoryAdapter.save}. This is deliberately the shape every real memory + * provider (mem0, honcho, hindsight, …) already exposes — "what's relevant for this + * query?" and "remember this turn". The middleware ({@link memoryMiddleware}) is thin: + * it calls `recall` before the model runs and defers `save` after the turn finishes. + * + * Adapters own everything else. Extraction (turning a turn into stored facts), + * ranking, rendering into a prompt, scope isolation, and expiry are all the + * adapter's responsibility — the middleware never inspects records. The built-in + * `inMemory()` / `redis()` adapters keep their store/scoring internals private + * behind `recall`/`save`; vendor adapters map these two verbs onto their APIs. + */ + +import type { Tool } from '@tanstack/ai' + +// =========================== +// Scope & turn primitives +// =========================== + +/** + * Isolation scope for memory reads and writes. Opaque to the middleware — + * each adapter interprets it (vendors map it to bank/user ids; the built-in + * stores key their internal record space by it). + * + * Derive scope server-side from trusted session state — never from client + * input, or one user's request can read or write another user's memory. + */ +export interface MemoryScope { + /** Conversation/session identifier. Required — the minimal isolation key. */ + sessionId: string + /** Optional durable end-user identity, for cross-session recall. */ + userId?: string +} + +/** A completed conversation turn handed to {@link MemoryAdapter.save}. */ +export interface MemoryTurn { + user: string + assistant: string +} + +// =========================== +// Recall +// =========================== + +/** A discrete recalled item, when the adapter produces them. */ +export interface MemoryFragment { + /** The recalled text. */ + text: string + /** Provenance hint (record id, vendor result type, etc.). */ + source: string +} + +/** + * Result of {@link MemoryAdapter.recall}. Everything the middleware needs to + * augment the run: a pre-rendered prompt block, optional discrete fragments, + * and optional tools the adapter wants exposed to the model this turn. + */ +export interface RecallResult { + /** + * Pre-rendered block to inject into the system prompt. An empty string means + * "nothing to inject" — the middleware skips it. + */ + systemPrompt: string + /** + * Discrete recalled items, when the adapter produces them. Omitted for + * engines that return synthesized output (e.g. honcho's dialectic answer). + */ + fragments?: Array + /** + * Tools the adapter wants exposed to the model for this turn (e.g. hindsight's + * retain/recall/reflect tools). Merged into the run's tool set by the + * middleware. Omit or `[]` when the adapter exposes no tools. + */ + tools?: Array + /** + * System-prompt text explaining when/how to use {@link RecallResult.tools}. + * Injected ahead of `systemPrompt`. Omit or `''` when there are no tools. + */ + toolGuidance?: string + /** Raw vendor payload, surfaced for devtools/inspection. */ + raw?: unknown +} + +// =========================== +// Save +// =========================== + +/** + * Receipt for a single underlying write performed by {@link MemoryAdapter.save}. + * One turn can produce several receipts (e.g. hindsight writes the user and + * assistant utterances separately), so `save` returns an array. + */ +export interface SaveReceipt { + ok: boolean + /** Optional adapter-reported write latency (ms), for devtools. */ + latencyMs?: number + /** Present when `ok` is `false`. */ + error?: string + /** Raw vendor payload, surfaced for devtools/inspection. */ + raw?: unknown +} + +// =========================== +// Optional introspection (devtools / admin panels) +// =========================== + +/** Full snapshot returned by the optional {@link MemoryAdapter.inspect}. */ +export interface MemorySnapshot { + /** ISO timestamp when the snapshot was taken. */ + takenAt: string + /** Adapter-defined snapshot payload. */ + data: unknown +} + +/** A flat fact row returned by the optional {@link MemoryAdapter.listFacts}. */ +export interface MemoryFact { + id: string + text: string + source?: string + /** ISO timestamp, when the adapter tracks creation time. */ + createdAt?: string +} + +// =========================== +// Adapter contract +// =========================== + +/** + * The single memory adapter contract. All backends — the built-in `inMemory()` + * and `redis()` adapters as well as vendor adapters (`hindsight()`, `mem0()`, + * `honcho()`) — implement `recall` + `save`. `inspect`/`listFacts` are optional + * and exist only for devtools/admin surfaces. + */ +export interface MemoryAdapter { + /** Stable id used in logs, devtools, and event payloads (e.g. 'in-memory', 'hindsight'). */ + readonly id: string + /** Optional human-readable label; defaults to {@link MemoryAdapter.id} in logs. */ + readonly name?: string + + /** + * Read side — retrieve what's relevant to `query` within `scope`. The ranking + * strategy (lexical, semantic, hybrid, vendor-native) is entirely the + * adapter's concern. + */ + recall: (scope: MemoryScope, query: string) => Promise + + /** + * Write side — persist a completed turn. Extraction (turn → stored facts) + * happens HERE, inside the adapter. Returns one receipt per underlying write. + */ + save: (scope: MemoryScope, turn: MemoryTurn) => Promise> + + /** Optional — full snapshot for a devtools inspection panel. */ + inspect?: (scope: MemoryScope) => Promise + /** Optional — flat fact list for a devtools panel. */ + listFacts?: (scope: MemoryScope) => Promise> +} diff --git a/packages/ai-memory/tests/contract.ts b/packages/ai-memory/tests/contract.ts new file mode 100644 index 000000000..7b409e633 --- /dev/null +++ b/packages/ai-memory/tests/contract.ts @@ -0,0 +1,85 @@ +import { beforeEach, describe, expect, it } from 'vitest' +import type { MemoryAdapter, MemoryScope } from '../src' + +/** + * Shared contract suite for any `recall`/`save` {@link MemoryAdapter}. Point it + * at a factory that returns a fresh adapter and it verifies the round-trip, + * scope isolation, empty recall, receipt shape, and the optional introspection + * methods. If your adapter passes, the middleware works. + */ +export function runMemoryAdapterContract( + label: string, + factory: () => Promise | MemoryAdapter, +) { + describe(label, () => { + let adapter: MemoryAdapter + const scopeA: MemoryScope = { sessionId: 's1', userId: 'u1' } + const scopeB: MemoryScope = { sessionId: 's2', userId: 'u2' } + + beforeEach(async () => { + adapter = await factory() + }) + + describe('save', () => { + it('returns a non-empty array of ok receipts', async () => { + const receipts = await adapter.save(scopeA, { + user: 'I love hiking in the mountains', + assistant: 'Noted — hiking it is.', + }) + expect(Array.isArray(receipts)).toBe(true) + expect(receipts.length).toBeGreaterThan(0) + expect(receipts.every((r) => typeof r.ok === 'boolean')).toBe(true) + expect(receipts.some((r) => r.ok)).toBe(true) + }) + }) + + describe('recall', () => { + it('round-trips: a saved turn surfaces in a later recall', async () => { + await adapter.save(scopeA, { + user: 'My favorite programming language is TypeScript', + assistant: 'Great choice.', + }) + const result = await adapter.recall(scopeA, 'programming language') + expect(result.systemPrompt.toLowerCase()).toContain('typescript') + }) + + it('returns an empty result for a scope with nothing saved', async () => { + const result = await adapter.recall(scopeA, 'anything at all') + expect(result.systemPrompt).toBe('') + expect(result.fragments ?? []).toHaveLength(0) + }) + + it('isolates scopes — recall never crosses into another scope', async () => { + await adapter.save(scopeA, { + user: 'The secret code is alpha-bravo', + assistant: 'Understood.', + }) + const other = await adapter.recall(scopeB, 'secret code') + expect(other.systemPrompt).toBe('') + expect(other.fragments ?? []).toHaveLength(0) + }) + }) + + describe('optional introspection', () => { + it('inspect (when present) returns a well-formed snapshot after a save', async () => { + if (!adapter.inspect) return + await adapter.save(scopeA, { user: 'hello world', assistant: 'hi' }) + const snap = await adapter.inspect(scopeA) + expect(typeof snap.takenAt).toBe('string') + expect(snap.data).toBeDefined() + }) + + it('listFacts (when present) returns rows after a save', async () => { + if (!adapter.listFacts) return + await adapter.save(scopeA, { user: 'hello world', assistant: 'hi' }) + const facts = await adapter.listFacts(scopeA) + expect(Array.isArray(facts)).toBe(true) + expect( + facts.every( + (f) => typeof f.id === 'string' && typeof f.text === 'string', + ), + ).toBe(true) + }) + }) + }) +} diff --git a/packages/ai-memory/tests/middleware.test.ts b/packages/ai-memory/tests/middleware.test.ts new file mode 100644 index 000000000..095e9339d --- /dev/null +++ b/packages/ai-memory/tests/middleware.test.ts @@ -0,0 +1,245 @@ +import { describe, expect, it, vi } from 'vitest' +import { aiEventClient } from '@tanstack/ai-event-client' +import { MEMORY_STATE_EVENT, memoryMiddleware } from '../src' +import type { StreamChunk } from '@tanstack/ai' +import type { + ChatMiddlewareConfig, + ChatMiddlewareContext, + FinishInfo, + Tool, +} from '@tanstack/ai' +import type { MemoryAdapter, MemoryScope, MemoryTurn } from '../src' + +const catTool: Tool = { + name: 'cat_tool', + description: 'A tool about cats.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, +} + +function makeConfig(userText: string): ChatMiddlewareConfig { + return { + messages: [{ role: 'user', content: userText }], + systemPrompts: ['base prompt'], + tools: [], + } +} + +function makeCtx( + config: ChatMiddlewareConfig, + deferred: Array>, +): ChatMiddlewareContext { + return { + phase: 'init', + messages: config.messages, + defer: (p: Promise) => deferred.push(p), + } as unknown as ChatMiddlewareContext +} + +function fakeAdapter( + saved: Array<{ scope: MemoryScope; turn: MemoryTurn }>, +): MemoryAdapter { + return { + id: 'fake', + recall: async () => ({ + systemPrompt: 'MEMORY: the user likes cats', + fragments: [{ text: 'the user likes cats', source: 'f1' }], + tools: [catTool], + toolGuidance: 'Use cat_tool when relevant.', + }), + save: async (scope, turn) => { + saved.push({ scope, turn }) + return [{ ok: true }] + }, + } +} + +const scope: MemoryScope = { sessionId: 's1', userId: 'u1' } + +describe('memoryMiddleware', () => { + it('injects recalled systemPrompt + toolGuidance + tools at init', async () => { + const mw = memoryMiddleware({ adapter: fakeAdapter([]), scope }) + const config = makeConfig('tell me about my pets') + const result = await mw.onConfig?.(makeCtx(config, []), config) + + expect(result).toBeTruthy() + const patch = result as Partial + expect(patch.systemPrompts).toEqual([ + 'base prompt', + 'Use cat_tool when relevant.', + 'MEMORY: the user likes cats', + ]) + expect(patch.tools?.map((t) => t.name)).toEqual(['cat_tool']) + }) + + it('save-only role skips recall entirely', async () => { + const mw = memoryMiddleware({ + adapter: fakeAdapter([]), + scope, + role: 'save-only', + }) + const config = makeConfig('hello') + const result = await mw.onConfig?.(makeCtx(config, []), config) + expect(result).toBeUndefined() + }) + + it('defers save of the finished turn and reports receipts', async () => { + const saved: Array<{ scope: MemoryScope; turn: MemoryTurn }> = [] + const onSave = vi.fn() + const mw = memoryMiddleware({ adapter: fakeAdapter(saved), scope, onSave }) + + const deferred: Array> = [] + const config = makeConfig('remember I like cats') + const ctx = makeCtx(config, deferred) + // Prime per-request state (captures lastUserText) via onConfig. + await mw.onConfig?.(ctx, config) + + const info: FinishInfo = { + finishReason: 'stop', + duration: 1, + content: 'You like cats!', + } + mw.onFinish?.(ctx, info) + await Promise.all(deferred) + + expect(saved).toHaveLength(1) + expect(saved[0]?.turn).toEqual({ + user: 'remember I like cats', + assistant: 'You like cats!', + }) + expect(onSave).toHaveBeenCalledOnce() + }) + + it('emits memory:snapshot after save when the adapter supports inspection', async () => { + const base = fakeAdapter([]) + const inspectable: MemoryAdapter = { + ...base, + inspect: async () => ({ + takenAt: '2026-07-22T00:00:00.000Z', + data: { + records: [{ id: 'r1', text: 'You like cats!', kind: 'message' }], + }, + }), + listFacts: async () => [ + { id: 'r1', text: 'You like cats!', source: 'assistant' }, + ], + } + const emit = vi.spyOn(aiEventClient, 'emit').mockImplementation(() => {}) + try { + const mw = memoryMiddleware({ adapter: inspectable, scope }) + const deferred: Array> = [] + const config = makeConfig('remember I like cats') + const ctx = makeCtx(config, deferred) + await mw.onConfig?.(ctx, config) + mw.onFinish?.(ctx, { + finishReason: 'stop', + duration: 1, + content: 'You like cats!', + }) + await Promise.all(deferred) + + const snapshotCall = emit.mock.calls.find( + (c) => c[0] === 'memory:snapshot', + ) + expect(snapshotCall).toBeTruthy() + expect(snapshotCall?.[1]).toMatchObject({ + adapter: 'fake', + takenAt: '2026-07-22T00:00:00.000Z', + facts: [{ id: 'r1', text: 'You like cats!' }], + }) + } finally { + emit.mockRestore() + } + }) + + it('does not emit memory:snapshot for adapters without inspect', async () => { + const emit = vi.spyOn(aiEventClient, 'emit').mockImplementation(() => {}) + try { + const mw = memoryMiddleware({ adapter: fakeAdapter([]), scope }) + const deferred: Array> = [] + const config = makeConfig('hi there') + const ctx = makeCtx(config, deferred) + await mw.onConfig?.(ctx, config) + mw.onFinish?.(ctx, { + finishReason: 'stop', + duration: 1, + content: 'hello', + }) + await Promise.all(deferred) + + expect(emit.mock.calls.some((c) => c[0] === 'memory:snapshot')).toBe( + false, + ) + } finally { + emit.mockRestore() + } + }) + + it('injects one memory:state CUSTOM chunk carrying recall metrics + snapshot', async () => { + const inspectable = { + ...fakeAdapter([]), + inspect: async () => ({ + takenAt: '2026-07-22T00:00:00.000Z', + data: { records: [{ id: 'r1', text: 'likes cats', kind: 'message' }] }, + }), + listFacts: async () => [{ id: 'r1', text: 'likes cats', source: 'user' }], + } + const mw = memoryMiddleware({ adapter: inspectable, scope }) + const config = makeConfig('what do I like?') + const ctx = makeCtx(config, []) + await mw.onConfig?.(ctx, config) + + const runStarted = { + type: 'RUN_STARTED', + threadId: 't1', + runId: 'run1', + } as unknown as StreamChunk + const out = await mw.onChunk?.(ctx, runStarted) + + expect(Array.isArray(out)).toBe(true) + const chunks = out as Array + expect(chunks[0]).toBe(runStarted) + const custom = chunks[1] as Extract + expect(custom.type).toBe('CUSTOM') + expect(custom.name).toBe(MEMORY_STATE_EVENT) + expect(custom.value).toMatchObject({ + adapter: 'fake', + query: 'what do I like?', + recall: { fragmentCount: 1, hasTools: true }, + snapshot: { + takenAt: '2026-07-22T00:00:00.000Z', + facts: [{ id: 'r1', text: 'likes cats' }], + }, + }) + + // Injected exactly once per turn — a second chunk passes through untouched. + const again = await mw.onChunk?.(ctx, runStarted) + expect(again).toBeUndefined() + }) + + it('omits the snapshot in memory:state for adapters without inspect', async () => { + const mw = memoryMiddleware({ adapter: fakeAdapter([]), scope }) + const config = makeConfig('hi') + const ctx = makeCtx(config, []) + await mw.onConfig?.(ctx, config) + const out = (await mw.onChunk?.(ctx, { + type: 'RUN_STARTED', + } as unknown as StreamChunk)) as Array + const custom = out[1] as Extract + expect(custom.name).toBe(MEMORY_STATE_EVENT) + expect((custom.value as { snapshot?: unknown }).snapshot).toBeUndefined() + }) + + it('recall failures are non-fatal (no throw, no injection)', async () => { + const failing: MemoryAdapter = { + id: 'boom', + recall: async () => { + throw new Error('recall exploded') + }, + save: async () => [{ ok: true }], + } + const mw = memoryMiddleware({ adapter: failing, scope }) + const config = makeConfig('hi') + const result = await mw.onConfig?.(makeCtx(config, []), config) + expect(result).toBeUndefined() + }) +}) diff --git a/packages/ai-memory/tests/providers/hindsight.test.ts b/packages/ai-memory/tests/providers/hindsight.test.ts new file mode 100644 index 000000000..77a9d2735 --- /dev/null +++ b/packages/ai-memory/tests/providers/hindsight.test.ts @@ -0,0 +1,127 @@ +import { describe, expect, it, vi } from 'vitest' +import { + hindsight, + makeHindsightTools, +} from '../../src/providers/hindsight/index' +import type { + HindsightClientLike, + HindsightRuntime, +} from '../../src/providers/hindsight/index' +import type { SaveReceipt } from '../../src/types' + +/** + * These tests never touch a real Hindsight server. The factory-level assertions + * check the adapter shape; the tool-level assertions drive `makeHindsightTools` + * with a fake {@link HindsightRuntime} so we can verify the retain/recall/reflect + * wiring (and the `onToolRetain`/`onToolRecall` callbacks) in isolation. + */ + +function fakeRuntime( + overrides: Partial = {}, +): HindsightRuntime { + const client: HindsightClientLike = { + retain: async () => ({ stored: true }), + recall: async () => ({ + results: [{ text: 'the user likes penguins', type: 'fact', id: 'm1' }], + }), + reflect: async () => ({ text: 'synthesized reflection' }), + listMemories: async () => ({ items: [] }), + getBankProfile: async () => ({}), + deleteBank: async () => ({}), + ...overrides, + } + return { client, recallToPrompt: (data) => JSON.stringify(data) } +} + +describe('hindsight factory', () => { + it('exposes the recall/save contract with a stable id', () => { + const adapter = hindsight({ user: 'u1' }) + expect(adapter.id).toBe('hindsight') + expect(typeof adapter.recall).toBe('function') + expect(typeof adapter.save).toBe('function') + expect(typeof adapter.inspect).toBe('function') + expect(typeof adapter.listFacts).toBe('function') + }) +}) + +describe('makeHindsightTools', () => { + const deps = () => ({ + getRuntime: async () => fakeRuntime(), + bankId: 'u1__s1', + budget: 'mid', + }) + + it('returns the three memory tools with valid input schemas', () => { + const tools = makeHindsightTools(deps()) + expect(tools.map((t) => t.name)).toEqual([ + 'hindsight_retain', + 'hindsight_recall', + 'hindsight_reflect', + ]) + for (const tool of tools) { + expect(tool.inputSchema).toMatchObject({ + type: 'object', + additionalProperties: false, + }) + } + expect(tools[0]?.inputSchema).toMatchObject({ required: ['content'] }) + expect(tools[1]?.inputSchema).toMatchObject({ required: ['query'] }) + }) + + it('retain tool stores content and fires onToolRetain', async () => { + const retain = vi.fn(async () => ({ id: 'stored-1' })) + const receipts: Array = [] + const tools = makeHindsightTools({ + getRuntime: async () => fakeRuntime({ retain }), + bankId: 'u1__s1', + budget: 'mid', + onToolRetain: (r) => receipts.push(r), + }) + const result = await tools[0]?.execute?.({ content: 'remember this' }) + expect(result).toMatchObject({ ok: true }) + expect(retain).toHaveBeenCalledWith('u1__s1', 'remember this', { + context: 'chat:tool', + timestamp: expect.any(Date), + }) + expect(receipts).toHaveLength(1) + expect(receipts[0]?.ok).toBe(true) + }) + + it('recall tool renders memories and fires onToolRecall', async () => { + const seen: Array<{ query: string }> = [] + const tools = makeHindsightTools({ + getRuntime: async () => fakeRuntime(), + bankId: 'u1__s1', + budget: 'mid', + onToolRecall: (query) => seen.push({ query }), + }) + const result = await tools[1]?.execute?.({ query: 'penguins' }) + expect(String(result)).toContain('penguins') + expect(seen).toEqual([{ query: 'penguins' }]) + }) + + it('reflect tool returns the synthesized text', async () => { + const tools = makeHindsightTools(deps()) + const result = await tools[2]?.execute?.({ query: 'what do I know?' }) + expect(result).toBe('synthesized reflection') + }) + + it('retain tool degrades to an error receipt when the client throws', async () => { + const receipts: Array = [] + const tools = makeHindsightTools({ + getRuntime: async () => + fakeRuntime({ + retain: async () => { + throw new Error('server down') + }, + }), + bankId: 'u1__s1', + budget: 'mid', + onToolRetain: (r) => receipts.push(r), + }) + const result = await tools[0]?.execute?.({ content: 'x' }) + expect(result).toMatchObject({ ok: false }) + expect(receipts[0]?.ok).toBe(false) + expect(receipts[0]?.error).toContain('server down') + }) +}) diff --git a/packages/ai-memory/tests/providers/honcho.test.ts b/packages/ai-memory/tests/providers/honcho.test.ts new file mode 100644 index 000000000..1b638b7d5 --- /dev/null +++ b/packages/ai-memory/tests/providers/honcho.test.ts @@ -0,0 +1,99 @@ +import { describe, expect, it, vi } from 'vitest' +import { + honcho, + parseHonchoRepresentation, +} from '../../src/providers/honcho/index' + +/** + * `@honcho-ai/sdk` is mocked here — no server, no socket. The mock returns + * canned peers/sessions so we can assert the adapter maps `recall`/`save` onto + * the SDK correctly. `parseHonchoRepresentation` is a pure function and needs + * no mocking at all. + */ +vi.mock('@honcho-ai/sdk', () => { + class Honcho { + constructor(_opts: unknown) {} + peer(id: string) { + return { + id, + message: (content: string) => ({ peer: id, content }), + chat: async (query: string) => `dialectic answer about: ${query}`, + representation: async () => + '[2024-05-01T10:00:00Z] user lives in Berlin\n[2024-05-02T11:00:00Z] user likes hiking', + } + } + session(id: string) { + return { + id, + addMessages: async (messages: Array) => ({ + added: messages.length, + }), + messages: async () => [], + summaries: async () => [], + } + } + } + return { Honcho } +}) + +describe('parseHonchoRepresentation', () => { + it('parses timestamped observation lines into fact rows', () => { + const facts = parseHonchoRepresentation( + '[2024-05-01T10:00:00Z] user lives in Berlin', + ) + expect(facts).toHaveLength(1) + expect(facts[0]).toMatchObject({ + text: 'user lives in Berlin', + source: 'observation', + createdAt: '2024-05-01T10:00:00Z', + }) + }) + + it('keeps plain lines and skips headers/blank lines', () => { + const facts = parseHonchoRepresentation( + ['## Explicit Observations', '', 'user prefers dark mode'].join('\n'), + ) + expect(facts.map((f) => f.text)).toEqual(['user prefers dark mode']) + expect(facts[0]?.source).toBe('representation') + }) +}) + +describe('honcho factory', () => { + it('exposes the recall/save contract with a stable id', () => { + const adapter = honcho({ user: 'u1' }) + expect(adapter.id).toBe('honcho') + expect(typeof adapter.recall).toBe('function') + expect(typeof adapter.save).toBe('function') + expect(typeof adapter.inspect).toBe('function') + expect(typeof adapter.listFacts).toBe('function') + }) + + it('save appends the turn and returns an ok receipt', async () => { + const adapter = honcho({ user: 'u1' }) + const receipts = await adapter.save( + { sessionId: 's1', userId: 'u1' }, + { user: 'I live in Berlin', assistant: 'noted' }, + ) + expect(receipts).toHaveLength(1) + expect(receipts[0]?.ok).toBe(true) + expect(receipts[0]?.raw).toMatchObject({ added: 2 }) + }) + + it('recall returns the dialectic answer as the systemPrompt', async () => { + const adapter = honcho({ user: 'u1' }) + const result = await adapter.recall( + { sessionId: 's1', userId: 'u1' }, + 'where do I live', + ) + expect(result.systemPrompt).toBe('dialectic answer about: where do I live') + }) + + it('listFacts parses the peer representation into rows', async () => { + const adapter = honcho({ user: 'u1' }) + const facts = await adapter.listFacts?.({ sessionId: 's1', userId: 'u1' }) + expect(facts?.map((f) => f.text)).toEqual([ + 'user lives in Berlin', + 'user likes hiking', + ]) + }) +}) diff --git a/packages/ai-memory/tests/providers/in-memory.test.ts b/packages/ai-memory/tests/providers/in-memory.test.ts new file mode 100644 index 000000000..d4c0a0c1e --- /dev/null +++ b/packages/ai-memory/tests/providers/in-memory.test.ts @@ -0,0 +1,36 @@ +import { describe, expect, it } from 'vitest' +import { inMemory } from '../../src/providers/in-memory' +import { runMemoryAdapterContract } from '../contract' + +runMemoryAdapterContract('inMemory', () => inMemory()) + +describe('inMemory options', () => { + it('runs an extractor on save and surfaces extracted facts on recall', async () => { + const adapter = inMemory({ + extract: (turn) => [ + { text: `fact: ${turn.user}`, kind: 'fact', importance: 1 }, + ], + }) + const scope = { sessionId: 's1', userId: 'u1' } + await adapter.save(scope, { user: 'I live in Berlin', assistant: 'ok' }) + const result = await adapter.recall(scope, 'Berlin') + expect(result.systemPrompt).toContain('fact:') + expect(result.systemPrompt.toLowerCase()).toContain('berlin') + }) + + it('respects the userId dimension of scope', async () => { + const adapter = inMemory() + await adapter.save( + { sessionId: 's', userId: 'a' }, + { + user: 'apples are red', + assistant: 'ok', + }, + ) + const sameSessionOtherUser = await adapter.recall( + { sessionId: 's', userId: 'b' }, + 'apples', + ) + expect(sameSessionOtherUser.systemPrompt).toBe('') + }) +}) diff --git a/packages/ai-memory/tests/providers/mem0.test.ts b/packages/ai-memory/tests/providers/mem0.test.ts new file mode 100644 index 000000000..4bf938ea3 --- /dev/null +++ b/packages/ai-memory/tests/providers/mem0.test.ts @@ -0,0 +1,126 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { mem0 } from '../../src/providers/mem0/index' + +/** + * mem0 talks to its server over plain `fetch`, so these tests stub `fetch` and + * never open a socket. They assert the `recall`/`save` mapping onto mem0's + * `/memories` and `/search` endpoints — request shape out, contract shape back. + */ + +interface FetchCall { + url: string + method: string + body: unknown +} + +function stubFetch( + handler: (call: FetchCall) => { + ok?: boolean + status?: number + data: unknown + }, +): Array { + const calls: Array = [] + vi.stubGlobal( + 'fetch', + vi.fn(async (url: string, init?: RequestInit) => { + const body = + typeof init?.body === 'string' ? JSON.parse(init.body) : undefined + const call: FetchCall = { url, method: init?.method ?? 'GET', body } + calls.push(call) + const res = handler(call) + const ok = res.ok ?? true + return { + ok, + status: res.status ?? (ok ? 200 : 500), + json: async () => res.data, + text: async () => JSON.stringify(res.data), + } as Response + }), + ) + return calls +} + +afterEach(() => { + vi.unstubAllGlobals() +}) + +describe('mem0 factory', () => { + it('exposes the recall/save contract with a stable id', () => { + const adapter = mem0() + expect(adapter.id).toBe('mem0') + expect(typeof adapter.recall).toBe('function') + expect(typeof adapter.save).toBe('function') + expect(typeof adapter.inspect).toBe('function') + expect(typeof adapter.listFacts).toBe('function') + }) +}) + +describe('mem0 save', () => { + it('POSTs the turn to /memories and returns an ok receipt', async () => { + const calls = stubFetch(() => ({ data: { id: 'mem-1' } })) + const adapter = mem0({ baseUrl: 'http://mem0.test', user: 'u1' }) + const receipts = await adapter.save( + { sessionId: 's1', userId: 'u1' }, + { user: 'I live in Berlin', assistant: 'noted' }, + ) + expect(receipts).toHaveLength(1) + expect(receipts[0]?.ok).toBe(true) + expect(calls[0]?.url).toBe('http://mem0.test/memories') + expect(calls[0]?.method).toBe('POST') + expect(calls[0]?.body).toMatchObject({ + user_id: 'u1', + messages: [ + { role: 'user', content: 'I live in Berlin' }, + { role: 'assistant', content: 'noted' }, + ], + }) + }) +}) + +describe('mem0 recall', () => { + it('maps /search results into fragments and a rendered systemPrompt', async () => { + const calls = stubFetch(() => ({ + data: { + results: [ + { memory: 'lives in Berlin', id: 'm1' }, + { memory: 'likes hiking', id: 'm2' }, + ], + }, + })) + const adapter = mem0({ baseUrl: 'http://mem0.test', user: 'u1' }) + const result = await adapter.recall( + { sessionId: 's1', userId: 'u1' }, + 'where do I live', + ) + expect(calls[0]?.url).toBe('http://mem0.test/search') + expect(calls[0]?.body).toMatchObject({ + query: 'where do I live', + user_id: 'u1', + }) + expect(result.fragments).toHaveLength(2) + expect(result.fragments?.[0]).toMatchObject({ + text: 'lives in Berlin', + source: 'm1', + }) + expect(result.systemPrompt).toContain('lives in Berlin') + expect(result.systemPrompt).toContain('likes hiking') + }) + + it('returns an empty result when the server finds nothing', async () => { + stubFetch(() => ({ data: { results: [] } })) + const adapter = mem0({ baseUrl: 'http://mem0.test' }) + const result = await adapter.recall({ sessionId: 's1' }, 'anything') + expect(result.systemPrompt).toBe('') + expect(result.fragments).toHaveLength(0) + }) + + it('degrades to an empty result on an HTTP error', async () => { + stubFetch(() => ({ ok: false, status: 500, data: 'boom' })) + const adapter = mem0({ baseUrl: 'http://mem0.test' }) + const result = await adapter.recall({ sessionId: 's1' }, 'anything') + expect(result.systemPrompt).toBe('') + expect(result.fragments).toHaveLength(0) + expect(result.raw).toBeDefined() + }) +}) diff --git a/packages/ai-memory/tests/providers/redis.test.ts b/packages/ai-memory/tests/providers/redis.test.ts new file mode 100644 index 000000000..643565916 --- /dev/null +++ b/packages/ai-memory/tests/providers/redis.test.ts @@ -0,0 +1,133 @@ +// @ts-expect-error -- ioredis-mock has no bundled types; the adapter only uses +// the lowercase RedisLike subset ioredis-mock implements (cast below). +import RedisMock from 'ioredis-mock' +import { describe, expect, it, vi } from 'vitest' +import { fromNodeRedis, redis } from '../../src/providers/redis' +import type { RedisLike } from '../../src/providers/redis' +import { runMemoryAdapterContract } from '../contract' + +function mockClient(): RedisLike { + return new RedisMock() as unknown as RedisLike +} + +runMemoryAdapterContract('redis', () => + redis({ redis: mockClient(), prefix: `test:${crypto.randomUUID()}` }), +) + +describe('redis malformed rows', () => { + it('skips a malformed record on read but does NOT delete it', async () => { + const prefix = `test:${crypto.randomUUID()}` + const client = mockClient() + const adapter = redis({ redis: client, prefix }) + const scope = { sessionId: 's', userId: 'u' } + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + try { + await adapter.save(scope, { + user: 'good memory about penguins', + assistant: 'noted', + }) + // Find the stored record ids from the scope index and corrupt one. + const indexKey = `${prefix}:index:u:s` + const ids = await client.smembers(indexKey) + expect(ids.length).toBeGreaterThan(0) + const badId = ids[0] as string + const badKey = `${prefix}:record:${badId}` + await client.set(badKey, '{ not valid json') + + // recall skips the corrupted row without throwing. + const result = await adapter.recall(scope, 'penguins') + expect(result.fragments?.some((f) => f.source === badId)).toBeFalsy() + + // Load-bearing: the malformed row is LEFT IN PLACE, not deleted. + expect(await client.get(badKey)).toBe('{ not valid json') + expect(warn).toHaveBeenCalled() + } finally { + warn.mockRestore() + } + }) +}) + +describe('redis scope-key hardening', () => { + it('escapes the delimiter so scope values containing ":" cannot collide', async () => { + const prefix = `test:${crypto.randomUUID()}` + const client = mockClient() + const adapter = redis({ redis: client, prefix }) + + // Without escaping, { userId: 'a:b', sessionId: 'c' } and + // { userId: 'a', sessionId: 'b:c' } both serialize to key `a:b:c`. + await adapter.save( + { userId: 'a:b', sessionId: 'c' }, + { + user: 'confidential tenant one data', + assistant: 'ok', + }, + ) + const other = await adapter.recall( + { userId: 'a', sessionId: 'b:c' }, + 'confidential', + ) + expect(other.systemPrompt).toBe('') + expect(other.fragments ?? []).toHaveLength(0) + }) +}) + +describe('fromNodeRedis', () => { + it('translates camelCase node-redis methods into lowercase RedisLike calls', async () => { + const calls: Array<{ method: string; args: Array }> = [] + const fakeNodeRedis = { + get: async (key: string) => { + calls.push({ method: 'get', args: [key] }) + return null + }, + set: async (key: string, value: string) => { + calls.push({ method: 'set', args: [key, value] }) + return 'OK' + }, + del: async (keys: Array | string) => { + calls.push({ method: 'del', args: [keys] }) + return Array.isArray(keys) ? keys.length : 1 + }, + sAdd: async (key: string, members: string | Array) => { + calls.push({ method: 'sAdd', args: [key, members] }) + return Array.isArray(members) ? members.length : 1 + }, + sRem: async (key: string, members: string | Array) => { + calls.push({ method: 'sRem', args: [key, members] }) + return Array.isArray(members) ? members.length : 1 + }, + sMembers: async (key: string) => { + calls.push({ method: 'sMembers', args: [key] }) + return [] + }, + mGet: async (keys: Array) => { + calls.push({ method: 'mGet', args: [keys] }) + return [] + }, + } + + const wrapped = fromNodeRedis(fakeNodeRedis) + await wrapped.set('k', 'v') + await wrapped.sadd('s', 'a', 'b') + await wrapped.mget('k1', 'k2') + await wrapped.del('d1', 'd2') + + expect(calls.find((c) => c.method === 'set')).toMatchObject({ + args: ['k', 'v'], + }) + // Variadic members forwarded as an array so node-redis' overload resolves right. + expect( + calls.find( + (c) => + c.method === 'sAdd' && + Array.isArray(c.args[1]) && + (c.args[1] as Array).length === 2, + ), + ).toBeTruthy() + expect(calls.find((c) => c.method === 'mGet')).toMatchObject({ + args: [['k1', 'k2']], + }) + expect(calls.find((c) => c.method === 'del')).toMatchObject({ + args: [['d1', 'd2']], + }) + }) +}) diff --git a/packages/ai-memory/tsconfig.json b/packages/ai-memory/tsconfig.json new file mode 100644 index 000000000..29112eff9 --- /dev/null +++ b/packages/ai-memory/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "dist" + }, + "include": ["vite.config.ts", "./src", "./tests"], + "exclude": ["node_modules", "dist"] +} diff --git a/packages/ai-memory/vite.config.ts b/packages/ai-memory/vite.config.ts new file mode 100644 index 000000000..2ecd6238e --- /dev/null +++ b/packages/ai-memory/vite.config.ts @@ -0,0 +1,42 @@ +import { defineConfig, mergeConfig } from 'vitest/config' +import { tanstackViteConfig } from '@tanstack/vite-config' +import packageJson from './package.json' + +const config = defineConfig({ + test: { + name: packageJson.name, + dir: './', + watch: false, + globals: true, + environment: 'node', + include: ['tests/**/*.test.ts'], + coverage: { + provider: 'v8', + reporter: ['text', 'json', 'html', 'lcov'], + exclude: [ + 'node_modules/', + 'dist/', + 'tests/', + '**/*.test.ts', + '**/*.config.ts', + ], + include: ['src/**/*.ts'], + }, + }, +}) + +export default mergeConfig( + config, + tanstackViteConfig({ + entry: [ + './src/index.ts', + './src/providers/in-memory/index.ts', + './src/providers/redis/index.ts', + './src/providers/hindsight/index.ts', + './src/providers/mem0/index.ts', + './src/providers/honcho/index.ts', + ], + srcDir: './src', + cjs: false, + }), +) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9812eaf84..45e6aa0f4 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1942,6 +1942,34 @@ importers: specifier: ^4.2.0 version: 4.3.6 + packages/ai-memory: + dependencies: + '@tanstack/ai-event-client': + specifier: workspace:* + version: link:../ai-event-client + ioredis: + specifier: '>=5.0.0' + version: 5.9.2 + devDependencies: + '@honcho-ai/sdk': + specifier: ^2.1.1 + version: 2.2.0 + '@tanstack/ai': + specifier: workspace:* + version: link:../ai + '@vectorize-io/hindsight-client': + specifier: ^0.6.1 + version: 0.6.2 + '@vitest/coverage-v8': + specifier: 4.0.14 + version: 4.0.14(supports-color@7.2.0)(vitest@4.1.10) + ioredis-mock: + specifier: ^8.9.0 + version: 8.13.1(@types/ioredis-mock@8.2.7(ioredis@5.9.2))(ioredis@5.9.2) + redis: + specifier: ^4.7.0 + version: 4.7.1 + packages/ai-mistral: dependencies: '@mistralai/mistralai': @@ -2584,6 +2612,9 @@ importers: '@tanstack/ai-mcp': specifier: workspace:* version: link:../../packages/ai-mcp + '@tanstack/ai-memory': + specifier: workspace:* + version: link:../../packages/ai-memory '@tanstack/ai-mistral': specifier: workspace:* version: link:../../packages/ai-mistral @@ -2699,6 +2730,9 @@ importers: '@tanstack/ai-grok': specifier: workspace:* version: link:../../packages/ai-grok + '@tanstack/ai-memory': + specifier: workspace:* + version: link:../../packages/ai-memory '@tanstack/ai-ollama': specifier: workspace:* version: link:../../packages/ai-ollama @@ -2717,6 +2751,12 @@ importers: '@tanstack/nitro-v2-vite-plugin': specifier: ^1.155.0 version: 1.155.0(aws4fetch@1.0.20)(rolldown@1.1.5)(vite@8.1.4(@types/node@24.10.3)(esbuild@0.28.1)(jiti@2.7.0)(less@4.6.6)(sass@1.101.0)(terser@5.44.1)(tsx@4.21.0)(yaml@2.9.0)) + '@tanstack/react-ai-devtools': + specifier: workspace:* + version: link:../../packages/react-ai-devtools + '@tanstack/react-devtools': + specifier: ^0.9.10 + version: 0.9.10(@types/react-dom@19.2.3(@types/react@19.2.7))(@types/react@19.2.7)(csstype@3.2.3)(react-dom@19.2.3(react@19.2.3))(react@19.2.3)(solid-js@1.9.10) '@tanstack/react-router': specifier: ^1.158.4 version: 1.159.5(react-dom@19.2.3(react@19.2.3))(react@19.2.3) @@ -4957,6 +4997,9 @@ packages: '@harperfast/extended-iterable@1.0.3': resolution: {integrity: sha512-sSAYhQca3rDWtQUHSAPeO7axFIUJOI6hn1gjRC5APVE1a90tuyT8f5WIgRsFhhWA7htNkju2veB9eWL6YHi/Lw==} + '@honcho-ai/sdk@2.2.0': + resolution: {integrity: sha512-SyygN+BrpUB2fRjhwcYmT+tcEhHrKmbj9nOZLVUFY7M5YBswJ+mZb/CeLpNbRh+QQTU8F8JWY3lQat95c6nwmA==} + '@hono/node-server@1.19.14': resolution: {integrity: sha512-GwtvgtXxnWsucXvbQXkRgqksiH2Qed37H9xHZocE5sA3N8O8O8/8FA3uclQXxXVzc9XBZuEOMK7+r02FmSpHtw==} engines: {node: '>=18.14.1'} @@ -5183,6 +5226,9 @@ packages: '@types/node': optional: true + '@ioredis/as-callback@3.0.0': + resolution: {integrity: sha512-Kqv1rZ3WbgOrS+hgzJ5xG5WQuhvzzSTRYvNeyPMLOAM78MHSnuKI20JeJGbpuAt//LCuP0vsexZcorqW7kWhJg==} + '@ioredis/commands@1.5.0': resolution: {integrity: sha512-eUgLqrMf8nJkZxT24JvVRrQya1vZkQh8BBeYNwGDqa5I0VUi8ACx7uFvAaLxintokpTenkK6DASvo/bvNbBGow==} @@ -7466,6 +7512,35 @@ packages: '@types/react': optional: true + '@redis/bloom@1.2.0': + resolution: {integrity: sha512-HG2DFjYKbpNmVXsa0keLHp/3leGJz1mjh09f2RLGGLQZzSHpkmZWuwJbAvo3QcRY8p80m5+ZdXZdYOSBLlp7Cg==} + peerDependencies: + '@redis/client': ^1.0.0 + + '@redis/client@1.6.1': + resolution: {integrity: sha512-/KCsg3xSlR+nCK8/8ZYSknYxvXHwubJrU82F3Lm1Fp6789VQ0/3RJKfsmRXjqfaTA++23CvC3hqmqe/2GEt6Kw==} + engines: {node: '>=14'} + + '@redis/graph@1.1.1': + resolution: {integrity: sha512-FEMTcTHZozZciLRl6GiiIB4zGm5z5F3F6a6FZCyrfxdKOhFlGkiAqlexWMBzCi4DcRoyiOsuLfW+cjlGWyExOw==} + peerDependencies: + '@redis/client': ^1.0.0 + + '@redis/json@1.0.7': + resolution: {integrity: sha512-6UyXfjVaTBTJtKNG4/9Z8PSpKE6XgSyEb8iwaqDcy+uKrd/DGYHTWkUdnQDyzm727V7p21WUMhsqz5oy65kPcQ==} + peerDependencies: + '@redis/client': ^1.0.0 + + '@redis/search@1.2.0': + resolution: {integrity: sha512-tYoDBbtqOVigEDMAcTGsRlMycIIjwMCgD8eR2t0NANeQmgK/lvxNAvYyb6bZDD4frHRhIHkJu2TBRvB0ERkOmw==} + peerDependencies: + '@redis/client': ^1.0.0 + + '@redis/time-series@1.1.0': + resolution: {integrity: sha512-c1Q99M5ljsIuc4YdaCwfUEXsofakb9c8+Zse2qxTadu8TalLXuAESzLvFAvNVbkmSlvlzIQOLpBCmWI9wTOt+g==} + peerDependencies: + '@redis/client': ^1.0.0 + '@rolldown/binding-android-arm64@1.0.0-beta.53': resolution: {integrity: sha512-Ok9V8o7o6YfSdTTYA/uHH30r3YtOxLD6G3wih/U9DO0ucBBFq8WPt/DslU53OgfteLRHITZny9N/qCUxMf9kjQ==} engines: {node: ^20.19.0 || >=22.12.0} @@ -9121,6 +9196,11 @@ packages: '@types/hast@3.0.4': resolution: {integrity: sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ==} + '@types/ioredis-mock@8.2.7': + resolution: {integrity: sha512-YsGiaOIYBKeVvu/7GYziAD8qX3LJem5LK00d5PKykzsQJMLysAqXA61AkNuYWCekYl64tbMTqVOMF4SYoCPbQg==} + peerDependencies: + ioredis: '>=5' + '@types/istanbul-lib-coverage@2.0.6': resolution: {integrity: sha512-2QF/t/auWm0lsy8XtKVPG19v3sSOQlJe/YHZgfjb/KBBHOGSV+J2q/S671rcq9uTBrLAXmZpqJiaQbMT+zNU1w==} @@ -9422,6 +9502,9 @@ packages: resolution: {integrity: sha512-WmoN8qaIAo7WTYWbAZuG8PYEhn5fkz7dZrqTBZ7dtt//lL2Gwms1IcnQ5yHqjDfX8Ft5j4YzDM23f87zBfDe9g==} deprecated: Potential CWE-502 - Update to 1.3.1 or higher + '@vectorize-io/hindsight-client@0.6.2': + resolution: {integrity: sha512-bymmlMWI1z0zOjgY+wRMLudNxzqcW20VHMtyV3QLhwJm63NeQN/nEZ4plWPR0p28DffaUM5nk2VSxzQljN+Mow==} + '@vercel/nft@1.3.0': resolution: {integrity: sha512-i4EYGkCsIjzu4vorDUbqglZc5eFtQI2syHb++9ZUDm6TU4edVywGpVnYDein35x9sevONOn9/UabfQXuNXtuzQ==} engines: {node: '>=20'} @@ -11271,6 +11354,14 @@ packages: picomatch: optional: true + fengari-interop@0.1.4: + resolution: {integrity: sha512-4/CW/3PJUo3ebD4ACgE1g/3NGEYSq7OQAyETyypsAl/WeySDBbxExikkayNkZzbpgyC9GyJp8v1DU2VOXxNq7Q==} + peerDependencies: + fengari: ^0.1.0 + + fengari@0.1.5: + resolution: {integrity: sha512-0DS4Nn4rV8qyFlQCpKK8brT61EUtswynrpfFTcgLErcilBIBskSMQ86fO2WVuybr14ywyKdRjv91FiRZwnEuvQ==} + fetch-blob@3.2.0: resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==} engines: {node: ^12.20 || >= 14.13} @@ -11457,6 +11548,10 @@ packages: resolution: {integrity: sha512-zV/5HKTfCeKWnxG0Dmrw51hEWFGfcF2xiXqcA3+J90WDuP0SvoiSO5ORvcBsifmx/FoIjgQN3oNOGaQ5PhLFkg==} engines: {node: '>=18'} + generic-pool@3.9.0: + resolution: {integrity: sha512-hymDOu5B53XvN4QT9dBmZxPX4CWhBPPLguTZ9MMFeFa/Kg0xWVfylOVNlJji/E7yTZWFd/q9GO5TxDLq156D7g==} + engines: {node: '>= 4'} + gensync@1.0.0-beta.2: resolution: {integrity: sha512-3hN7NaskYvMDLQY55gnW3NQ+mesEAepTqlg+VEbj7zzqEMBVNhzcGYYeqFo/TlYz6eQiFcp1HcsCZO+nGgS8zg==} engines: {node: '>=6.9.0'} @@ -11863,6 +11958,13 @@ packages: invariant@2.2.4: resolution: {integrity: sha512-phJfQVBuaJM5raOpJjSfkiD6BpbCE4Ns//LaXl6wGYtUBY83nWS6Rf9tXm2e8VaK60JEjYldbPif/A2B1C2gNA==} + ioredis-mock@8.13.1: + resolution: {integrity: sha512-Wsi50AU+cMiI32nAgfwpUaJVBtb4iQdVsOHl9M6R3tePCO/8vGsToCVIG82XWAxN4Se55TZoOzVseu+QngFLyw==} + engines: {node: '>=12.22'} + peerDependencies: + '@types/ioredis-mock': ^8 + ioredis: ^5 + ioredis@5.9.2: resolution: {integrity: sha512-tAAg/72/VxOUW7RQSX1pIxJVucYKcjFjfvj60L57jrZpYCHC3XN0WCQ3sNYL4Gmvv+7GPvTAjc+KSdeNuE8oWQ==} engines: {node: '>=12.22.0'} @@ -14022,6 +14124,10 @@ packages: resolution: {integrity: sha512-9u/XQ1pvrQtYyMpZe7DXKv2p5CNvyVwzUB6uhLAnQwHMSgKMBR62lc7AHljaeteeHXn11XTAaLLUVZYVZyuRBQ==} engines: {node: '>= 20.19.0'} + readline-sync@1.4.10: + resolution: {integrity: sha512-gNva8/6UAe8QYepIQH/jQ2qn91Qj0B9sYjMBBs3QOB8F2CXcKgLxQaJRP76sWVRQt+QU+8fAkCbCvjjMFu7Ycw==} + engines: {node: '>= 0.8.0'} + recast@0.23.11: resolution: {integrity: sha512-YTUo+Flmw4ZXiWfQKGcwwc11KnoRAYgzAE2E7mXKCjSviTKShtxBsN6YUUBB2gtaBzKzeKunxhUwNHQuRryhWA==} engines: {node: '>= 4'} @@ -14045,6 +14151,9 @@ packages: resolution: {integrity: sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==} engines: {node: '>=4'} + redis@4.7.1: + resolution: {integrity: sha512-S1bJDnqLftzHXHP8JsT5II/CtHWQrASX5K96REjWjlmWKrviSOLWmM7QnRLstAWsu1VBBV1ffV6DzCvxNP0UJQ==} + reflect-metadata@0.2.2: resolution: {integrity: sha512-urBwgfrvVP/eAyXx4hluJivBKzuEbSQs9rKWCrCkbSxNv8mxPcUZKeuoF3Uy4mJl3Lwprp6yy5/39VWigZ4K6Q==} @@ -14589,6 +14698,9 @@ packages: sprintf-js@1.0.3: resolution: {integrity: sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g==} + sprintf-js@1.1.3: + resolution: {integrity: sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA==} + srvx@0.11.17: resolution: {integrity: sha512-43yM4luKfCJamyCMhrUeHUPOrf8TdZe7kN8s5zayZCH5OeprYqi49Aso5ZvHXR4aB+DHaRNO/diNFgZSMNG8Xw==} engines: {node: '>=20.16.0'} @@ -16115,6 +16227,9 @@ packages: zod@3.25.76: resolution: {integrity: sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==} + zod@4.0.0: + resolution: {integrity: sha512-9diLdTPc/L7w/5jI4C3gHYNiGHDV9IZYxo1e5LSD8cabi65WVTWWb+g2BGPEpUUCOxR4D+6O5B0AzyMdUAXwrw==} + zod@4.2.1: resolution: {integrity: sha512-0wZ1IRqGGhMP76gLqz8EyfBXKk0J2qo2+H3fi4mcUP/KtTocoX08nmIAHl1Z2kJIZbZee8KOpBCSNPRgauucjw==} @@ -18460,7 +18575,7 @@ snapshots: progress: 2.0.3 prompts: 2.4.2 resolve-from: 5.0.0 - semver: 7.7.4 + semver: 7.8.4 send: 0.19.0 slugify: 1.6.9 stacktrace-parser: 0.1.11 @@ -18535,7 +18650,7 @@ snapshots: progress: 2.0.3 prompts: 2.4.2 resolve-from: 5.0.0 - semver: 7.7.4 + semver: 7.8.4 send: 0.19.0 slugify: 1.6.9 stacktrace-parser: 0.1.11 @@ -18575,7 +18690,7 @@ snapshots: debug: 4.4.3(supports-color@7.2.0) getenv: 2.0.0 glob: 13.0.0 - semver: 7.7.4 + semver: 7.8.4 slugify: 1.6.9 xcode: 3.0.1 xml2js: 0.6.0 @@ -18595,7 +18710,7 @@ snapshots: getenv: 2.0.0 glob: 13.0.0 resolve-workspace-root: 2.0.1 - semver: 7.7.4 + semver: 7.8.4 slugify: 1.6.9 transitivePeerDependencies: - supports-color @@ -18656,7 +18771,7 @@ snapshots: ignore: 5.3.2 minimatch: 10.2.5 resolve-from: 5.0.0 - semver: 7.7.4 + semver: 7.8.4 transitivePeerDependencies: - supports-color @@ -18934,6 +19049,10 @@ snapshots: '@harperfast/extended-iterable@1.0.3': optional: true + '@honcho-ai/sdk@2.2.0': + dependencies: + zod: 4.0.0 + '@hono/node-server@1.19.14(hono@4.12.23)': dependencies: hono: 4.12.23 @@ -19087,6 +19206,8 @@ snapshots: optionalDependencies: '@types/node': 24.10.3 + '@ioredis/as-callback@3.0.0': {} + '@ioredis/commands@1.5.0': {} '@isaacs/cliui@8.0.2': @@ -20336,7 +20457,7 @@ snapshots: extract-zip: 2.0.1 progress: 2.0.3 proxy-agent: 6.5.0 - semver: 7.7.4 + semver: 7.8.4 tar-fs: 3.1.2 yargs: 17.7.2 transitivePeerDependencies: @@ -21131,7 +21252,7 @@ snapshots: metro: 0.84.4 metro-config: 0.84.4 metro-core: 0.84.4 - semver: 7.7.4 + semver: 7.8.4 transitivePeerDependencies: - bufferutil - supports-color @@ -21190,6 +21311,32 @@ snapshots: optionalDependencies: '@types/react': 19.2.7 + '@redis/bloom@1.2.0(@redis/client@1.6.1)': + dependencies: + '@redis/client': 1.6.1 + + '@redis/client@1.6.1': + dependencies: + cluster-key-slot: 1.1.2 + generic-pool: 3.9.0 + yallist: 4.0.0 + + '@redis/graph@1.1.1(@redis/client@1.6.1)': + dependencies: + '@redis/client': 1.6.1 + + '@redis/json@1.0.7(@redis/client@1.6.1)': + dependencies: + '@redis/client': 1.6.1 + + '@redis/search@1.2.0(@redis/client@1.6.1)': + dependencies: + '@redis/client': 1.6.1 + + '@redis/time-series@1.1.0(@redis/client@1.6.1)': + dependencies: + '@redis/client': 1.6.1 + '@rolldown/binding-android-arm64@1.0.0-beta.53': optional: true @@ -23440,6 +23587,10 @@ snapshots: dependencies: '@types/unist': 3.0.3 + '@types/ioredis-mock@8.2.7(ioredis@5.9.2)': + dependencies: + ioredis: 5.9.2 + '@types/istanbul-lib-coverage@2.0.6': {} '@types/istanbul-lib-report@3.0.3': @@ -23737,6 +23888,8 @@ snapshots: '@ungap/structured-clone@1.3.0': {} + '@vectorize-io/hindsight-client@0.6.2': {} + '@vercel/nft@1.3.0(rollup@4.60.1)': dependencies: '@mapbox/node-pre-gyp': 2.0.3 @@ -26112,6 +26265,16 @@ snapshots: optionalDependencies: picomatch: 4.0.5 + fengari-interop@0.1.4(fengari@0.1.5): + dependencies: + fengari: 0.1.5 + + fengari@0.1.5: + dependencies: + readline-sync: 1.4.10 + sprintf-js: 1.1.3 + tmp: 0.2.7 + fetch-blob@3.2.0: dependencies: node-domexception: 1.0.0 @@ -26305,6 +26468,8 @@ snapshots: transitivePeerDependencies: - supports-color + generic-pool@3.9.0: {} + gensync@1.0.0-beta.2: {} get-caller-file@2.0.5: {} @@ -26794,6 +26959,16 @@ snapshots: dependencies: loose-envify: 1.4.0 + ioredis-mock@8.13.1(@types/ioredis-mock@8.2.7(ioredis@5.9.2))(ioredis@5.9.2): + dependencies: + '@ioredis/as-callback': 3.0.0 + '@ioredis/commands': 1.5.0 + '@types/ioredis-mock': 8.2.7(ioredis@5.9.2) + fengari: 0.1.5 + fengari-interop: 0.1.4(fengari@0.1.5) + ioredis: 5.9.2 + semver: 7.8.4 + ioredis@5.9.2: dependencies: '@ioredis/commands': 1.5.0 @@ -27008,7 +27183,7 @@ snapshots: '@babel/parser': 7.29.7 '@istanbuljs/schema': 0.1.6 istanbul-lib-coverage: 3.2.2 - semver: 7.7.4 + semver: 7.8.4 transitivePeerDependencies: - supports-color @@ -29825,6 +30000,8 @@ snapshots: readdirp@5.0.0: {} + readline-sync@1.4.10: {} + recast@0.23.11: dependencies: ast-types: 0.16.1 @@ -29856,6 +30033,15 @@ snapshots: dependencies: redis-errors: 1.2.0 + redis@4.7.1: + dependencies: + '@redis/bloom': 1.2.0(@redis/client@1.6.1) + '@redis/client': 1.6.1 + '@redis/graph': 1.1.1(@redis/client@1.6.1) + '@redis/json': 1.0.7(@redis/client@1.6.1) + '@redis/search': 1.2.0(@redis/client@1.6.1) + '@redis/time-series': 1.1.0(@redis/client@1.6.1) + reflect-metadata@0.2.2: {} regenerate-unicode-properties@10.2.2: @@ -30583,6 +30769,8 @@ snapshots: sprintf-js@1.0.3: {} + sprintf-js@1.1.3: {} + srvx@0.11.17: {} srvx@0.8.16: {} @@ -32071,8 +32259,7 @@ snapshots: yallist@3.1.1: {} - yallist@4.0.0: - optional: true + yallist@4.0.0: {} yallist@5.0.0: {} @@ -32155,6 +32342,8 @@ snapshots: zod@3.25.76: {} + zod@4.0.0: {} + zod@4.2.1: {} zod@4.3.6: {} diff --git a/testing/e2e/package.json b/testing/e2e/package.json index 78ffbf406..b84b22a92 100644 --- a/testing/e2e/package.json +++ b/testing/e2e/package.json @@ -27,6 +27,7 @@ "@tanstack/ai-grok": "workspace:*", "@tanstack/ai-groq": "workspace:*", "@tanstack/ai-mcp": "workspace:*", + "@tanstack/ai-memory": "workspace:*", "@tanstack/ai-mistral": "workspace:*", "@tanstack/ai-ollama": "workspace:*", "@tanstack/ai-openai": "workspace:*", diff --git a/testing/e2e/src/lib/devtools-memory-store.ts b/testing/e2e/src/lib/devtools-memory-store.ts new file mode 100644 index 000000000..1869eea95 --- /dev/null +++ b/testing/e2e/src/lib/devtools-memory-store.ts @@ -0,0 +1,9 @@ +import { inMemory } from '@tanstack/ai-memory/in-memory' + +/** + * Shared process-local memory adapter for the `/devtools-memory` E2E route. + * The default `inMemory()` stores raw user/assistant turns (kind `message`) + * with zero deps — legible for asserting "what's in memory" in the devtools + * panel. Scope is keyed per-test by `sessionId` (the Playwright `testId`). + */ +export const devtoolsMemoryAdapter = inMemory() diff --git a/testing/e2e/src/lib/memory-capture.ts b/testing/e2e/src/lib/memory-capture.ts new file mode 100644 index 000000000..489593e53 --- /dev/null +++ b/testing/e2e/src/lib/memory-capture.ts @@ -0,0 +1,52 @@ +/** + * Per-testId capture for the `memory` mode of `/middleware-test`. A recorder + * middleware placed AFTER `memoryMiddleware` records the config the model + * actually sees (post-injection system prompts + tool names) plus a flag for + * each deferred `save`. The page fetches it via + * `GET /api/middleware-test?testId=...&kind=memory` and surfaces it in the DOM + * for the Playwright spec. Mirrors `phase-capture.ts`. + */ + +export interface MemoryConfigRecord { + /** System-prompt strings present in the config at `init` (post memory injection). */ + systemPrompts: Array + /** Tool names present in the config at `init` (post memory injection). */ + toolNames: Array +} + +export interface MemoryCapture { + /** One entry per `onConfig(init)` observed by the recorder. */ + configs: Array + /** Count of `save` calls the fake adapter observed. */ + saveCount: number +} + +const captures: Map = new Map() + +function bucketFor(captureId: string): MemoryCapture { + let bucket = captures.get(captureId) + if (!bucket) { + bucket = { configs: [], saveCount: 0 } + captures.set(captureId, bucket) + } + return bucket +} + +export function resetMemoryCapture(captureId: string): void { + captures.set(captureId, { configs: [], saveCount: 0 }) +} + +export function getMemoryCapture(captureId: string): MemoryCapture { + return bucketFor(captureId) +} + +export function recordMemoryConfig( + captureId: string, + record: MemoryConfigRecord, +): void { + bucketFor(captureId).configs.push(record) +} + +export function recordMemorySave(captureId: string): void { + bucketFor(captureId).saveCount += 1 +} diff --git a/testing/e2e/src/routeTree.gen.ts b/testing/e2e/src/routeTree.gen.ts index 8687d013b..657fe4e17 100644 --- a/testing/e2e/src/routeTree.gen.ts +++ b/testing/e2e/src/routeTree.gen.ts @@ -18,6 +18,7 @@ import { Route as DevtoolsToolsRouteImport } from './routes/devtools-tools' import { Route as DevtoolsStructuredRouteImport } from './routes/devtools-structured' import { Route as DevtoolsRouteBRouteImport } from './routes/devtools-route-b' import { Route as DevtoolsRouteARouteImport } from './routes/devtools-route-a' +import { Route as DevtoolsMemoryRouteImport } from './routes/devtools-memory' import { Route as DevtoolsGenerationHooksRouteImport } from './routes/devtools-generation-hooks' import { Route as DevtoolsChatRouteImport } from './routes/devtools-chat' import { Route as ChatClientDefaultBridgeRouteImport } from './routes/chat-client-default-bridge' @@ -51,6 +52,7 @@ import { Route as ApiInterruptsTestRouteImport } from './routes/api.interrupts-t import { Route as ApiImageRouteImport } from './routes/api.image' import { Route as ApiForeignInterruptRouteImport } from './routes/api.foreign-interrupt' import { Route as ApiDurableDeliveryRouteImport } from './routes/api.durable-delivery' +import { Route as ApiDevtoolsMemoryRouteImport } from './routes/api.devtools-memory' import { Route as ApiChatRouteImport } from './routes/api.chat' import { Route as ApiAudioRouteImport } from './routes/api.audio' import { Route as ApiArktypeToolWireRouteImport } from './routes/api.arktype-tool-wire' @@ -109,6 +111,11 @@ const DevtoolsRouteARoute = DevtoolsRouteARouteImport.update({ path: '/devtools-route-a', getParentRoute: () => rootRouteImport, } as any) +const DevtoolsMemoryRoute = DevtoolsMemoryRouteImport.update({ + id: '/devtools-memory', + path: '/devtools-memory', + getParentRoute: () => rootRouteImport, +} as any) const DevtoolsGenerationHooksRoute = DevtoolsGenerationHooksRouteImport.update({ id: '/devtools-generation-hooks', path: '/devtools-generation-hooks', @@ -278,6 +285,11 @@ const ApiDurableDeliveryRoute = ApiDurableDeliveryRouteImport.update({ path: '/api/durable-delivery', getParentRoute: () => rootRouteImport, } as any) +const ApiDevtoolsMemoryRoute = ApiDevtoolsMemoryRouteImport.update({ + id: '/api/devtools-memory', + path: '/api/devtools-memory', + getParentRoute: () => rootRouteImport, +} as any) const ApiChatRoute = ApiChatRouteImport.update({ id: '/api/chat', path: '/api/chat', @@ -345,6 +357,7 @@ export interface FileRoutesByFullPath { '/chat-client-default-bridge': typeof ChatClientDefaultBridgeRoute '/devtools-chat': typeof DevtoolsChatRoute '/devtools-generation-hooks': typeof DevtoolsGenerationHooksRoute + '/devtools-memory': typeof DevtoolsMemoryRoute '/devtools-route-a': typeof DevtoolsRouteARoute '/devtools-route-b': typeof DevtoolsRouteBRoute '/devtools-structured': typeof DevtoolsStructuredRoute @@ -361,6 +374,7 @@ export interface FileRoutesByFullPath { '/api/arktype-tool-wire': typeof ApiArktypeToolWireRoute '/api/audio': typeof ApiAudioRouteWithChildren '/api/chat': typeof ApiChatRoute + '/api/devtools-memory': typeof ApiDevtoolsMemoryRoute '/api/durable-delivery': typeof ApiDurableDeliveryRoute '/api/foreign-interrupt': typeof ApiForeignInterruptRoute '/api/image': typeof ApiImageRouteWithChildren @@ -401,6 +415,7 @@ export interface FileRoutesByTo { '/chat-client-default-bridge': typeof ChatClientDefaultBridgeRoute '/devtools-chat': typeof DevtoolsChatRoute '/devtools-generation-hooks': typeof DevtoolsGenerationHooksRoute + '/devtools-memory': typeof DevtoolsMemoryRoute '/devtools-route-a': typeof DevtoolsRouteARoute '/devtools-route-b': typeof DevtoolsRouteBRoute '/devtools-structured': typeof DevtoolsStructuredRoute @@ -417,6 +432,7 @@ export interface FileRoutesByTo { '/api/arktype-tool-wire': typeof ApiArktypeToolWireRoute '/api/audio': typeof ApiAudioRouteWithChildren '/api/chat': typeof ApiChatRoute + '/api/devtools-memory': typeof ApiDevtoolsMemoryRoute '/api/durable-delivery': typeof ApiDurableDeliveryRoute '/api/foreign-interrupt': typeof ApiForeignInterruptRoute '/api/image': typeof ApiImageRouteWithChildren @@ -458,6 +474,7 @@ export interface FileRoutesById { '/chat-client-default-bridge': typeof ChatClientDefaultBridgeRoute '/devtools-chat': typeof DevtoolsChatRoute '/devtools-generation-hooks': typeof DevtoolsGenerationHooksRoute + '/devtools-memory': typeof DevtoolsMemoryRoute '/devtools-route-a': typeof DevtoolsRouteARoute '/devtools-route-b': typeof DevtoolsRouteBRoute '/devtools-structured': typeof DevtoolsStructuredRoute @@ -474,6 +491,7 @@ export interface FileRoutesById { '/api/arktype-tool-wire': typeof ApiArktypeToolWireRoute '/api/audio': typeof ApiAudioRouteWithChildren '/api/chat': typeof ApiChatRoute + '/api/devtools-memory': typeof ApiDevtoolsMemoryRoute '/api/durable-delivery': typeof ApiDurableDeliveryRoute '/api/foreign-interrupt': typeof ApiForeignInterruptRoute '/api/image': typeof ApiImageRouteWithChildren @@ -516,6 +534,7 @@ export interface FileRouteTypes { | '/chat-client-default-bridge' | '/devtools-chat' | '/devtools-generation-hooks' + | '/devtools-memory' | '/devtools-route-a' | '/devtools-route-b' | '/devtools-structured' @@ -532,6 +551,7 @@ export interface FileRouteTypes { | '/api/arktype-tool-wire' | '/api/audio' | '/api/chat' + | '/api/devtools-memory' | '/api/durable-delivery' | '/api/foreign-interrupt' | '/api/image' @@ -572,6 +592,7 @@ export interface FileRouteTypes { | '/chat-client-default-bridge' | '/devtools-chat' | '/devtools-generation-hooks' + | '/devtools-memory' | '/devtools-route-a' | '/devtools-route-b' | '/devtools-structured' @@ -588,6 +609,7 @@ export interface FileRouteTypes { | '/api/arktype-tool-wire' | '/api/audio' | '/api/chat' + | '/api/devtools-memory' | '/api/durable-delivery' | '/api/foreign-interrupt' | '/api/image' @@ -628,6 +650,7 @@ export interface FileRouteTypes { | '/chat-client-default-bridge' | '/devtools-chat' | '/devtools-generation-hooks' + | '/devtools-memory' | '/devtools-route-a' | '/devtools-route-b' | '/devtools-structured' @@ -644,6 +667,7 @@ export interface FileRouteTypes { | '/api/arktype-tool-wire' | '/api/audio' | '/api/chat' + | '/api/devtools-memory' | '/api/durable-delivery' | '/api/foreign-interrupt' | '/api/image' @@ -685,6 +709,7 @@ export interface RootRouteChildren { ChatClientDefaultBridgeRoute: typeof ChatClientDefaultBridgeRoute DevtoolsChatRoute: typeof DevtoolsChatRoute DevtoolsGenerationHooksRoute: typeof DevtoolsGenerationHooksRoute + DevtoolsMemoryRoute: typeof DevtoolsMemoryRoute DevtoolsRouteARoute: typeof DevtoolsRouteARoute DevtoolsRouteBRoute: typeof DevtoolsRouteBRoute DevtoolsStructuredRoute: typeof DevtoolsStructuredRoute @@ -701,6 +726,7 @@ export interface RootRouteChildren { ApiArktypeToolWireRoute: typeof ApiArktypeToolWireRoute ApiAudioRoute: typeof ApiAudioRouteWithChildren ApiChatRoute: typeof ApiChatRoute + ApiDevtoolsMemoryRoute: typeof ApiDevtoolsMemoryRoute ApiDurableDeliveryRoute: typeof ApiDurableDeliveryRoute ApiForeignInterruptRoute: typeof ApiForeignInterruptRoute ApiImageRoute: typeof ApiImageRouteWithChildren @@ -797,6 +823,13 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof DevtoolsRouteARouteImport parentRoute: typeof rootRouteImport } + '/devtools-memory': { + id: '/devtools-memory' + path: '/devtools-memory' + fullPath: '/devtools-memory' + preLoaderRoute: typeof DevtoolsMemoryRouteImport + parentRoute: typeof rootRouteImport + } '/devtools-generation-hooks': { id: '/devtools-generation-hooks' path: '/devtools-generation-hooks' @@ -1028,6 +1061,13 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof ApiDurableDeliveryRouteImport parentRoute: typeof rootRouteImport } + '/api/devtools-memory': { + id: '/api/devtools-memory' + path: '/api/devtools-memory' + fullPath: '/api/devtools-memory' + preLoaderRoute: typeof ApiDevtoolsMemoryRouteImport + parentRoute: typeof rootRouteImport + } '/api/chat': { id: '/api/chat' path: '/api/chat' @@ -1178,6 +1218,7 @@ const rootRouteChildren: RootRouteChildren = { ChatClientDefaultBridgeRoute: ChatClientDefaultBridgeRoute, DevtoolsChatRoute: DevtoolsChatRoute, DevtoolsGenerationHooksRoute: DevtoolsGenerationHooksRoute, + DevtoolsMemoryRoute: DevtoolsMemoryRoute, DevtoolsRouteARoute: DevtoolsRouteARoute, DevtoolsRouteBRoute: DevtoolsRouteBRoute, DevtoolsStructuredRoute: DevtoolsStructuredRoute, @@ -1194,6 +1235,7 @@ const rootRouteChildren: RootRouteChildren = { ApiArktypeToolWireRoute: ApiArktypeToolWireRoute, ApiAudioRoute: ApiAudioRouteWithChildren, ApiChatRoute: ApiChatRoute, + ApiDevtoolsMemoryRoute: ApiDevtoolsMemoryRoute, ApiDurableDeliveryRoute: ApiDurableDeliveryRoute, ApiForeignInterruptRoute: ApiForeignInterruptRoute, ApiImageRoute: ApiImageRouteWithChildren, diff --git a/testing/e2e/src/routes/api.devtools-memory.ts b/testing/e2e/src/routes/api.devtools-memory.ts new file mode 100644 index 000000000..7a4d5bbd7 --- /dev/null +++ b/testing/e2e/src/routes/api.devtools-memory.ts @@ -0,0 +1,96 @@ +import { createFileRoute } from '@tanstack/react-router' +import { + chat, + chatParamsFromRequestBody, + maxIterations, + toServerSentEventsResponse, +} from '@tanstack/ai' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { createTextAdapter } from '@/lib/providers' +import { devtoolsMemoryAdapter } from '@/lib/devtools-memory-store' +import type { StreamChunk } from '@tanstack/ai' + +/** + * Chat endpoint for the `/devtools-memory` E2E route. Mirrors `/api/chat` but + * wires `memoryMiddleware({ adapter: inMemory() })` so recall/save run and the + * middleware injects the `memory:state` CUSTOM chunk the client devtools bridge + * re-emits as `memory:*`. Scope is the per-test `testId`. + */ +export const Route = createFileRoute('/api/devtools-memory')({ + server: { + handlers: { + POST: async ({ request }) => { + await import('@/lib/llmock-server').then((m) => m.ensureLLMock()) + if (request.signal.aborted) { + return new Response(null, { status: 499 }) + } + + const abortController = new AbortController() + + let params + try { + params = await chatParamsFromRequestBody(await request.json()) + } catch (error) { + return new Response( + error instanceof Error ? error.message : 'Bad request', + { status: 400 }, + ) + } + + const fp = params.forwardedProps as Record + const testId = typeof fp.testId === 'string' ? fp.testId : undefined + const aimockPort = + fp.aimockPort != null ? Number(fp.aimockPort) : undefined + const sessionId = testId ?? 'devtools-memory' + + const adapterOptions = createTextAdapter( + 'openai', + undefined, + aimockPort, + testId, + 'chat', + ) + + try { + const memory = memoryMiddleware({ + adapter: devtoolsMemoryAdapter, + scope: { sessionId }, + }) + + const stream = chat({ + ...adapterOptions, + tools: [], + systemPrompts: [ + 'You are a helpful assistant with long-term memory.', + ], + middleware: [memory], + agentLoopStrategy: maxIterations(5), + messages: params.messages, + threadId: params.threadId, + runId: params.runId, + abortController, + }) + + return toServerSentEventsResponse( + stream as AsyncIterable, + { abortController }, + ) + } catch (error) { + console.error('[api.devtools-memory] Error:', error) + if ( + (error instanceof Error && error.name === 'AbortError') || + abortController.signal.aborted + ) { + return new Response(null, { status: 499 }) + } + return new Response( + JSON.stringify({ + error: error instanceof Error ? error.message : 'error', + }), + { status: 500, headers: { 'Content-Type': 'application/json' } }, + ) + } + }, + }, + }, +}) diff --git a/testing/e2e/src/routes/api.middleware-test.ts b/testing/e2e/src/routes/api.middleware-test.ts index b9b8a918c..8e5bead4d 100644 --- a/testing/e2e/src/routes/api.middleware-test.ts +++ b/testing/e2e/src/routes/api.middleware-test.ts @@ -8,9 +8,17 @@ import { toolDefinition, } from '@tanstack/ai' import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import { memoryMiddleware } from '@tanstack/ai-memory' +import type { MemoryAdapter } from '@tanstack/ai-memory' +import { + getMemoryCapture, + recordMemoryConfig, + recordMemorySave, + resetMemoryCapture, +} from '@/lib/memory-capture' import { SpanStatusCode } from '@opentelemetry/api' import { z } from 'zod' -import type { ChatMiddleware, StreamChunk } from '@tanstack/ai' +import type { ChatMiddleware, StreamChunk, Tool } from '@tanstack/ai' import type { AttributeValue, Attributes, @@ -165,6 +173,56 @@ async function* teeForPhaseCapture( } } +/** + * Fake memory adapter for `memory` mode. `recall` unconditionally returns a + * known system-prompt block plus a memory tool (so the spec can assert both the + * prompt injection AND tool injection reach the model config), and `save` + * records that the deferred write ran. Deterministic — no real vendor, no + * cross-request state. + */ +const RECALL_MORE_TOOL: Tool = { + name: 'recall_more', + description: 'Look up additional long-term memory by query.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, +} + +function createFakeMemoryAdapter(captureId: string): MemoryAdapter { + return { + id: 'fake-memory', + recall: async () => ({ + systemPrompt: 'MEMORY: the user previously said they love TanStack.', + fragments: [{ text: 'the user loves TanStack', source: 'f1' }], + tools: [RECALL_MORE_TOOL], + toolGuidance: 'Use recall_more to look up additional memory when needed.', + }), + save: async () => { + recordMemorySave(captureId) + return [{ ok: true }] + }, + } +} + +/** + * Recorder placed AFTER `memoryMiddleware` so it observes the config the model + * actually sees — i.e. WITH the recalled system prompt + tools already merged + * in. Records that into the per-testId memory capture. + */ +function createMemoryConfigRecorder(captureId: string): ChatMiddleware { + return { + name: 'memory-config-recorder', + onConfig(ctx, config) { + if (ctx.phase !== 'init') return + recordMemoryConfig(captureId, { + systemPrompts: config.systemPrompts.map((p) => + typeof p === 'string' ? p : p.content, + ), + toolNames: config.tools.map((t) => t.name), + }) + return + }, + } +} + // Minimal in-memory tracer/meter. Captures into a per-testId bucket so that // the Playwright spec can fetch the recorded state via GET after the stream // finishes. Not exported — only used to build otelMiddleware for the test. @@ -352,6 +410,25 @@ export const Route = createFileRoute('/api/middleware-test')({ resetPhaseCapture(testId) middleware.push(createPhaseRecorderMiddleware(testId)) } + if (middlewareMode === 'memory') { + if (!testId) { + return new Response( + JSON.stringify({ error: 'memory mode requires testId' }), + { + status: 400, + headers: { 'Content-Type': 'application/json' }, + }, + ) + } + resetMemoryCapture(testId) + middleware.push( + memoryMiddleware({ + adapter: createFakeMemoryAdapter(testId), + scope: { sessionId: testId }, + }), + createMemoryConfigRecorder(testId), + ) + } if (middlewareMode === 'otel') { if (!OTEL_TEST_ENABLED) { return new Response(null, { status: 404 }) @@ -450,6 +527,19 @@ export const Route = createFileRoute('/api/middleware-test')({ }) } + // Memory capture — like phase capture, available without the OTEL gate. + if (kind === 'memory') { + if (!testId) { + return new Response( + JSON.stringify({ error: 'testId query param required' }), + { status: 400, headers: { 'Content-Type': 'application/json' } }, + ) + } + return new Response(JSON.stringify(getMemoryCapture(testId)), { + headers: { 'Content-Type': 'application/json' }, + }) + } + // OTEL capture remains gated — the GET cannot act as an oracle in a // production-like build. if (!OTEL_TEST_ENABLED) { diff --git a/testing/e2e/src/routes/devtools-memory.tsx b/testing/e2e/src/routes/devtools-memory.tsx new file mode 100644 index 000000000..2f6c45dd7 --- /dev/null +++ b/testing/e2e/src/routes/devtools-memory.tsx @@ -0,0 +1,46 @@ +import { createFileRoute } from '@tanstack/react-router' +import { fetchServerSentEvents, useChat } from '@tanstack/ai-react' +import { ChatUI } from '@/components/ChatUI' +import { DevtoolsHarness } from '@/components/DevtoolsHarness' +import { parseDevtoolsRouteSearch } from '@/lib/devtools-test' + +export const Route = createFileRoute('/devtools-memory')({ + component: DevtoolsMemoryRoute, + validateSearch: parseDevtoolsRouteSearch, +}) + +function DevtoolsMemoryRoute() { + const { testId, aimockPort } = Route.useSearch() + const chat = useChat({ + id: 'devtools-memory:primary', + connection: fetchServerSentEvents('/api/devtools-memory'), + body: { feature: 'chat', testId, aimockPort }, + devtools: { name: 'Memory Chat' }, + }) + + return ( + +
+
+
+ Memory Chat +
+
+ {chat.status} +
+
+ { + void chat.sendMessage(text) + }} + onStop={chat.stop} + /> +
+ + ) +} diff --git a/testing/e2e/src/routes/middleware-test.tsx b/testing/e2e/src/routes/middleware-test.tsx index 784c4067a..ce36ed656 100644 --- a/testing/e2e/src/routes/middleware-test.tsx +++ b/testing/e2e/src/routes/middleware-test.tsx @@ -9,6 +9,7 @@ const MIDDLEWARE_MODES = [ { id: 'capability', label: 'Capability (provide/consume prefix)' }, { id: 'phase-recorder', label: 'Phase Recorder (capture phase + chunks)' }, { id: 'otel', label: 'OpenTelemetry (capture spans/metrics)' }, + { id: 'memory', label: 'Memory (recall/save)' }, ] as const interface PhaseCaptureSnapshot { @@ -84,6 +85,10 @@ function MiddlewareTestPage() { const [testComplete, setTestComplete] = useState(false) const [phaseCapture, setPhaseCapture] = useState(EMPTY_PHASE_CAPTURE) + const [memoryCapture, setMemoryCapture] = useState<{ + configs: Array<{ systemPrompts: Array; toolNames: Array }> + saveCount: number + }>({ configs: [], saveCount: 0 }) const { messages, sendMessage, isLoading } = useChat({ id: `mw-test-${scenario}-${middlewareMode}-${provider ?? 'openai'}-${model ?? 'default'}`, @@ -110,6 +115,21 @@ function MiddlewareTestPage() { }) return } + if (middlewareMode === 'memory' && testId) { + void fetch( + `/api/middleware-test?testId=${encodeURIComponent(testId)}&kind=memory`, + ) + .then((res) => (res.ok ? res.json() : { configs: [], saveCount: 0 })) + .then((data) => { + setMemoryCapture(data) + setTestComplete(true) + }) + .catch(() => { + setMemoryCapture({ configs: [], saveCount: 0 }) + setTestComplete(true) + }) + return + } setTestComplete(true) }, }) @@ -224,6 +244,9 @@ function MiddlewareTestPage() { 
         {JSON.stringify(phaseCapture.yieldedChunks)}
+ 
+        {JSON.stringify(memoryCapture)}
+
{ + await page.addInitScript(() => localStorage.clear()) +}) + +test('memory middleware surfaces recall + stored records in the devtools Memory tab', async ({ + page, + testId, + aimockPort, +}) => { + await page.goto(devtoolsUrl('/devtools-memory', testId, aimockPort)) + + // Turn 1: the save is deferred until after the turn, so this turn's + // start-of-turn snapshot is empty — it seeds memory for turn 2. + await sendMessage(page, '[chat] recommend a guitar') + await waitForResponse(page) + await expect(page.getByTestId('assistant-message').first()).toBeVisible() + + // Turn 2: recall runs and the transported snapshot now reflects turn 1's + // saved user/assistant turn. + await sendMessage(page, '[chat] recommend a guitar') + await waitForResponse(page) + + await openDevtools(page) + await selectHook(page, 'Memory Chat') + await selectDevtoolsTab(page, 'Memory') + + await expect(page.getByTestId('ai-devtools-memory-panel')).toBeVisible() + + // Operations timeline: at least the recall for turn 2 was re-emitted. + await expect( + page.getByTestId('ai-devtools-memory-event').first(), + ).toBeVisible() + + // Live contents: turn 1's user + assistant messages are stored and shown. + await expect + .poll(async () => page.getByTestId('ai-devtools-memory-record').count()) + .toBeGreaterThanOrEqual(2) +}) diff --git a/testing/e2e/tests/middleware.spec.ts b/testing/e2e/tests/middleware.spec.ts index 6b15d6759..d1f146a7f 100644 --- a/testing/e2e/tests/middleware.spec.ts +++ b/testing/e2e/tests/middleware.spec.ts @@ -365,4 +365,50 @@ test.describe('Middleware Lifecycle', () => { expect(textPart?.content).not.toContain('[MW]') expect(textPart?.content).toContain('Hello') }) + + test('memory middleware injects recalled prompt + tools and defers save', async ({ + page, + testId, + aimockPort, + }) => { + const params = new URLSearchParams() + if (testId) params.set('testId', testId) + if (aimockPort) params.set('aimockPort', String(aimockPort)) + const qs = params.toString() + await page.goto(`/middleware-test${qs ? '?' + qs : ''}`) + await page.waitForTimeout(2000) // hydration + await page.locator('#mw-scenario-select').selectOption('basic-text') + await page.locator('#mw-mode-select').selectOption('memory') + await page.locator('#mw-run-button').click() + + await page.waitForFunction( + () => + document + .querySelector('#mw-metadata') + ?.getAttribute('data-test-complete') === 'true', + { timeout: 10000 }, + ) + + const memoryJson = await page.locator('#mw-memory-json').textContent() + const capture = JSON.parse(memoryJson || '{}') as { + configs: Array<{ systemPrompts: Array; toolNames: Array }> + saveCount: number + } + + // The recorder placed after memoryMiddleware saw the config the model + // receives: the recalled system prompt (+ tool guidance) and the injected + // tool must both be present. + expect(capture.configs.length).toBeGreaterThan(0) + const injected = capture.configs.find((c) => + c.systemPrompts.some((p) => p.includes('love TanStack')), + ) + expect(injected).toBeTruthy() + expect(injected?.systemPrompts.some((p) => p.includes('recall_more'))).toBe( + true, + ) + expect(injected?.toolNames).toContain('recall_more') + + // The finished turn deferred a save through the adapter. + expect(capture.saveCount).toBeGreaterThanOrEqual(1) + }) }) diff --git a/testing/panel/package.json b/testing/panel/package.json index d33648586..c0da6d9a1 100644 --- a/testing/panel/package.json +++ b/testing/panel/package.json @@ -18,12 +18,15 @@ "@tanstack/ai-event-client": "workspace:*", "@tanstack/ai-gemini": "workspace:*", "@tanstack/ai-grok": "workspace:*", + "@tanstack/ai-memory": "workspace:*", "@tanstack/ai-ollama": "workspace:*", "@tanstack/ai-openai": "workspace:*", "@tanstack/ai-openrouter": "workspace:*", "@tanstack/ai-react": "workspace:*", "@tanstack/ai-react-ui": "workspace:*", "@tanstack/nitro-v2-vite-plugin": "^1.155.0", + "@tanstack/react-ai-devtools": "workspace:*", + "@tanstack/react-devtools": "^0.9.10", "@tanstack/react-router": "^1.158.4", "@tanstack/react-start": "^1.159.0", "@tanstack/start": "^1.120.20", diff --git a/testing/panel/src/components/Header.tsx b/testing/panel/src/components/Header.tsx index 21e94f96f..b7711d91a 100644 --- a/testing/panel/src/components/Header.tsx +++ b/testing/panel/src/components/Header.tsx @@ -3,6 +3,7 @@ import { Link } from '@tanstack/react-router' import { useState } from 'react' import { Beaker, + BrainCircuit, ChefHat, FileText, FlaskConical, @@ -120,6 +121,24 @@ export default function Header() {
+ setIsOpen(false)} + className="flex items-center gap-3 p-3 rounded-lg hover:bg-gray-800 transition-colors mb-2" + activeProps={{ + className: + 'flex items-center gap-3 p-3 rounded-lg bg-cyan-600 hover:bg-cyan-700 transition-colors mb-2', + }} + > + +
+ Memory + + recall/save + +
+ +

Activities diff --git a/testing/panel/src/lib/memory-store.ts b/testing/panel/src/lib/memory-store.ts new file mode 100644 index 000000000..337294765 --- /dev/null +++ b/testing/panel/src/lib/memory-store.ts @@ -0,0 +1,24 @@ +import { inMemory } from '@tanstack/ai-memory/in-memory' +import type { RecallResult } from '@tanstack/ai-memory' + +/** + * Process-local memory backing the `/memory` demo page. + * + * `inMemory()` stores everything in an in-process `Map`, so the chat route + * (which writes via `memoryMiddleware`) and the inspect route (which reads via + * `inspect`/`listFacts`) MUST share this exact singleton — a second + * `inMemory()` call would have its own, empty store. + * + * Defaults are deliberate: no `embedder`/`extract`, so `save` just stores the + * raw user/assistant turn (kind `message`). That keeps the demo zero-dep and + * makes the stored content legible in the panel. To demo derived facts or + * semantic recall, pass `{ extract, embedder }` to `inMemory()` here. + */ +export const memoryAdapter = inMemory() + +/** + * Records what the last `recall` injected for each session, so the page can + * show "what memory fed into this turn". Populated from the middleware's + * `onRecall` callback in the chat route; read by the inspect route. + */ +export const lastRecallBySession = new Map() diff --git a/testing/panel/src/routeTree.gen.ts b/testing/panel/src/routeTree.gen.ts index 837280778..c9ce6fdaf 100644 --- a/testing/panel/src/routeTree.gen.ts +++ b/testing/panel/src/routeTree.gen.ts @@ -16,6 +16,7 @@ import { Route as SummarizeRouteImport } from './routes/summarize' import { Route as StructuredRouteImport } from './routes/structured' import { Route as StreamDebuggerRouteImport } from './routes/stream-debugger' import { Route as SimulatorRouteImport } from './routes/simulator' +import { Route as MemoryRouteImport } from './routes/memory' import { Route as ImageRouteImport } from './routes/image' import { Route as AddonManagerRouteImport } from './routes/addon-manager' import { Route as IndexRouteImport } from './routes/index' @@ -25,6 +26,8 @@ import { Route as ApiTranscriptionRouteImport } from './routes/api.transcription import { Route as ApiSummarizeRouteImport } from './routes/api.summarize' import { Route as ApiStructuredRouteImport } from './routes/api.structured' import { Route as ApiSimulatorChatRouteImport } from './routes/api.simulator-chat' +import { Route as ApiMemoryInspectRouteImport } from './routes/api.memory-inspect' +import { Route as ApiMemoryChatRouteImport } from './routes/api.memory-chat' import { Route as ApiLoadTraceRouteImport } from './routes/api.load-trace' import { Route as ApiListTracesRouteImport } from './routes/api.list-traces' import { Route as ApiImageRouteImport } from './routes/api.image' @@ -66,6 +69,11 @@ const SimulatorRoute = SimulatorRouteImport.update({ path: '/simulator', getParentRoute: () => rootRouteImport, } as any) +const MemoryRoute = MemoryRouteImport.update({ + id: '/memory', + path: '/memory', + getParentRoute: () => rootRouteImport, +} as any) const ImageRoute = ImageRouteImport.update({ id: '/image', path: '/image', @@ -111,6 +119,16 @@ const ApiSimulatorChatRoute = ApiSimulatorChatRouteImport.update({ path: '/api/simulator-chat', getParentRoute: () => rootRouteImport, } as any) +const ApiMemoryInspectRoute = ApiMemoryInspectRouteImport.update({ + id: '/api/memory-inspect', + path: '/api/memory-inspect', + getParentRoute: () => rootRouteImport, +} as any) +const ApiMemoryChatRoute = ApiMemoryChatRouteImport.update({ + id: '/api/memory-chat', + path: '/api/memory-chat', + getParentRoute: () => rootRouteImport, +} as any) const ApiLoadTraceRoute = ApiLoadTraceRouteImport.update({ id: '/api/load-trace', path: '/api/load-trace', @@ -141,6 +159,7 @@ export interface FileRoutesByFullPath { '/': typeof IndexRoute '/addon-manager': typeof AddonManagerRoute '/image': typeof ImageRoute + '/memory': typeof MemoryRoute '/simulator': typeof SimulatorRoute '/stream-debugger': typeof StreamDebuggerRoute '/structured': typeof StructuredRoute @@ -153,6 +172,8 @@ export interface FileRoutesByFullPath { '/api/image': typeof ApiImageRoute '/api/list-traces': typeof ApiListTracesRoute '/api/load-trace': typeof ApiLoadTraceRoute + '/api/memory-chat': typeof ApiMemoryChatRoute + '/api/memory-inspect': typeof ApiMemoryInspectRoute '/api/simulator-chat': typeof ApiSimulatorChatRoute '/api/structured': typeof ApiStructuredRoute '/api/summarize': typeof ApiSummarizeRoute @@ -164,6 +185,7 @@ export interface FileRoutesByTo { '/': typeof IndexRoute '/addon-manager': typeof AddonManagerRoute '/image': typeof ImageRoute + '/memory': typeof MemoryRoute '/simulator': typeof SimulatorRoute '/stream-debugger': typeof StreamDebuggerRoute '/structured': typeof StructuredRoute @@ -176,6 +198,8 @@ export interface FileRoutesByTo { '/api/image': typeof ApiImageRoute '/api/list-traces': typeof ApiListTracesRoute '/api/load-trace': typeof ApiLoadTraceRoute + '/api/memory-chat': typeof ApiMemoryChatRoute + '/api/memory-inspect': typeof ApiMemoryInspectRoute '/api/simulator-chat': typeof ApiSimulatorChatRoute '/api/structured': typeof ApiStructuredRoute '/api/summarize': typeof ApiSummarizeRoute @@ -188,6 +212,7 @@ export interface FileRoutesById { '/': typeof IndexRoute '/addon-manager': typeof AddonManagerRoute '/image': typeof ImageRoute + '/memory': typeof MemoryRoute '/simulator': typeof SimulatorRoute '/stream-debugger': typeof StreamDebuggerRoute '/structured': typeof StructuredRoute @@ -200,6 +225,8 @@ export interface FileRoutesById { '/api/image': typeof ApiImageRoute '/api/list-traces': typeof ApiListTracesRoute '/api/load-trace': typeof ApiLoadTraceRoute + '/api/memory-chat': typeof ApiMemoryChatRoute + '/api/memory-inspect': typeof ApiMemoryInspectRoute '/api/simulator-chat': typeof ApiSimulatorChatRoute '/api/structured': typeof ApiStructuredRoute '/api/summarize': typeof ApiSummarizeRoute @@ -213,6 +240,7 @@ export interface FileRouteTypes { | '/' | '/addon-manager' | '/image' + | '/memory' | '/simulator' | '/stream-debugger' | '/structured' @@ -225,6 +253,8 @@ export interface FileRouteTypes { | '/api/image' | '/api/list-traces' | '/api/load-trace' + | '/api/memory-chat' + | '/api/memory-inspect' | '/api/simulator-chat' | '/api/structured' | '/api/summarize' @@ -236,6 +266,7 @@ export interface FileRouteTypes { | '/' | '/addon-manager' | '/image' + | '/memory' | '/simulator' | '/stream-debugger' | '/structured' @@ -248,6 +279,8 @@ export interface FileRouteTypes { | '/api/image' | '/api/list-traces' | '/api/load-trace' + | '/api/memory-chat' + | '/api/memory-inspect' | '/api/simulator-chat' | '/api/structured' | '/api/summarize' @@ -259,6 +292,7 @@ export interface FileRouteTypes { | '/' | '/addon-manager' | '/image' + | '/memory' | '/simulator' | '/stream-debugger' | '/structured' @@ -271,6 +305,8 @@ export interface FileRouteTypes { | '/api/image' | '/api/list-traces' | '/api/load-trace' + | '/api/memory-chat' + | '/api/memory-inspect' | '/api/simulator-chat' | '/api/structured' | '/api/summarize' @@ -283,6 +319,7 @@ export interface RootRouteChildren { IndexRoute: typeof IndexRoute AddonManagerRoute: typeof AddonManagerRoute ImageRoute: typeof ImageRoute + MemoryRoute: typeof MemoryRoute SimulatorRoute: typeof SimulatorRoute StreamDebuggerRoute: typeof StreamDebuggerRoute StructuredRoute: typeof StructuredRoute @@ -295,6 +332,8 @@ export interface RootRouteChildren { ApiImageRoute: typeof ApiImageRoute ApiListTracesRoute: typeof ApiListTracesRoute ApiLoadTraceRoute: typeof ApiLoadTraceRoute + ApiMemoryChatRoute: typeof ApiMemoryChatRoute + ApiMemoryInspectRoute: typeof ApiMemoryInspectRoute ApiSimulatorChatRoute: typeof ApiSimulatorChatRoute ApiStructuredRoute: typeof ApiStructuredRoute ApiSummarizeRoute: typeof ApiSummarizeRoute @@ -354,6 +393,13 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof SimulatorRouteImport parentRoute: typeof rootRouteImport } + '/memory': { + id: '/memory' + path: '/memory' + fullPath: '/memory' + preLoaderRoute: typeof MemoryRouteImport + parentRoute: typeof rootRouteImport + } '/image': { id: '/image' path: '/image' @@ -417,6 +463,20 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof ApiSimulatorChatRouteImport parentRoute: typeof rootRouteImport } + '/api/memory-inspect': { + id: '/api/memory-inspect' + path: '/api/memory-inspect' + fullPath: '/api/memory-inspect' + preLoaderRoute: typeof ApiMemoryInspectRouteImport + parentRoute: typeof rootRouteImport + } + '/api/memory-chat': { + id: '/api/memory-chat' + path: '/api/memory-chat' + fullPath: '/api/memory-chat' + preLoaderRoute: typeof ApiMemoryChatRouteImport + parentRoute: typeof rootRouteImport + } '/api/load-trace': { id: '/api/load-trace' path: '/api/load-trace' @@ -459,6 +519,7 @@ const rootRouteChildren: RootRouteChildren = { IndexRoute: IndexRoute, AddonManagerRoute: AddonManagerRoute, ImageRoute: ImageRoute, + MemoryRoute: MemoryRoute, SimulatorRoute: SimulatorRoute, StreamDebuggerRoute: StreamDebuggerRoute, StructuredRoute: StructuredRoute, @@ -471,6 +532,8 @@ const rootRouteChildren: RootRouteChildren = { ApiImageRoute: ApiImageRoute, ApiListTracesRoute: ApiListTracesRoute, ApiLoadTraceRoute: ApiLoadTraceRoute, + ApiMemoryChatRoute: ApiMemoryChatRoute, + ApiMemoryInspectRoute: ApiMemoryInspectRoute, ApiSimulatorChatRoute: ApiSimulatorChatRoute, ApiStructuredRoute: ApiStructuredRoute, ApiSummarizeRoute: ApiSummarizeRoute, diff --git a/testing/panel/src/routes/__root.tsx b/testing/panel/src/routes/__root.tsx index 3801738e2..d4e517b83 100644 --- a/testing/panel/src/routes/__root.tsx +++ b/testing/panel/src/routes/__root.tsx @@ -1,4 +1,6 @@ import { createRootRoute, HeadContent, Scripts } from '@tanstack/react-router' +import { TanStackDevtools } from '@tanstack/react-devtools' +import { aiDevtoolsPlugin } from '@tanstack/react-ai-devtools' import Header from '@/components/Header' import appCss from '../styles.css?url' @@ -36,6 +38,11 @@ function RootDocument({ children }: { children: React.ReactNode }) {

{children}
+ diff --git a/testing/panel/src/routes/api.memory-chat.ts b/testing/panel/src/routes/api.memory-chat.ts new file mode 100644 index 000000000..be1146576 --- /dev/null +++ b/testing/panel/src/routes/api.memory-chat.ts @@ -0,0 +1,121 @@ +import { createFileRoute } from '@tanstack/react-router' +import { + chat, + createChatOptions, + maxIterations, + toServerSentEventsResponse, +} from '@tanstack/ai' +import { memoryMiddleware } from '@tanstack/ai-memory' +import { anthropicText } from '@tanstack/ai-anthropic' +import { geminiText } from '@tanstack/ai-gemini' +import { grokText } from '@tanstack/ai-grok' +import { openaiText } from '@tanstack/ai-openai' +import { ollamaText } from '@tanstack/ai-ollama' +import { openRouterText } from '@tanstack/ai-openrouter' +import { lastRecallBySession, memoryAdapter } from '@/lib/memory-store' +import type { Provider } from '@/lib/model-selection' + +const SYSTEM_PROMPT = `You are a helpful, friendly assistant with long-term memory. + +Earlier facts the user shared may be injected into your system prompt under a +"memory" heading. When they are, use them to answer — for example, if the user +tells you their name in one turn and asks for it in a later turn, recall it from +memory rather than saying you don't know.` + +/** + * Chat endpoint for the `/memory` demo. Identical in spirit to `/api/chat`, + * minus the guitar tools and trace recording, plus a `memoryMiddleware` wired + * to the shared {@link memoryAdapter} singleton so recall/save persist across + * requests. The middleware is built per request with a static scope derived + * from the client-supplied `sessionId`. + */ +export const Route = createFileRoute('/api/memory-chat')({ + server: { + handlers: { + POST: async ({ request }) => { + const requestSignal = request.signal + if (requestSignal.aborted) { + return new Response(null, { status: 499 }) + } + + const abortController = new AbortController() + const body = await request.json() + const messages = body.messages + const data = body.data || {} + + const provider: Provider = data.provider || 'openai' + const model: string | undefined = data.model + const sessionId: string = data.sessionId || 'panel-default-session' + + try { + const adapterConfig = { + anthropic: () => + createChatOptions({ + adapter: anthropicText((model || 'claude-sonnet-4-5') as any), + }), + gemini: () => + createChatOptions({ + adapter: geminiText((model || 'gemini-2.5-flash') as any), + }), + grok: () => + createChatOptions({ + adapter: grokText((model || 'grok-build-0.1') as any), + }), + ollama: () => + createChatOptions({ + adapter: ollamaText((model || 'mistral:7b') as any), + }), + openai: () => + createChatOptions({ + adapter: openaiText((model || 'gpt-4o') as any), + }), + openrouter: () => + createChatOptions({ + adapter: openRouterText((model || 'openai/gpt-4o') as any), + }), + } + + const options = adapterConfig[provider]() + const { adapter } = options + + console.log( + `>> memory chat: model ${model} on ${provider} (session ${sessionId})`, + ) + + const memory = memoryMiddleware({ + adapter: memoryAdapter, + scope: { sessionId }, + onRecall: (info) => { + lastRecallBySession.set(sessionId, info.result) + }, + }) + + const stream = chat({ + ...options, + adapter, + tools: [], + systemPrompts: [SYSTEM_PROMPT], + middleware: [memory], + agentLoopStrategy: maxIterations(5), + messages, + abortController, + }) + + return toServerSentEventsResponse(stream, { abortController }) + } catch (error: any) { + console.error('[api.memory-chat] Error:', error?.message) + if (error.name === 'AbortError' || abortController.signal.aborted) { + return new Response(null, { status: 499 }) + } + return new Response( + JSON.stringify({ error: error.message || 'An error occurred' }), + { + status: 500, + headers: { 'Content-Type': 'application/json' }, + }, + ) + } + }, + }, + }, +}) diff --git a/testing/panel/src/routes/api.memory-inspect.ts b/testing/panel/src/routes/api.memory-inspect.ts new file mode 100644 index 000000000..9a859487b --- /dev/null +++ b/testing/panel/src/routes/api.memory-inspect.ts @@ -0,0 +1,36 @@ +import { createFileRoute } from '@tanstack/react-router' +import { lastRecallBySession, memoryAdapter } from '@/lib/memory-store' + +/** + * Read side of the `/memory` demo. Returns everything the panel needs to show + * "what's in memory" for a session, straight off the shared singleton adapter: + * the full record snapshot, the flat fact list, and what the most recent + * `recall` injected into the prompt. + */ +export const Route = createFileRoute('/api/memory-inspect')({ + server: { + handlers: { + GET: async ({ request }) => { + const sessionId = + new URL(request.url).searchParams.get('sessionId') ?? '' + const scope = { sessionId } + + const snapshot = await memoryAdapter.inspect?.(scope) + const facts = await memoryAdapter.listFacts?.(scope) + const lastRecall = lastRecallBySession.get(sessionId) ?? null + + return new Response( + JSON.stringify({ + snapshot: snapshot ?? null, + facts: facts ?? [], + lastRecall, + }), + { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }, + ) + }, + }, + }, +}) diff --git a/testing/panel/src/routes/memory.tsx b/testing/panel/src/routes/memory.tsx new file mode 100644 index 000000000..34fec675f --- /dev/null +++ b/testing/panel/src/routes/memory.tsx @@ -0,0 +1,338 @@ +import { useCallback, useEffect, useMemo, useRef, useState } from 'react' +import { createFileRoute } from '@tanstack/react-router' +import { RefreshCw, RotateCcw, Send } from 'lucide-react' +import { fetchServerSentEvents, useChat } from '@tanstack/ai-react' +import type { UIMessage } from '@tanstack/ai-react' +import { MODEL_OPTIONS, getDefaultModelOption } from '@/lib/model-selection' +import type { ModelOption } from '@/lib/model-selection' + +const SESSION_STORAGE_KEY = 'panel-memory-session' + +// Shapes returned by /api/memory-inspect. These mirror the `inMemory()` +// snapshot payload + the RecallResult contract; kept local so the page has no +// build-time dependency on server internals. +interface RecordRow { + id: string + text: string + kind: string + role?: 'user' | 'assistant' + createdAt: number + importance?: number +} +interface FactRow { + id: string + text: string + source?: string + createdAt?: string +} +interface Fragment { + text: string + source: string +} +interface LastRecall { + systemPrompt: string + fragments?: Array + toolGuidance?: string +} +interface InspectResponse { + snapshot: { takenAt: string; data: { records?: Array } } | null + facts: Array + lastRecall: LastRecall | null +} + +function getMessageText(parts: UIMessage['parts']): string { + return parts + .filter((part) => part.type === 'text' && 'content' in part && part.content) + .map((part) => (part as { type: 'text'; content: string }).content) + .join('') +} + +function formatTime(value: number | string | undefined): string { + if (value === undefined) return '' + const date = new Date(value) + if (Number.isNaN(date.getTime())) return '' + return date.toLocaleTimeString() +} + +function MemoryPage() { + const [selectedModel, setSelectedModel] = useState( + getDefaultModelOption(), + ) + const [sessionId, setSessionId] = useState('') + const [inspect, setInspect] = useState(null) + const [input, setInput] = useState('') + + // Resolve (or create) a stable session id, persisted so memory survives reloads. + useEffect(() => { + let existing = localStorage.getItem(SESSION_STORAGE_KEY) + if (!existing) { + existing = crypto.randomUUID() + localStorage.setItem(SESSION_STORAGE_KEY, existing) + } + setSessionId(existing) + }, []) + + const body = useMemo( + () => ({ + provider: selectedModel.provider, + model: selectedModel.model, + sessionId, + }), + [selectedModel.provider, selectedModel.model, sessionId], + ) + + const { messages, sendMessage, isLoading } = useChat({ + connection: fetchServerSentEvents('/api/memory-chat'), + body, + devtools: { name: 'Memory' }, + }) + + const refreshInspect = useCallback(async () => { + if (!sessionId) return + try { + const res = await fetch( + `/api/memory-inspect?sessionId=${encodeURIComponent(sessionId)}`, + ) + if (res.ok) setInspect(await res.json()) + } catch { + // Non-fatal: the inspector is a read-only view; leave the last snapshot. + } + }, [sessionId]) + + // Refresh the inspector whenever the session changes and each time a turn + // finishes (isLoading falls back to false). + const wasLoading = useRef(false) + useEffect(() => { + if (wasLoading.current && !isLoading) refreshInspect() + wasLoading.current = isLoading + }, [isLoading, refreshInspect]) + useEffect(() => { + refreshInspect() + }, [refreshInspect]) + + const startNewSession = () => { + const next = crypto.randomUUID() + localStorage.setItem(SESSION_STORAGE_KEY, next) + setSessionId(next) + setInspect(null) + } + + const submit = () => { + const text = input.trim() + if (!text || isLoading) return + sendMessage(text) + setInput('') + } + + const records = inspect?.snapshot?.data.records ?? [] + const facts = inspect?.facts ?? [] + const lastRecall = inspect?.lastRecall ?? null + + return ( +
+ {/* Left: chat */} +
+
+ + +
+ +
+ {messages.length === 0 ? ( +

+ Say something like "My name is Jack and I love guitars", then in a + later turn ask "What's my name?" — the answer comes from memory. +

+ ) : ( + messages.map(({ id, role, parts }) => ( +
+
+ {getMessageText(parts)} +
+
+ )) + )} +
+ +
+
+ setInput(e.target.value)} + onKeyDown={(e) => { + if (e.key === 'Enter' && !e.shiftKey) { + e.preventDefault() + submit() + } + }} + placeholder="Type a message…" + disabled={isLoading} + className="flex-1 rounded-lg border border-cyan-500/20 bg-gray-800 px-3 py-2 text-sm text-white focus:outline-none focus:ring-2 focus:ring-cyan-500/50 disabled:opacity-50" + /> + +
+
+
+ + {/* Right: memory inspector */} +
+
+
+

What's in memory

+

+ session: {sessionId ? sessionId.slice(0, 8) : '…'} +

+
+
+ + +
+
+ +
+ {/* Last recalled */} +
+

+ Last recalled (injected into the prompt) +

+ {lastRecall && lastRecall.systemPrompt ? ( +
+ 
+                  {lastRecall.systemPrompt}
+
+ {lastRecall.fragments && lastRecall.fragments.length > 0 && ( +
    + {lastRecall.fragments.map((frag, i) => ( +
  • + {frag.source}:{' '} + {frag.text} +
    • + ))} +
+ )} +
+ ) : ( +

+ Nothing recalled yet — send a follow-up question that relates to + an earlier message. +

+ )} +
+ + {/* Records */} +
+

+ Stored records ({records.length}) +

+ {records.length === 0 ? ( +

+ No memories yet — send a message to store the first turn. +

+ ) : ( +
    + {records.map((rec) => ( +
  • +
    + + {rec.kind} + + {rec.role && {rec.role}} + {rec.importance !== undefined && ( + importance {rec.importance.toFixed(2)} + )} + + {formatTime(rec.createdAt)} + +
    +

    {rec.text}

    +
    • + ))} +
+ )} +
+ + {/* Facts */} +
+

+ listFacts() ({facts.length}) +

+ {facts.length === 0 ? ( +

No facts.

+ ) : ( +
    + {facts.map((fact) => ( +
  • + {fact.source && ( + {fact.source}: + )} + {fact.text} +
    • + ))} +
+ )} +
+
+
+
+ ) +} + +export const Route = createFileRoute('/memory')({ + component: MemoryPage, +})