Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .changeset/yellow-carpets-admire.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
"@voltagent/core": patch
---

feat: add tool routing for agents with router tools, pool/expose controls, and embedding routing.

Embedding model strings also accept provider-qualified IDs like `openai/text-embedding-3-small` using the same model registry as agent model strings.

Basic embedding router:

```ts
import { openai } from "@ai-sdk/openai";
import { Agent, createTool } from "@voltagent/core";
import { z } from "zod";

const getWeather = createTool({
name: "get_weather",
description: "Get the current weather for a city",
parameters: z.object({ location: z.string() }),
execute: async ({ location }) => ({ location, temperatureC: 22 }),
});

const agent = new Agent({
name: "Tool Routing Agent",
instructions: "Use tool_router for tools. Pass the user request as the query.",
model: "openai/gpt-4o-mini",
tools: [getWeather],
toolRouting: {
embedding: openai.embedding("text-embedding-3-small"),
topK: 2,
},
});
```

Pool and expose:

```ts
const agent = new Agent({
name: "Support Agent",
instructions: "Use tool_router for tools.",
model: "openai/gpt-4o-mini",
toolRouting: {
embedding: "text-embedding-3-small",
pool: [getWeather],
expose: [getStatus],
},
});
```

Custom router strategy + resolver mode:

```ts
import { createToolRouter, type ToolArgumentResolver } from "@voltagent/core";

const resolver: ToolArgumentResolver = async ({ query, tool }) => {
if (tool.name === "get_weather") return { location: query };
return {};
};

const router = createToolRouter({
name: "tool_router",
description: "Route requests with a resolver",
embedding: "text-embedding-3-small",
mode: "resolver",
resolver,
});
```
99 changes: 99 additions & 0 deletions docs/tool-routing-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Tool Routing Implementation Plan

This document captures the agreed implementation plan for VoltAgent tool routing with router tools, tool pools, and optional embedding-based selection. We will track execution with Markdown checkboxes.

## Decisions (Locked)

- Tool routing config is supported at both Agent and VoltAgent levels (global default + per-agent override).
- Pool includes user-defined tools, provider-defined tools, and MCP tools.
- Default router execution mode is "agent"; users can override.
- Agent mode uses the same model by default; can be overridden with executionModel.
- Args are generated via generateText with structured output (schema-based output).
- Provider tool selection triggers agent-mode fallback and emits an info log.
- Embedding selection auto-activates when an embedding model or adapter is provided.
- Embedding index uses in-memory cache (extensible later).
- Router executes multiple selected tools in parallel.
- API visibility includes pool tools (not hidden); observability also includes pool tools.
- Tool approvals and hooks (tool hooks + agent onToolStart/onToolEnd) still run for pool tools.

## Scope

- Core API: ToolRoutingConfig, ToolRouterStrategy, ToolRouter, execution modes.
- Agent runtime: tool pool, router execution, tool execution path reuse.
- Embedding strategy: optional selector using EmbeddingAdapter / AiSdkEmbeddingAdapter.
- Documentation: recipe + usage examples.

## Checklist

### 1) API + Types

- [x] Add ToolRoutingConfig (global + per-agent) to types.
- [x] Define ToolRouterStrategy interface and ToolRouter types.
- [x] Define execution mode enums and router result types.
- [x] Define embedding strategy config (embedding model/adapter, topK, cache).

### 2) Registry + Defaults

- [x] Add global toolRouting defaults to AgentRegistry.
- [x] Wire VoltAgentOptions.toolRouting to registry defaults.
- [x] Add agent internal setter to apply default tool routing when unset.

### 3) Tool Pool Manager

- [x] Introduce ToolPoolManager (or extend ToolManager) to hold pool tools.
- [x] Add lookup by name (for executing pool tools).
- [x] Ensure pool supports user-defined, provider-defined, and MCP tools.

### 4) Router Tool Runtime

- [x] Implement createToolRouter (router tool factory).
- [x] Agent.prepareTools uses routers + exposed tools; pool tools are not added to LLM tools by default.
- [x] Router execution path:
- [x] Select tools via strategy.
- [x] Execute selected tools in parallel.
- [x] Return structured router output.
- [x] Ensure tool hooks + approvals run (no bypass).

### 5) Agent Mode Execution

- [x] Implement agent-mode arg generation via generateText with structured output.
- [x] Default to agent model; allow executionModel override.
- [x] Provider tool fallback:
- [x] Force toolChoice to the selected provider tool.
- [x] Log info for fallback.

### 6) Embedding Strategy (Optional)

- [x] Add embedding-based ToolRouterStrategy.
- [x] Auto-enable when embedding model/adapter is provided.
- [x] Implement tool-to-text serialization and in-memory embedding cache.
- [x] Invalidate cache when tool pool changes.

### 7) Observability + API

- [x] Include pool tools in API responses (getToolsForApi / /agents).
- [x] Add router + selection metadata to spans/logs (safeStringify).
- [x] Ensure pool tools appear in observability with correct tool names.

### 8) Tests

- [ ] Unit tests for strategy selection and router output shape.
- [ ] Agent-mode arg generation tests (schema output).
- [ ] Provider tool fallback tests.
- [ ] Embedding strategy tests (cache + selection order).

### 9) Docs + Recipes

- [x] New recipe: tool routing with router + pool.
- [x] Embedding-based routing example.
- [x] Update sidebars (if needed).

## Open Questions

- None.

## Notes

- Use safeStringify for logs and span attributes.
- Keep output schemas for router results explicit.
- Maintain compatibility with PlanAgent (router tools should work there too).
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Create a multi-agent research workflow where different AI agents collaborate to
- [Supabase](./with-supabase) — Use Supabase auth/database in tools and server endpoints.
- [Tavily Search](./with-tavily-search) — Augment answers with web results from Tavily.
- [Thinking Tool](./with-thinking-tool) — Structured reasoning via a dedicated “thinking” tool and schema.
- [Tool Routing](./with-tool-routing) — Route large tool pools through a small set of router tools.
- [Tools](./with-tools) — Author Zod‑typed tools with cancellation and streaming support.
- [VoltOps Actions + Airtable](./with-voltagent-actions) — Call VoltOps Actions as tools to create and list Airtable records.
- [Turso](./with-turso) — Persist memory on LibSQL/Turso with simple setup.
Expand Down
8 changes: 2 additions & 6 deletions examples/base/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
import { openai } from "@ai-sdk/openai";
import { Agent, Memory, VoltAgent } from "@voltagent/core";
import { LibSQLMemoryAdapter, LibSQLVectorAdapter } from "@voltagent/libsql";
import { createPinoLogger } from "@voltagent/logger";
import { honoServer } from "@voltagent/server-hono";

// Import Memory and TelemetryStore from core
import { AiSdkEmbeddingAdapter, InMemoryVectorAdapter } from "@voltagent/core";
import { LibSQLMemoryAdapter, LibSQLVectorAdapter } from "@voltagent/libsql";

// Create logger
const logger = createPinoLogger({
name: "base",
Expand All @@ -16,7 +12,7 @@ const logger = createPinoLogger({
// Create Memory instance with vector support for semantic search and working memory
const memory = new Memory({
storage: new LibSQLMemoryAdapter(),
embedding: new AiSdkEmbeddingAdapter(openai.embedding("text-embedding-3-small")),
embedding: "openai/text-embedding-3-small",
vector: new LibSQLVectorAdapter(),
});

Expand Down
11 changes: 2 additions & 9 deletions examples/github-repo-analyzer/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,4 @@
import { openai } from "@ai-sdk/openai";
import {
Agent,
AiSdkEmbeddingAdapter,
InMemoryVectorAdapter,
Memory,
VoltAgent,
} from "@voltagent/core";
import { Agent, InMemoryVectorAdapter, Memory, VoltAgent } from "@voltagent/core";
import { LibSQLMemoryAdapter } from "@voltagent/libsql";
import { createPinoLogger } from "@voltagent/logger";
import { honoServer } from "@voltagent/server-hono";
Expand All @@ -20,7 +13,7 @@ const logger = createPinoLogger({

const memory = new Memory({
storage: new LibSQLMemoryAdapter({}),
embedding: new AiSdkEmbeddingAdapter(openai.embeddingModel("text-embedding-3-small")),
embedding: "openai/text-embedding-3-small",
vector: new InMemoryVectorAdapter(),
});

Expand Down
8 changes: 2 additions & 6 deletions examples/with-auth/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
import { openai } from "@ai-sdk/openai";
import { Agent, Memory, VoltAgent } from "@voltagent/core";
import { LibSQLMemoryAdapter, LibSQLVectorAdapter } from "@voltagent/libsql";
import { createPinoLogger } from "@voltagent/logger";
import { authNext, honoServer, jwtAuth } from "@voltagent/server-hono";

// Import Memory and TelemetryStore from core
import { AiSdkEmbeddingAdapter, InMemoryVectorAdapter } from "@voltagent/core";
import { LibSQLMemoryAdapter, LibSQLVectorAdapter } from "@voltagent/libsql";

// Import tools
import { weatherTool } from "./tools/index.js";

Expand All @@ -19,7 +15,7 @@ const logger = createPinoLogger({
// Create Memory instance with vector support for semantic search and working memory
const memory = new Memory({
storage: new LibSQLMemoryAdapter(),
embedding: new AiSdkEmbeddingAdapter(openai.embedding("text-embedding-3-small")),
embedding: "openai/text-embedding-3-small",
vector: new LibSQLVectorAdapter(),
});

Expand Down
2 changes: 1 addition & 1 deletion examples/with-cloudflare-workers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ This example uses in-memory storage adapters:
```typescript
const memory = new Memory({
storage: new InMemoryStorageAdapter(),
embedding: new AiSdkEmbeddingAdapter(openai.embedding("text-embedding-3-small")),
embedding: "openai/text-embedding-3-small",
vector: new InMemoryVectorAdapter(),
});
```
Expand Down
3 changes: 1 addition & 2 deletions examples/with-lancedb/src/retriever/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import fs from "node:fs/promises";
import path from "node:path";
import { openai } from "@ai-sdk/openai";
import { type Connection, type Table, connect } from "@lancedb/lancedb";
import { type BaseMessage, BaseRetriever, type RetrieveOptions } from "@voltagent/core";
import { embed } from "ai";
Expand Down Expand Up @@ -40,7 +39,7 @@ let table: Table | null = null;

async function getEmbedding(text: string): Promise<number[]> {
const { embedding } = await embed({
model: openai.embedding("text-embedding-3-small"),
model: "openai/text-embedding-3-small",
value: text,
});
return embedding;
Expand Down
4 changes: 1 addition & 3 deletions examples/with-subagents/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
import { openai } from "@ai-sdk/openai";
import {
Agent,
AiSdkEmbeddingAdapter,
InMemoryVectorAdapter,
Memory,
VoltAgent,
Expand All @@ -21,7 +19,7 @@ const logger = createPinoLogger({

const memory = new Memory({
storage: new LibSQLMemoryAdapter(),
embedding: new AiSdkEmbeddingAdapter(openai.embeddingModel("text-embedding-3-small")),
embedding: "openai/text-embedding-3-small",
vector: new InMemoryVectorAdapter(),
});

Expand Down
53 changes: 53 additions & 0 deletions examples/with-tool-routing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
<div align="center">
<a href="https://voltagent.dev/">
<img width="1800" alt="435380213-b6253409-8741-462b-a346-834cd18565a9" src="https://github.com/user-attachments/assets/452a03e7-eeda-4394-9ee7-0ffbcf37245c" />
</a>

<br/>
<br/>

<div align="center">
<a href="https://voltagent.dev">Home Page</a> |
<a href="https://voltagent.dev/docs/">Documentation</a> |
<a href="https://github.com/voltagent/voltagent/tree/main/examples">Examples</a> |
<a href="https://s.voltagent.dev/discord">Discord</a> |
<a href="https://voltagent.dev/blog/">Blog</a>
</div>
</div>

<br/>

<div align="center">
<strong>VoltAgent is an open source TypeScript framework for building and orchestrating AI agents.</strong><br>
Escape the limitations of no-code builders and the complexity of starting from scratch.
<br />
<br />
</div>

<div align="center">

[![npm version](https://img.shields.io/npm/v/@voltagent/core.svg)](https://www.npmjs.com/package/@voltagent/core)
[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.0-4baaaa.svg)](CODE_OF_CONDUCT.md)
[![Discord](https://img.shields.io/discord/1361559153780195478.svg?label=&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2)](https://s.voltagent.dev/discord)
[![Twitter Follow](https://img.shields.io/twitter/follow/voltagent_dev?style=social)](https://twitter.com/voltagent_dev)

</div>

<br/>

<div align="center">
<a href="https://voltagent.dev/">
<img width="896" alt="VoltAgent Schema" src="https://github.com/user-attachments/assets/f0627868-6153-4f63-ba7f-bdfcc5dd603d" />
</a>

</div>

## VoltAgent: Build AI Agents Fast and Flexibly

VoltAgent is an open-source TypeScript framework for creating and managing AI agents. It provides modular components to build, customize, and scale agents with ease. From connecting to APIs and memory management to supporting multiple LLMs, VoltAgent simplifies the process of creating sophisticated AI systems. It enables fast development, maintains clean code, and offers flexibility to switch between models and tools without vendor lock-in.

## Try Example

```bash
npm create voltagent-app@latest -- --example with-tool-routing
```
38 changes: 38 additions & 0 deletions examples/with-tool-routing/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"name": "voltagent-example-with-tool-routing",
"author": "",
"dependencies": {
"@ai-sdk/openai": "^3.0.0",
"@voltagent/cli": "^0.1.21",
"@voltagent/core": "^2.1.6",
"@voltagent/logger": "^2.0.2",
"@voltagent/server-hono": "^2.0.4",
"ai": "^6.0.0",
"zod": "^3.25.76"
},
"devDependencies": {
"@types/node": "^24.2.1",
"tsx": "^4.19.3",
"typescript": "^5.8.2"
},
"keywords": [
"agent",
"ai",
"tool-routing",
"voltagent"
],
"license": "MIT",
"private": true,
"repository": {
"type": "git",
"url": "https://github.com/VoltAgent/voltagent.git",
"directory": "examples/with-tool-routing"
},
"scripts": {
"build": "tsc",
"dev": "tsx watch --env-file=.env ./src",
"start": "node dist/index.js",
"volt": "volt"
},
"type": "module"
}
Loading