Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ write yourself.
**Launcher routing is explicit.** By default, launchers use the built-in
LLM-classifier router, which you tune with `--weak-model`, `--classifier-model`,
`--profile`, and `--classifier-min-confidence`. Use `--model X` for
single-model passthrough. The `--routing-profiles FILE` path is deprecated and
single-model route. The `--routing-profiles FILE` path is deprecated and
remains only for launcher-owned legacy bundles.

## Features
Expand Down Expand Up @@ -72,11 +72,11 @@ switchyard launch openclaw --model openai/gpt-4o-mini --api-key "$OPENROUTER_API
```

Each launcher starts a local proxy, points the agent at it, and shuts the proxy
down when the agent exits. Use `--model` for single-model passthrough. The
down when the agent exits. Use `--model` for single-model route. The
deprecated `--routing-profiles` flag remains for launcher-owned legacy bundles:

```bash
switchyard launch claude --model openai/gpt-4o-mini --base-url https://openrouter.ai/api/v1 # single-model passthrough
switchyard launch claude --model openai/gpt-4o-mini --base-url https://openrouter.ai/api/v1 # single-model route
switchyard --routing-profiles routes.yaml -- launch claude # legacy route bundle
```

Expand Down
14 changes: 7 additions & 7 deletions docs/cli_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,12 @@ launchers:

| Flag | Purpose |
|---|---|
| `--model ID` | Single-model passthrough. Every request is rewritten to `model=ID` and forwarded to `--base-url`. |
| `--routing-profiles PATH` | Deprecated path to a routing-profile YAML bundle. Each entry under `routes:` builds its own chain. Public route types are `model`, `passthrough`, `random_routing`, `stage_router`, and `deterministic`. Falls back to the path persisted by `switchyard --routing-profiles PATH -- configure` when omitted. |
| `--model ID` | Single-model route. Every request is rewritten to `model=ID` and forwarded to `--base-url`. |
| `--routing-profiles PATH` | Deprecated path to a routing-profile YAML bundle. Each entry under `routes:` builds its own chain. Public route types are `model`, `random_routing`, `routellm`, `latency_service`, `stage_router`, `deterministic`, and `plan_execute`; use `model` for standalone target execution. Falls back to the path persisted by `switchyard --routing-profiles PATH -- configure` when omitted. |

On the launchers, the two flags are mutually exclusive: pass one or the other, not both.

- `--model ID`: single-model passthrough. Any model id from `GET /v1/models` is accepted; every request is rewritten to `model=ID` and forwarded upstream.
- `--model ID`: single-model route. Any model id from `GET /v1/models` is accepted; every request is rewritten to `model=ID` and forwarded upstream.
- `--routing-profiles PATH`: loads a multi-chain YAML bundle; the first declared route becomes the initial model.

For legacy route-bundle `type: deterministic` routes, the `classifier:` block also accepts:
Expand Down Expand Up @@ -277,7 +277,7 @@ switchyard [--routing-profiles PATH] launch claude [--model ID]
```

When neither CLI routing flag is passed, `launch claude` uses a saved routing bundle
first, then a saved `claude.model` as single-model passthrough. The built-in
first, then a saved `claude.model` as single-model route. The built-in
LLM-as-classifier route is the default only when neither a bundle nor a single-model
default is resolved.

Expand Down Expand Up @@ -310,7 +310,7 @@ default is resolved.
**Examples**

```bash
# Single-model passthrough
# Single-model route
switchyard launch claude --model openai/gpt-4o-mini

# Use a routing bundle
Expand Down Expand Up @@ -343,7 +343,7 @@ switchyard [--routing-profiles PATH] launch codex [--model ID]
```

When neither CLI routing flag is passed, `launch codex` uses a saved routing bundle
first, then a saved `codex.model` as single-model passthrough. The built-in
first, then a saved `codex.model` as single-model route. The built-in
LLM-as-classifier route is the default only when neither a bundle nor a single-model
default is resolved. `--weak-model`, `--classifier-model`, `--profile`, and
`--classifier-min-confidence` tune only that built-in route. `CODEX_ARGS` are forwarded
Expand Down Expand Up @@ -379,7 +379,7 @@ switchyard [--routing-profiles PATH] launch openclaw [--model ID]
```

When neither CLI routing flag is passed, `launch openclaw` uses a saved routing bundle
first, then a saved `openclaw.model` as single-model passthrough. The built-in
first, then a saved `openclaw.model` as single-model route. The built-in
LLM-as-classifier route is the default only when neither a bundle nor a single-model
default is resolved. `--weak-model`, `--classifier-model`, `--profile`, and
`--classifier-min-confidence` tune only that built-in route.
Expand Down
23 changes: 10 additions & 13 deletions docs/core_concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,19 +61,16 @@ faster. A tier is a role you hand to a target rather than a fixed property of it
so the same target can be the strong tier in one profile and the weak tier in
another.

A profile's `type` sets the strategy. `passthrough` sends everything to one
target with no routing. `random-routing` splits traffic on a fixed probability.
`llm-routing` asks a classifier model to pick a tier for each turn. `stage_router`
escalates from weak to strong when request signals call for it. The
[Routing Overview](routing_algorithms/overview.md) covers when to use each and
how to tune it.

!!! warning "There is no `type: model`"
The current profile config does not have a `type: model`. To expose a single
model, point clients at a target ID directly, or add a `passthrough` profile
when you want a second name for it. `type: model` survives only in the
deprecated `--routing-profiles` bundles used by the launcher compatibility
path.
A profile's `type` sets the strategy. `random-routing` splits traffic on a
fixed probability. `llm-routing` asks a classifier model to pick a tier for each
turn. `stage_router` escalates from weak to strong when request signals call for
it. The [Routing Overview](routing_algorithms/overview.md) covers when to use
each and how to tune it.

!!! note "Single-model routes"
In profile configs, target IDs are directly addressable model IDs. In the
deprecated `--routing-profiles` route-bundle format, use `type: model` for a
standalone target alias.

Session affinity, or sticky routing, pins a conversation to one tier so later
turns reuse it instead of being classified again. It belongs to `llm-routing`
Expand Down
7 changes: 4 additions & 3 deletions docs/guides/agent_launchers.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ single model.

## Override the default route

Use `--model` for a single-model passthrough session:
Use `--model` for a single-model route session:

```bash
switchyard launch claude --model openai/gpt-4o-mini
Expand Down Expand Up @@ -100,8 +100,9 @@ simultaneously.
2. Switch back to the validated default route. If that works, the issue is
model-specific.

In passthrough mode, Switchyard forwards the empty response as-is. Multi-target
routing profiles treat empty completions as errors and surface them explicitly.
For a single-model route, Switchyard forwards the empty response as-is.
Multi-target routing profiles treat empty completions as errors and surface them
explicitly.

### Claude Code with MCP tools

Expand Down
3 changes: 1 addition & 2 deletions docs/operations/context_window.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ also overflows, the request fails with a 400 in the client's inbound wire
format.

Any multi-target route (stage-router, random_routing, or deterministic) supports
this. Set `fallback_target_on_evict` on the route. Single-target routes
(`type: passthrough`, `type: model`) have no alternative target, so the original
this. Set `fallback_target_on_evict` on the route. Single-target `type: model` routes have no alternative target, so the original
overflow propagates unchanged.

## Configuration
Expand Down
40 changes: 12 additions & 28 deletions docs/routing_algorithms/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,37 +71,24 @@ profile and target appears on `GET /v1/models`:

```yaml
profiles:
fast:
type: passthrough
target: weak

smart:
type: random-routing
strong: strong
weak: weak
strong_probability: 0.3
```

Use the profile ID to select policy behavior (`fast` or `smart`) and a target ID
to bypass routing (`weak` or `strong`).

## Direct targets and passthrough aliases
Use the profile ID to select policy behavior (`smart`) and a target ID to bypass
routing (`weak` or `strong`).

For new profile configs, use one public model concept:
## Direct targets and model routes

- Select a target ID directly when its configured name is the client-facing
name you want.
- Add a `type: passthrough` profile only when you need another stable alias for
that target, such as `fast` above.
For new profile configs, select a target ID directly when you want to call one
upstream model without a routing policy. The target ID is already a public model
ID on `GET /v1/models`.

Both choices call the same single target. There is no `type: model` profile in
the current profile schema.

The deprecated `--routing-profiles` route-bundle format has a separate legacy
distinction: `type: model` registers one explicit alias without model discovery,
while `type: passthrough` queries the upstream model catalog and registers the
discovered models too. That distinction applies only to legacy `routes:`
bundles used by the launcher compatibility path.
The deprecated `--routing-profiles` route-bundle format uses `type: model` for
standalone target aliases.

## Self-hosted targets

Expand All @@ -126,15 +113,12 @@ targets:
model: my-rl-qwen
format: openai

profiles:
fast-local:
type: passthrough
target: local-weak
```

Reference `local-weak` from any routing profile field that accepts a target ID,
including `strong`, `weak`, `target`, or `targets`. Switchyard does not start or
manage the model server; it only sends requests to the configured endpoint.
Clients can select `local-weak` directly as a model ID, or reference it from any
routing profile field that accepts a target ID, including `strong`, `weak`,
`target`, or `targets`. Switchyard does not start or manage the model server; it
only sends requests to the configured endpoint.

## How session affinity composes

Expand Down
2 changes: 1 addition & 1 deletion docs/routing_algorithms/stage_router_routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ curl -s http://localhost:4000/v1/stats > routing_stats_final.json

## When *not* to use stage-router

- **Single-model deployments.** Use a plain passthrough profile instead.
- **Single-model deployments.** Select a target ID directly instead.
- **Probabilistic A/B splits.** Use
[Random Routing](random_routing.md) (`type: random-routing` in profile configs).
The stage-router's signals are wasted on a fixed traffic ratio.
Expand Down
2 changes: 1 addition & 1 deletion examples/minimal.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ async def main():
print("Set it with: export OPENAI_API_KEY='sk-...'")
return

# Create a passthrough proxy that forwards to OpenAI
# Create a single-model route that forwards to OpenAI
switchyard = SwitchyardRecipes.passthrough_recipe(
api_key=api_key,
base_url="https://api.openai.com/v1",
Expand Down
6 changes: 1 addition & 5 deletions examples/profiles.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#
# Each profile id and each target id is exposed as a model on GET /v1/models, so
# a client (or `switchyard launch claude --model <id>`) selects a profile by id:
# fast passthrough to the efficient model
# target ids direct single-model routes
# smart-stage-router signal-based stage-router routing between the efficient and capable models
# the model ids the targets, addressable directly
#
Expand All @@ -30,10 +30,6 @@ targets:
format: openai

profiles:
fast:
type: passthrough
target: moonshotai/kimi-k2.6

smart-stage-router:
type: stage_router
capable: anthropic/claude-opus-4.7
Expand Down
8 changes: 2 additions & 6 deletions examples/python_profile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
#
# switchyard serve --config examples/python_profile.yaml
#
# `direct` is a Rust passthrough; `smart` is the shipped Python `header-routing`
# profile (switchyard/lib/profiles/header_routing.py), which picks a tier from the
# `smart` is the shipped Python `header-routing` profile
# (switchyard/lib/profiles/header_routing.py), which picks a tier from the
# x-switchyard-tier request header and delegates the call to a Rust target
# backend. Both share the same resolved targets.

Expand All @@ -31,10 +31,6 @@ targets:
format: openai

profiles:
direct:
type: passthrough
target: weak

smart:
type: header-routing
strong: strong
Expand Down
6 changes: 0 additions & 6 deletions examples/route.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,13 @@ routes:
type: model
target: moonshotai/kimi-k2.6

openrouter-catalog: # `type: passthrough` — auto-discovers available models via GET /v1/models
type: passthrough

random-router: # `type: random_routing` — weighted coin
type: random_routing
strong: anthropic/claude-opus-4.7
weak: moonshotai/kimi-k2.6
strong_probability: 0.3
fallback_target_on_evict: strong

bench: # `type: noop` — for benchmarking
type: noop

llm-classifier: # `type: deterministic` — LLM-as-classifier
type: deterministic
profile: coding_agent # general | coding_agent | openclaw
Expand Down
10 changes: 5 additions & 5 deletions switchyard/cli/launch_command.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ def _is_deterministic_launch(
"""Return ``True`` when the resolved launch will use deterministic routing.

For ``claude`` / ``codex``: deterministic is the implicit default when
no single-model passthrough (CLI ``--model`` or saved
no single-model route (CLI ``--model`` or saved
``configured_route.model``) and no routing-profiles bundle (CLI
``--routing-profiles`` or saved bundle) is in play. Pass
``route=None`` to check before the route is resolved (e.g. inside
Expand Down Expand Up @@ -474,7 +474,7 @@ def cmd_launch_claude(args: argparse.Namespace) -> None:
validated coding-agent trio (Claude Opus 4.7 + Nemotron-3 Super
+ DeepSeek V4 Flash classifier). Override individual tiers with
``--weak-model`` / ``--classifier-model`` / ``--profile``.
* ``--model X`` — single-tier passthrough to X.
* ``--model X`` — single-model route to X.
* ``--routing-profiles FILE`` (global flag) — YAML-driven multi-chain table.
``--model`` is optional; falls back to the first YAML route.

Expand Down Expand Up @@ -536,7 +536,7 @@ def cmd_launch_claude(args: argparse.Namespace) -> None:
configured_route = LaunchRouteConfig()
route = route_from_launch_args(args, configured_route)
# Deterministic LLM-classifier routing is the implicit default for
# ``launch claude`` when neither single-model passthrough nor a
# ``launch claude`` when neither single-model route nor a
# routing-profiles bundle is in play.
deterministic = _is_deterministic_launch(
target="claude", args=args, route=route, routing_profiles=routing_profiles,
Expand Down Expand Up @@ -735,7 +735,7 @@ def cmd_launch_codex(args: argparse.Namespace) -> None:
configured_route = LaunchRouteConfig()
route = route_from_launch_args(args, configured_route)
# Deterministic LLM-classifier routing is the implicit default for
# ``launch codex`` when neither single-model passthrough nor a
# ``launch codex`` when neither single-model route nor a
# routing-profiles bundle is in play.
deterministic = _is_deterministic_launch(
target="codex", args=args, route=route, routing_profiles=routing_profiles,
Expand Down Expand Up @@ -938,7 +938,7 @@ def cmd_launch_openclaw(args: argparse.Namespace) -> None:
configured_route = LaunchRouteConfig()
route = route_from_launch_args(args, configured_route)
# LLM-classifier routing is the implicit default for ``launch openclaw``
# when neither single-model passthrough nor a routing-profiles bundle is
# when neither single-model route nor a routing-profiles bundle is
# in play.
deterministic = _is_deterministic_launch(
target="openclaw", args=args, route=route, routing_profiles=routing_profiles,
Expand Down
12 changes: 6 additions & 6 deletions switchyard/cli/launchers/claude_code_launcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@

Implements two UXes on top of a single chain-agnostic runner:

* ``switchyard launch claude --model <name>`` — single-model
passthrough. Spin up an in-process V2 passthrough proxy on a free
* ``switchyard launch claude --model <name>`` — single-model route.
Spin up an in-process V2 proxy on a free
local port, probe the backend for native ``POST /v1/messages``
support, build the matching chain
(:class:`AnthropicNativeBackend` or :class:`OpenAiNativeBackend`
Expand Down Expand Up @@ -297,7 +297,7 @@ def _run_claude_with_switchyard(

Takes a pre-built :class:`Switchyard` and a display-only model
name. The caller decides what's in the chain (single-model
passthrough, random routing across a preset, latency-service
single-model routing, random routing across a preset, latency-service
pool, …) — this function owns only the uvicorn-in-a-thread +
``claude`` supervision + env-var injection boilerplate that is
identical across those modes.
Expand Down Expand Up @@ -419,15 +419,15 @@ def launch_claude(
routing_profiles: str | None = None,
rl_log_dir: Path | None = None,
) -> int:
"""Start a passthrough proxy and run ``claude`` against it.
"""Start a single-model proxy and run ``claude`` against it.

Single-model UX — ``model`` seeds Claude Code's session, while the
proxy preserves any model Claude Code sends later so ``/model``
selections remain effective.

When ``routing_profiles`` is given, the launcher builds a
:class:`RouteTable` instead of a single chain: ``model`` is
registered as a tier passthrough, then every entry from the YAML file
registered as a direct tier route, then every entry from the YAML file
is merged on top via :meth:`RouteTable.register`. YAML routes
win on id conflict. The launcher's stats accumulator is threaded into
both the launcher's chain and the YAML loader so all traffic records
Expand Down Expand Up @@ -498,7 +498,7 @@ def launch_claude_deterministic_routing(
The LLM-classifier chain (classifier → tier selector → per-tier dispatch)
is wrapped in a :class:`RouteTable` whose virtual model is the
deterministic routing target. Configured strong + weak models register
as direct passthrough chains so the Claude Code ``/model`` picker can
as direct model-route chains so the Claude Code ``/model`` picker can
override the routing policy. The classifier is not user-selectable.
"""
from switchyard.cli.model_catalog.model_discovery import fetch_model_ids
Expand Down
Loading
Loading