Skip to content

(inference): add Fish Audio TTS #2102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
toubatbrian merged 2 commits into from
Jul 23, 2026
Merged

(inference): add Fish Audio TTS #2102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
55aa5cd
feat(inference): add Fish Audio TTS types
rosetta-livekit-bot[bot] Jul 23, 2026
c884c2a
Update fresh-fish-inference.md
toubatbrian Jul 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fresh-fish-inference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@livekit/agents': patch
---

Add Fish Audio model and option types to inference TTS.
42 changes: 40 additions & 2 deletions agents/src/inference/tts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,12 @@ export type RimeModels = 'rime/arcana' | 'rime/coda' | 'rime/mistv2' | 'rime/mis

export type XaiTTSModels = 'xai/tts-1';

export type FishAudioModels =
| 'fishaudio'
| 'fishaudio/s2.1-pro'
| 'fishaudio/s2.1-pro-free'
| 'fishaudio/s2-pro';

export interface CartesiaOptions {
emotion?: string;
/** Maximum duration of audio in seconds. */
Expand Down Expand Up @@ -142,13 +148,42 @@ export interface XaiTTSOptions {
speed?: number;
}

/** See https://docs.fish.audio/api-reference/endpoint/websocket/tts-live */
export interface FishAudioOptions {
/** Prosody speaking-rate multiplier, 1.0 is natural. */
speed?: number;
/** Prosody loudness adjustment in dB, 0 is natural. */
volume?: number;
/** Consistent output loudness; S2-Pro family only. */
normalize_loudness?: boolean;
/** Range 0-1, higher is more varied/expressive. */
temperature?: number;
/** Nucleus sampling probability mass, range 0-1. */
top_p?: number;
/** Normalize numbers/dates/abbreviations before synthesis. */
normalize?: boolean;
/** Streaming latency mode. */
latency?: 'normal' | 'balanced' | 'low';
/** Characters buffered before auto-synthesis, range 100-300. */
chunk_length?: number;
/** Max audio tokens per text chunk, default 1024. */
max_new_tokens?: number;
/** Min characters before splitting a new chunk, range 0-100. */
min_chunk_length?: number;
/** Use prior audio as context for consistency. */
condition_on_previous_chunks?: boolean;
/** Early-stopping threshold for batching, range 0-1. */
early_stop_threshold?: number;
}

type _TTSModels =
| CartesiaModels
| DeepgramTTSModels
| ElevenlabsModels
| RimeModels
| InworldModels
| XaiTTSModels;
| XaiTTSModels
| FishAudioModels;

export type TTSModels =
| CartesiaModels
Expand All @@ -157,6 +192,7 @@ export type TTSModels =
| RimeModels
| InworldModels
| XaiTTSModels
| FishAudioModels
| AnyString;

export type ModelWithVoice = `${_TTSModels}:${string}` | TTSModels;
Expand All @@ -173,7 +209,9 @@ export type TTSOptions<TModel extends TTSModels> = TModel extends CartesiaModels
? InworldOptions
: TModel extends XaiTTSModels
? XaiTTSOptions
: Record<string, unknown>;
: TModel extends FishAudioModels
? FishAudioOptions
: Record<string, unknown>;

/** Parse a model string into [model, voice]. Voice is undefined if not specified. */
export function parseTTSModelString(model: string): [string, string | undefined] {
Expand Down
Loading