Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
</p>

<p align="center">
<a href="#quick-start">Quick Start</a> · <a href="./examples/colab/">Colab</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/model_selection.md">Model selection</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/community_projects.md">Community integrations</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="https://www.funasr.com/">Deployment hub</a> · <a href="./docs/troubleshooting.md">Troubleshooting</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>
<a href="#quick-start">Quick Start</a> · <a href="./examples/colab/">Colab</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/model_selection.md">Model selection</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/community_projects.md">Community integrations</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="https://www.funasr.com/">Deployment hub</a> · <a href="./docs/troubleshooting.md">Troubleshooting</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="./integrations/openclaw/">OpenClaw</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>
</p>

---
Expand Down Expand Up @@ -113,6 +113,8 @@ results = model.generate(["audio1.wav", "audio2.wav"], language="auto")
> **Deploy as API server:** `funasr-server --device cuda` → OpenAI-compatible endpoint at localhost:8000
>
> **Use with AI agents:** [MCP Server](examples/mcp_server/) for Claude/Cursor · [OpenAI API](examples/openai_api/) for LangChain/Dify/AutoGen
>
> **Use with voice agents:** [OpenClaw realtime plugin](integrations/openclaw/) for self-hosted Talk and Voice Call transcription

### Why FunASR?

Expand Down Expand Up @@ -322,6 +324,7 @@ asset is published.

## What's new

- 2026/08/04: **OpenClaw realtime transcription integration** — the new [`openclaw-funasr`](integrations/openclaw/) source package connects self-hosted FunASR `online`, `offline`, and `2pass` WebSocket recognition to OpenClaw Talk and Voice Call. Its 8 kHz G.711 mu-law conversion, 60 ms framing, partial/final transcripts, reconnect limits, package installation, and runtime registration have been verified against OpenClaw `2026.7.2`; npm and ClawHub publication will follow the required [upstream SDK change](https://github.com/openclaw/openclaw/pull/118977).
- 2026/07/31: **v1.4.0 on PyPI** — `AutoModel` now rejects the common `vda_model` misspelling before model download and points to `vad_model`, so VAD-dependent segmentation, speaker processing, and `sentence_info` are not silently disabled. The GitHub source release also updates the legacy WebSocket file runtime: clients wait for an explicit end-of-input acknowledgement while the server flushes pending offline, online, and 2pass audio and reports finalization failures. Install the Python package with `python -m pip install -U "funasr==1.4.0"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.0)
- 2026/07/27: **v1.3.30 on PyPI** — container-formatted WAV, MP3, FLAC, OGG, MP4/M4A, and WebM audio bytes are now decoded through their codecs instead of being misread as raw PCM. OpenAI-compatible responses preserve speaker labels, VAD sentence timing survives punctuation mismatch, trusted browser clients can opt in to CORS, and vLLM VAD chunks are capped at 30 seconds. The GitHub release also includes the current prebuilt llama.cpp runtime for nine desktop and server targets. Install with `python -m pip install -U "funasr==1.3.30"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.3.30)
- 2026/07/24: **v1.3.29 hotfix on PyPI** — SenseVoice long-audio inference now returns each VAD speech region through `sentence_info` when token timestamps and a punctuation model are unavailable. Subtitle clients receive the recognized text with real millisecond start/end bounds instead of one zero-length or full-media cue. Install with `python -m pip install -U "funasr==1.3.29"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.3.29)
Expand Down
5 changes: 4 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
</p>

<p align="center">
<a href="#快速开始">快速开始</a> · <a href="./examples/colab/README_zh.md">Colab</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/model_selection_zh.md">模型选择</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/community_projects_zh.md">社区集成</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="https://www.funasr.com/">部署中心</a> · <a href="./docs/troubleshooting_zh.md">排障 FAQ</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>
<a href="#快速开始">快速开始</a> · <a href="./examples/colab/README_zh.md">Colab</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/model_selection_zh.md">模型选择</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/community_projects_zh.md">社区集成</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="https://www.funasr.com/">部署中心</a> · <a href="./docs/troubleshooting_zh.md">排障 FAQ</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="./integrations/openclaw/">OpenClaw</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>
</p>

---
Expand Down Expand Up @@ -103,6 +103,8 @@ results = model.generate(["audio1.wav", "audio2.wav"], language="auto")
> **部署为 API 服务:** `funasr-server --device cuda` → 本地 OpenAI 兼容接口 localhost:8000
>
> **接入 AI Agent:** [MCP 服务](examples/mcp_server/) 支持 Claude/Cursor · [OpenAI API](examples/openai_api/README_zh.md) 支持 LangChain/Dify/AutoGen
>
> **接入语音 Agent:** [OpenClaw 实时转写插件](integrations/openclaw/) 支持私有部署的 Talk 与 Voice Call 转写

### 为什么选 FunASR?

Expand Down Expand Up @@ -147,6 +149,7 @@ Whisper 是单个模型,**FunASR 是一个工具箱**——按场景挑模型

## 最新动态

- 2026/08/04:**OpenClaw 实时转写集成** — 新增 [`openclaw-funasr`](integrations/openclaw/) 源码包,把私有部署的 FunASR `online`、`offline` 与 `2pass` WebSocket 识别接入 OpenClaw Talk 和 Voice Call。8 kHz G.711 mu-law 转换、60 ms 分帧、partial/final 文本、重连上限、安装包与运行时注册均已基于 OpenClaw `2026.7.2` 验证;npm 与 ClawHub 发布将在所需的[上游 SDK 改动](https://github.com/openclaw/openclaw/pull/118977)合入后进行。
- 2026/07/31:**v1.4.0 已发布到 PyPI** — `AutoModel` 现在会在下载模型前拒绝常见的 `vda_model` 误拼写并明确提示使用 `vad_model`,避免依赖 VAD 的分段、说话人处理和 `sentence_info` 被静默关闭。GitHub 源码发布同时更新 legacy WebSocket 文件运行时:客户端会等待明确的输入结束确认,服务端先刷新待处理的 offline、online 与 2pass 音频,并把收尾失败返回给客户端。Python 包安装命令:`python -m pip install -U "funasr==1.4.0"`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.0)
- 2026/07/27:**v1.3.30 已发布到 PyPI** — WAV、MP3、FLAC、OGG、MP4/M4A 和 WebM 等容器格式的音频字节现在会通过对应编解码器解码,不再被误当作原始 PCM。OpenAI 兼容响应会保留说话人标签,标点不匹配时仍保留 VAD 分句时间,受信任的浏览器客户端可按需启用 CORS,vLLM 的 VAD 分段上限为 30 秒。GitHub 发布页还同时提供覆盖九种桌面和服务器目标的当前 llama.cpp 预编译运行包。安装命令:`python -m pip install -U "funasr==1.3.30"`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.3.30)
- 2026/07/24:**v1.3.29 热修复已发布到 PyPI** — SenseVoice 长音频在没有词级时间戳和标点模型时,现在会通过 `sentence_info` 返回每个 VAD 语音片段。字幕客户端可直接获得识别文本及真实的毫秒级起止时间,不再退化为零时长或覆盖整段媒体的单条字幕。安装命令:`python -m pip install -U "funasr==1.3.29"`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.3.29)
Expand Down
3 changes: 3 additions & 0 deletions integrations/openclaw/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dist/
node_modules/
*.tgz
21 changes: 21 additions & 0 deletions integrations/openclaw/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 FunASR

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
108 changes: 108 additions & 0 deletions integrations/openclaw/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# FunASR for OpenClaw

`openclaw-funasr` connects OpenClaw Talk and Voice Call to a self-hosted
[FunASR](https://github.com/modelscope/FunASR) WebSocket server. Audio stays on
infrastructure you control.

## Capabilities

- FunASR `online`, `offline`, and `2pass` recognition modes
- OpenClaw's 8 kHz G.711 mu-law input converted to 16 kHz PCM
- Official 60 ms FunASR audio frames and the required `binary` WebSocket subprotocol
- Partial and final transcripts, hotwords, inverse text normalization, and optional Bearer auth
- Bounded audio queues and transcript retention inherited from the OpenClaw session helper

## Compatibility

The first release requires OpenClaw `>=2026.7.2` with
[openclaw/openclaw#118977](https://github.com/openclaw/openclaw/pull/118977).
Publishing to npm and ClawHub is intentionally gated on that SDK change landing
in an OpenClaw release; the compatibility floor will be pinned to the exact
release before publication.

## Start FunASR

Run the official FunASR WebSocket server from this repository. The server must
be reachable from the OpenClaw Gateway and negotiate the `binary` subprotocol.
See [runtime/python/websocket](../../runtime/python/websocket) for Docker,
model, TLS, and client examples.

The default server endpoint is commonly `ws://127.0.0.1:10095`, but the plugin
does not assume a URL. Set it explicitly in OpenClaw or through
`FUNASR_WS_URL`.

## Build and test

```bash
cd integrations/openclaw
npm ci
OPENCLAW_ROOT=/absolute/path/to/openclaw npm test
npm pack
```

`OPENCLAW_ROOT` must point to an OpenClaw checkout containing #118977. The test
suite exercises the real shared WebSocket session helper, including protocol
negotiation, audio conversion, 60 ms framing, finalization, reconnect handling,
bounded transcript state, and package metadata.

Install the packed artifact before publication:

```bash
openclaw plugins install npm-pack:/absolute/path/to/openclaw-funasr-0.1.0.tgz --force
openclaw plugins inspect funasr --runtime --json
```

## Configure OpenClaw

FunASR provider options live under Voice Call's generic streaming provider map.
The same provider map is currently used by Talk transcription sessions.

```json5
{
plugins: {
entries: {
"voice-call": {
config: {
streaming: {
enabled: true,
provider: "funasr",
providers: {
funasr: {
baseUrl: "ws://127.0.0.1:10095",
mode: "2pass",
hotwords: "OpenClaw,FunASR",
useItn: true
}
}
}
}
}
}
}
}
```

Options:

| Option | Default | Description |
| --- | --- | --- |
| `baseUrl` | `FUNASR_WS_URL` | `http`, `https`, `ws`, or `wss` endpoint |
| `apiKey` | `FUNASR_API_KEY` | Optional Bearer token for a reverse proxy |
| `mode` | `2pass` | `online`, `offline`, or `2pass` |
| `hotwords` | empty | FunASR hotword string |
| `useItn` | `true` | Enable inverse text normalization |

Embedded credentials in `baseUrl` are rejected. Use `apiKey` or
`FUNASR_API_KEY` instead.

## Verified behavior

The end-to-end proof used the official FunASR server, a public 5.55 second
16 kHz sample converted to the same 8 kHz mu-law frames emitted by OpenClaw,
and the plugin's production adapter. It streamed 278 audio frames, produced
seven partial transcripts, returned the exact final transcript, and completed
without client errors.

## License

MIT
19 changes: 19 additions & 0 deletions integrations/openclaw/openclaw.plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"id": "funasr",
"name": "FunASR Realtime Transcription",
"description": "Self-hosted realtime speech recognition through FunASR",
"activation": {
"onStartup": false
},
"enabledByDefault": true,
"contracts": {
"realtimeTranscriptionProviders": [
"funasr"
]
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {}
}
}
Loading
Loading