Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
036d132
feat(macos): read cursor context from the host app (module only, not …
Aug 2, 2026
4e3a7cd
feat(polish): feed the cursor's surrounding text to LLM polish (defau…
Aug 2, 2026
2862e16
feat(macos): detect when the user hand-corrects text we just inserted
Aug 2, 2026
368ccf7
feat: turn detected hand-corrections into dictionary entries
Aug 2, 2026
81f28e1
feat(settings): add a cursor-context probe to the debug tools
Aug 2, 2026
5b70fbf
docs: add the cursor-context install test plan
Aug 2, 2026
1dd40c5
fix(macos): anchor the edit-watch baseline after the insertion lands
Aug 2, 2026
046965b
fix(macos): only judge an edit once the user has stopped typing
Aug 3, 2026
497896d
fix(macos): end an edit when the caret leaves it, not when a timer ex…
Aug 3, 2026
45c81e6
fix(macos): don't mistake typing's own caret event for the user leaving
Aug 3, 2026
fd5bac4
refactor: learn vocabulary entries, not correction rules
Aug 3, 2026
a4b9d08
feat: ask about a new word on a card, where the capsule sits
Aug 4, 2026
4dcebd9
fix(macos): give the capsule window back after the card closes
Aug 4, 2026
82cf687
fix(daily): 把光标上下文系列接到新版 beta 上
bigsongeth Aug 4, 2026
572a01f
chore(macos): quiet the edit-watch diagnostics down to debug
Aug 4, 2026
39917e4
docs: rewrite the test plan for what the feature actually became
Aug 4, 2026
7c2ebda
feat(vocab): 每条建议都要你点勾才入库,卡片挪到右下角
bigsongeth Aug 4, 2026
4342f3c
fix(asr): 别让刚添加的词把常用词挤出 ASR 词表预算
bigsongeth Aug 4, 2026
04fdcd8
fix(vocab): 学来的词条追加到词典末尾,不占 ASR 预算的保底席位
bigsongeth Aug 4, 2026
6829779
docs: README 补上「会自己长的词典」,并兑现它已经承诺过的那句话
bigsongeth Aug 4, 2026
e284d01
fix: 修 Android 编译,并把 AX 抓取挪出 tokio worker
bigsongeth Aug 4, 2026
09e97f0
fix(cursor-context): 关掉开关立刻解除观察器,并给「Drop 即停止」钉一个测试
bigsongeth Aug 4, 2026
2f14dc6
fix(cursor-context): 没有光标时别当成光标在开头;词条长度两侧都要量
bigsongeth Aug 4, 2026
1a01bfa
fix(cursor-context): 已解除的观察器不得再上报——迟到一条就能把胶囊弄没
bigsongeth Aug 4, 2026
86d7d38
fix(cursor-context): 解除观察器收口成一个入口;扩长按 trim 后的长度判
bigsongeth Aug 4, 2026
148f451
fix(vocab-card): 听写进行中一律不弹卡片——判据下沉到碰窗口前的最后一刻
bigsongeth Aug 4, 2026
84d066e
Merge origin/beta into feat/cursor-context-vocab
bigsongeth Aug 4, 2026
fbe24ce
docs(cursor-context): 写明 edit_is_within_typed_text 是按内容匹配,不按位置
bigsongeth Aug 4, 2026
63e9f3c
fix(cursor-context): 手改观察器也必须过安全闸门——取焦点元素收口成一个入口
bigsongeth Aug 5, 2026
380b7af
fix(cursor-context): 改完按回车不该把整句都算成改动
bigsongeth Aug 5, 2026
a5c2a05
fix(cursor-context): 学不到的改动不许吃掉基线——「删掉重打」这条路修好了
bigsongeth Aug 5, 2026
23637dd
fix(cursor-context): 回调也要看停止 flag,不能只有循环看
bigsongeth Aug 5, 2026
d3f9d3b
fix(cursor-context): CF 的 Boolean 是 unsigned char,别用 Rust bool 接
bigsongeth Aug 5, 2026
feac7a4
fix(cursor-context): 闸门改用元素自己的身份判定,不再信「谁在最前面」
bigsongeth Aug 5, 2026
45d599b
docs(cursor-context): 写明「改完词接着往下写」学不到,以及为什么这版不收紧
bigsongeth Aug 5, 2026
de608c6
fix(cursor-context): 去重挡掉的改动也要推进基线,否则后面的纠正全被带偏
bigsongeth Aug 5, 2026
f907c97
fix(cursor-context): 确认不了元素归属就不读——闸门必须失败关闭
bigsongeth Aug 5, 2026
1dde704
Merge origin/beta into feat/cursor-context-vocab
bigsongeth Aug 5, 2026
2f8c276
fix(selection): 合并时把 beta 的按平台 current_front_app 又带了回来,Linux/Windows …
bigsongeth Aug 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,9 @@ That is what **authorizing the infrastructure once, at launch** means: on first

## ✨ What's new

Two capabilities that sediment yet more of the coordination you used to repeat every day into defaults:
Capabilities that sediment yet more of the coordination you used to repeat every day into defaults:

- 📖 **A dictionary that learns.** Until now the dictionary only knew what you typed into it by hand. Now, when you correct a word OpenLess just wrote, it asks — once, on a small card — whether to remember it, and one click puts it in. Paired with **cursor context** (opt-in, macOS), which lets the polish model read what you are writing around your cursor, OpenLess stops being a transcriber that guesses at homophones and starts being an input method that knows your words. Every suggestion is reviewed by you; nothing is learned silently.
- 🎨 **Style Pack Marketplace.** OpenLess no longer ships a single fixed "polish" voice. Build your own **style packs** with custom system prompts, switch between them with a hotkey, and **install community packs in one click** — or publish your own to share. When a style is tuned to your exact task (cold emails, commit messages, 小红书 posts, formal reports, your team's tone), the output is not merely cleaner — it is *noticeably better*, because the model is finally writing the way you intend.
- ⚡ **Streaming insertion.** Text now flows to your cursor **character by character** as it is polished, rather than making you wait for the complete result. Perceived latency drops sharply, so dictation feels nearly as fast as thinking — and it automatically falls back to a one-shot paste when an application cannot accept streamed keystrokes.

Expand Down Expand Up @@ -167,7 +168,7 @@ OpenLess does one thing: it **turns speech into usable written text — AI promp

| Tool | Form | How OpenLess differs |
| --- | --- | --- |
| [Typeless](https://www.typeless.com/) | Closed-source macOS / Windows / iOS, subscription | Open source; explicit AI-prompt mode; bring-your-own ASR + LLM; data and dictionary stay on your machine |
| [Typeless](https://www.typeless.com/) | Closed-source macOS / Windows / iOS, subscription | Open source; explicit AI-prompt mode; bring-your-own ASR + LLM; data and dictionary stay on your machine — including what the dictionary learns from your corrections, which is never uploaded and never added without your confirmation |
| [Wispr Flow](https://wisprflow.ai) | Closed-source macOS / Windows, subscription | Open source; bring-your-own ASR + LLM; transparent prompt-handling rules |
| [Lazy](https://heylazy.com) | Closed-source notes / capture tool | Not a notes container — inserts straight into any input field |
| [Superwhisper](https://superwhisper.com) | Closed-source macOS, subscription | Open source; cloud ASR today, local ASR on the roadmap |
Expand Down Expand Up @@ -340,7 +341,16 @@ The dictionary handles your proper nouns, product names, names of people, and ne
- Manually adding the correct spelling, a category, and notes. You do not need to maintain misspellings or context hints.
- Enabled entries are sent to the ASR provider that supports hotwords (Volcengine `context.hotwords`, StepFun `hotwords`, Whisper-compatible `prompt` — except ZenMux, whose JSON protocol does not carry `prompt`/`hotwords`, Bailian vocabulary ID) so they are recognized correctly during transcription. iFlytek realtime ASR has no request-level hotword parameter — configure personalized hotwords in the iFlytek console instead.
- Entries are also injected into the polish prompt: the model decides per sentence whether to substitute. If "Cloud" clearly refers to the AI product `Claude` in context, it is corrected; if it genuinely means cloud computing, it is left as is.
- The app auto-learns candidate corrections such as `Claude`, `ChatGPT`, and `OpenLess` from your history and offers them later.
- **The dictionary learns from you.** When you hand-correct a word OpenLess just typed, a card appears asking whether to remember it. One ✓ and it is in — no settings page, no forms. Every suggestion is reviewed by you: nothing is ever added silently. Requires the opt-in **cursor context** setting below, and is macOS-only for now.
- **Entries that earn their keep get priority.** The hotword budget sent to ASR providers is finite (a few hundred characters). Entries are ranked by hit count, with a few reserved seats for words you just added by hand, so the terms you actually use keep their place instead of being pushed out by whatever you added most recently.

### Cursor context (opt-in, macOS)

Settings → Privacy → Data storage → **Cursor context**. Off by default.

When on, each dictation reads a few hundred characters around your cursor **in the app you are writing in** and sends them with the polish request, so the model knows what you are writing about. Chinese homophones (接口/借口, 大鱼/大禹) are indistinguishable to an acoustic model but obvious from context. This is also what makes dictionary learning possible: OpenLess can only notice that you fixed a word if it can see the text it just typed.

What it never reads: password fields, macOS Secure Input, password managers, and terminals — those are blocked before a single accessibility call is made. While the setting is off, no accessibility calls happen at all and the prompt is byte-for-byte identical to a build without the feature.

The main window is organized as Home / History / Dictionary / Settings. The Dictionary tab opens a separate editor window when you click "New". The Home tab shows total dictation time, total characters, average characters per minute, estimated time saved, and dictionary participation statistics.

Expand Down
16 changes: 13 additions & 3 deletions README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,9 @@ OpenLess 做的不是“更快的听写”,而是**消灭“想法 → 干净文

## ✨ 更新亮点

下面两项能力,把过去每天都要重复的协调,进一步沉降成了默认规则:
下面这些能力,把过去每天都要重复的协调,进一步沉降成了默认规则:

- 📖 **会自己长的词典。** 在此之前,词典里只有你亲手敲进去的东西。现在,当你改掉 OpenLess 刚写出来的某个词,它会在屏幕角落弹一张小卡片问一声要不要记住,点一下就进去了。配合**光标上下文**(需手动开启,仅 macOS)——让润色模型读得到你光标周围正在写的内容——OpenLess 不再是一个靠猜同音词的转写工具,而开始成为**一个认得你的词的输入法**。每一条建议都由你过目,没有任何东西是悄悄学走的。
- 🎨 **风格包市场(Style Pack Marketplace)。** OpenLess 不再只内置一种固定的“润色”语气。你可以用自定义系统提示词构建自己的**风格包**,用快捷键在它们之间切换,并**一键安装社区分享的风格包**——也可以发布自己的与他人分享。当风格与你的具体任务高度契合(冷启动邮件、commit message、小红书文案、正式报告、团队语气)时,产出的文本不只是更干净,而是*明显更好*,因为模型终于在按你真正想要的方式写作。
- ⚡ **流式插入。** 文本现在会随润色**逐字符**写入光标,而不必等待完整结果生成。感知延迟大幅下降,听写几乎和思考一样快——当某个应用无法接受流式按键时,它会自动回退为一次性粘贴。

Expand Down Expand Up @@ -167,7 +168,7 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提

| 工具 | 形态 | OpenLess 的不同之处 |
| --- | --- | --- |
| [Typeless](https://www.typeless.com/) | 闭源 macOS / Windows / iOS,订阅制 | 开源;显式的 AI 提示词模式;自带 ASR + LLM;数据与词典留在本机 |
| [Typeless](https://www.typeless.com/) | 闭源 macOS / Windows / iOS,订阅制 | 开源;显式的 AI 提示词模式;自带 ASR + LLM;数据与词典留在本机——包括词典从你的手改中学到的东西,不上传,也不会在你确认之前加进去 |
| [Wispr Flow](https://wisprflow.ai) | 闭源 macOS / Windows,订阅制 | 开源;自带 ASR + LLM;文本处理规则透明 |
| [Lazy](https://heylazy.com) | 闭源的笔记 / 速记工具 | 不是笔记容器——直接插入到任意输入框 |
| [Superwhisper](https://superwhisper.com) | 闭源 macOS,订阅制 | 开源;目前云端 ASR,本地 ASR 在路线图中 |
Expand Down Expand Up @@ -340,7 +341,16 @@ OpenLess 的润色模型只重塑文本。它不回答问题、不执行任务
- 手动添加正确拼写、分类与备注。你无需维护错误拼写或上下文提示。
- 启用的条目作为 Volcengine ASR 的 `context.hotwords` 发送,以便在转写时被正确识别。
- 条目同样注入润色提示词:模型逐句判断是否替换。如果“Cloud”在上下文中明显指 AI 产品 `Claude`,就会被纠正;如果它确实指云计算,则保持原样。
- 应用会从你的历史中自动学习候选纠正(如 `Claude`、`ChatGPT`、`OpenLess`),并在之后向你推荐。
- **词典会自己长。** 当你手动改掉 OpenLess 刚打出来的某个词,屏幕角落会弹一张小卡片问你要不要记住它。点一下勾就进去了——不用打开设置页,不用填表。**每一条都由你过目,没有任何东西是悄悄加进去的。** 需要开启下面的「光标上下文」,目前仅 macOS。
- **真正在用的词优先。** 发给 ASR 的热词预算是有限的(几百字符)。条目按命中次数排序,并给刚手动添加的词留几个保底席位——这样你天天在用的那些词不会被「最近刚加的」挤出去。

### 光标上下文(需手动开启,仅 macOS)

设置 → 隐私 → 数据存储 → **光标上下文**。默认关闭。

开启后,每次听写会读取**你正在写的那个应用里**光标附近的几百个字,随润色请求一起发出,让模型知道你在写什么。中文同音词(接口/借口、大鱼/大禹)声学模型分不出来,但上下文能分。词典的自我学习也建立在这之上——OpenLess 只有看得见自己刚打出去的文字,才可能发现你把某个词改掉了。

**永远不读的地方**:密码输入框、macOS Secure Input、密码管理器、终端——这些在发出任何一次辅助功能调用之前就被拦下。开关关闭时,一次辅助功能调用都不会发生,提示词与没有这个功能的版本逐字节相同。

主窗口组织为 首页 / 历史 / 词典 / 设置。点击“新建”时,词典页会打开一个独立的编辑窗口。首页展示总听写时长、总字数、平均每分钟字数、估算节省的时间,以及词典参与统计。

Expand Down
186 changes: 186 additions & 0 deletions openless-all/app/docs/cursor-context-test-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# 光标上下文 + 手改学习 —— 装机测试案例

装的版本:`local/daily`。开关在 **设置 → 隐私 → 数据存储 → 光标上下文(实验)**,默认关。

盯日志:

```bash
tail -f ~/Library/Logs/OpenLess/openless.log | grep -E "cursor-context|cursor context|vocab"
```

重装后辅助功能授权会短暂失效(ad-hoc 签名每次构建 cdhash 都变),但 app 自己会恢复——实测重试到第 23 次时自己起来,约 70 秒。看到 `hotkey listener installed` 就能用了;万一一直刷 `CGEventTapCreate 失败`,去 系统设置 → 隐私与安全性 → 辅助功能 → OpenLess 关掉再打开。

---

## 这个功能干什么

开关打开后,每次听写会读**你当时正在写的那个文档里、光标附近的几百个字**,跟着请求一起发给 LLM。这样它知道你在写什么,「接口」不会写成「借口」。

落字之后它还会盯一小会儿:如果你手动改了它插进去的某个词,那个词可能进你的词汇表。

**只写词汇表,不写纠正规则。** 词条是提示(送给 ASR、进润色 prompt 让 LLM 带上下文判断),错了最多是没帮上忙;纠正规则是字面替换,错了是静默的、全局的。学来的东西配不上后者那份权力。

---

## 1. AX 覆盖率 —— 不用开口说话

**设置 → 高级 → 调试工具 → 光标上下文探针**

点「探测(5 秒后)」→ 切到目标 app → 在正文里点一下让光标进去 → 切回来看结果。

```
ok · 11ms
备忘录 (com.apple.Notes)
我们这个模块的接口设计得不太好,⟦光标⟧
```

`⟦光标⟧` 是光标位置,左边上文右边下文。

| app | 预期 |
|---|---|
| 备忘录 / 文本编辑 | `ok` |
| VS Code / Notion / Claude 桌面版(Electron) | 能读到,但光标位置常常不准 |
| 微信 / 飞书 | `ok` |
| 浏览器普通输入框 | `ok` |
| 浏览器**密码框** | **必须 `blocked` / `secure_text_field`** |
| 终端 / iTerm / Warp | **必须 `blocked` / `blocked_app`** |
| 1Password | **必须 `blocked` / `blocked_app`** |

后三行是安全验收,任何一条没拦住立刻停下来说。

**屏幕上显示出来的原文,就是会发给 LLM 的内容。** 哪个 app 里蹦出了你不希望离开这台机器的东西,那是必须知道的发现。

---

## 2. 开关关闭时行为不变

关掉开关 → 听写几句 → 日志里**不该有任何 `cursor-context`**

关着的时候一次 AX 都不发,prompt 也和这个功能不存在时逐字节相同(有单测钉死)。

---

## 3. 上下文真的进 prompt 了吗

同一个 app、同一个位置,开/关各听一次同样的话:

```bash
grep "effective_prompt_chars" ~/Library/Logs/OpenLess/openless.log | tail -2
```

开着的那次应该多 **427 + 读到的字数**(427 = 上下文块固定措辞 314 + 注入防御条款 113)。

必须同一个 app 比——prompt 里带了前台应用名,「备忘录 (com.apple.Notes)」和「Claude (com.anthropic.claudefordesktop)」差 19 个字符,换 app 比会对不上账。

**更该盯的是 LLM 拿它干了什么。** 最容易翻车的是把上文复述进输出:光标前写着「这个模块的接口设计得不太好,」,你口述「还得再改」,它输出「这个模块的接口设计得不太好,还得再改」——把你已有的字又插了一遍。prompt 里明令禁止了,但那是软约束。

---

## 4. 手改 → 卡片 → 词条

前提:开关开着,**在备忘录这类原生 app 里测**(Electron 的通知不稳)。

**没有自动入库这条路了。** 任何一处手改都只会变成卡片上的一条建议,你点勾它才进词汇表。

早期版本让「中文改成英文」这一档静默入库,理由是「没人为了换语气把中文改成英文」。
真机跑两天,自动收进去 5 条只有 1 条是对的:

| 自动收的 | 实际是什么 |
|---|---|
| `Tailscale`(was telskill) | ✅ 唯一一条真纠错 |
| `ype`(was ap) | 逐字敲 `Type` 的中间态 |
| `ess`(was ice) | 同上 |
| `typeless`(was TypeScript) | 用户本来就要打这个词 |
| ` claude`(was cloud) | 带前导空格,永远匹配不上 |

观察器看到的是**编辑过程中的每一帧**,而中间态和一次真纠错在文本上没有区别。分不出来就别猜。

### 4a. 卡片长什么样

1. 口述一句,把其中某个词改掉
2. **把光标点到别处**(这是判定「你改完了」的信号)
3. 屏幕**右下角**弹出卡片:

```
要记住这个词吗?
扣德克斯 → Codex ✓ ✗
大禹 → 大鱼 ✓ ✗
```

-**** → 进词汇表,落在分割线下面的「自动收集」区
-**** → 丢掉,什么都不记(没有拒绝名单,下次再改同一个词它还会问)
- 等 10 秒不动 → 整张卡片消失,同样什么都不记
- 连着改两个词 → 合并到同一张卡片,倒计时重置
- 逐条点完最后一条 → 卡片自己收起

**位置必须是右下角,不是屏幕正下方居中。** 居中那块正是你在写字的地方,卡片停十秒会
直接盖住正在编辑的那一行 —— 这是真机上退回来的。

卡片只挡住它自己那一块的鼠标(窗口会缩到卡片大小,逐条点完还会跟着重算),周围照常能点。

### 4c. 该拒绝的时候确实拒绝了

| 操作 | 期待 |
|---|---|
| 改完切到别的 app 再改 | 无输出(观察器已解除) |
| 落字后等 60 秒再改 | 无输出(硬上限) |
| 落字后再听写一次,回头改第一段 | 无输出(新会话解除旧观察器) |
| 改你自己之前写的内容(不是它插的) | 无输出(只认落在插入文本里的改动) |
| 只是补几个字(纯插入) | 不学 |
| 把一个词删掉(纯删除) | 检测到但不入库 |
| 在聊天框里按回车发送 | 不该产生任何建议 ← 这条曾经翻过车 |
| 慢慢逐字打出一个英文词 | **可能**会弹出 `ap → ype` 这种半截建议 —— 点 ✗ 就行 |

最后一条不是 bug,是这个设计的已知代价:半截和真纠错在文本上没有区别,粗筛拦不住。
以前它会**静默入库**,现在最多是让你多点一次叉。

---

## 5. 词汇表页

```
词汇表
[你自己加的...]
───────── 自动收集(N) [全部删除]
[自动收的...]
```

试一下「全部删除」,确认**手动加的不会被一起删掉**

学来的词条在 `dictionary.json` 里是**追加到最后**的,手动添加的才插到最前。这条顺序是
跟 ASR 词表预算的接口约定:预算把最前面的几条当保底席位(理由是「你刚手动加它,多半刚
被它坑过」),点一下勾不该享受这个待遇。学来的词在 **LLM 热词块里立刻生效**(那一侧没有
名额限制),ASR 侧则靠命中次数自己爬进预算。

早期版本往**纠正规则**里写过 learned 条目(现在不写了)。如果你的 `correction-rules.json` 里还有,纠正规则区有个「只看自动收集的」筛选可以把它们挑出来删。

---

## 6. 不会冻住界面

对着一个卡死的 app 触发听写。AX 调用 200ms 超时、整次读取 1.2 秒封顶、跑在独立线程,不占 tokio worker。

---

## 已知限制

1. **建议只在内存里**,OpenLess 重启就没了。卡片消失即当没发生——下次改同一个词会再问。
2. **Electron 类 app 的光标位置常常不准**,日志里表现为 `before=0 after=N`(上文读成空)。上下文对润色的价值主要在上文,那种情况下收益有限。
3. **风格包预览里看不到 `<cursor_context>`**,跟 `front_app` 一样是运行时才有值的东西。

---

## 出问题时给我这些

```bash
# 相关日志(诊断细节是 debug 级别,日常不记;要更细的得改 LevelFilter 重编译)
grep -E "cursor-context|cursor context|vocab" ~/Library/Logs/OpenLess/openless.log | tail -100

# 学到的词条
python3 -c "import json,os;[print(' ',e['phrase']) for e in json.load(open(os.path.expanduser('~/Library/Application Support/OpenLess/dictionary.json'))) if e.get('note')=='从手改中自动收集']"

# 开关状态
python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/Library/Application Support/OpenLess/preferences.json'))).get('cursorContextEnabled'))"
```

`edit watch disarmed` 那一行带两个数字——收到几次通知、学到几处改动。这两个数字足够判断某个 app 到底发不发通知,也就是逐 app 的覆盖率数据。
Loading
Loading