Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions docs/github-tracking/issue-139-capsule-lifecycle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
## 现象 / Symptom

这不是单一的 click dead zone bug,而是一组已经在 Windows 实机上被观察到、且共享同一根因的 helper-window lifecycle 症状:

- click dead zone:原 Capsule 区域附近会挡住底层输入框或按钮
- screenshot selectable:截图工具仍然可以选中这块透明区域
- drag stutter:在该区域拖拽时出现明显卡顿或 compositor 异常
- lingering transparent overlay:录音结束后,Capsule 仍可能以透明顶层窗 linger

当前证据说明:这些现象不应拆成多个互不相关的问题,而应视为同一个生命周期语义偏差。

### 证据 / Evidence

运行与代码证据:

- `openless-all/app/src-tauri/tauri.conf.json:33-47`
- `capsule` 被配置为 `transparent + alwaysOnTop + focus:false + visible:false`
- `openless-all/app/src-tauri/src/lib.rs:594-623`
- Windows 端 `capsule` runtime host bounds 为 `220x84/118`,明显大于可见 pill `196x52`
- `openless-all/app/src-tauri/src/coordinator.rs:2398-2432`
- Windows 端显示路径走 `ShowWindow(SW_SHOWNOACTIVATE)` + `SetWindowPos(...SWP_NOACTIVATE...)`
- `openless-all/app/src-tauri/src/coordinator.rs:2455-2479`
- 结束阶段依赖 `window.hide()` 作为生命周期结束语义
- `openless-all/app/src/components/Capsule.tsx:278-281`
- 前端 `idle` 只把可见内容缩成 `0x0`,真正结束仍取决于后端窗口是否已完全退出参与
- [2026-05-02-platform-lifecycle-audit.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/2026-05-02-platform-lifecycle-audit.md)
- 审计已把该问题收敛为 Windows helper-window lifecycle contract 偏差

现场证据:

- 用户已在 Windows 上观察到 dead zone / screenshot selectable / drag stutter / lingering overlay
- 这些表现与透明顶层 helper window 未真正退出 OS 参与的形态一致

### 5 Whys / 根因分析

1. 为什么会出现点击死区、截图可选中、拖拽卡顿?
- 因为录音结束后,Windows 上的 Capsule host window 仍可能继续存在并参与桌面层级。
2. 为什么录音结束后窗口还会继续参与?
- 因为当前实现把“生命周期结束”主要建模成 `hide()`,而不是“保证 helper window 不再参与 hit-test / capture / z-order / compositor”。
3. 为什么这个问题在 Windows 上更容易暴露?
- 因为 Windows 的 Capsule host geometry 更大、show path 更特殊,并且是透明顶层窗;一旦 hide 语义失守,残留面积极大且更容易干扰系统行为。
4. 为什么这和 macOS 的原始设计意图不一致?
- macOS 的原始意图是:Capsule 只在 active stage 短暂出现,结束后自然收起,不再作为前台交互对象继续存在;Windows 当前更像“视觉结束了,但 OS 对象还挂着”。
5. 为什么之前没有被门禁拦住?
- 现有检查更多关注“窗口显示/隐藏”和几何配置,没有直接验证 inactive state 下它是否真的退出系统参与。

### 平台边界 / Platform Scope

- 直接症状范围:当前已确认是 Windows 实机问题。
- 问题层面:backend helper-window lifecycle contract + Windows native window participation。
- 全平台风险判断:根因模式不是 Windows 独有,任何透明 helper window 只要“视觉隐藏 != 生命周期结束”都可能中招;Capsule 目前是 Windows 上最先爆出来的样板案例。

### 认领 / Ownership

- owner intent:`@Cooper-X-Oak`
- 当前对应 draft PR:`#140`

### 当前状态 / Current status

- lifecycle 主线修复已完成第一波
- 人工桌面回归结果:
- click dead zone:通过
- screenshot selectable:通过
- drag stutter:通过
- 当前建议:从“问题收敛中”推进到“regression review 中”

## 影响 / Impact

- 直接影响 Windows 端核心输入体验与系统交互可信度
- 会误伤底层 app 的点击、截图、拖拽,用户容易误判成其他应用故障
- 因为残留对象透明且顶层,这类问题隐蔽、难复现、难定位
- 如果不从生命周期语义修,后续即使修掉某一个 dead zone,仍可能继续遗留 screenshot / z-order / compositor 问题

## 建议接受标准 / Proposed Acceptance Criteria

- [ ] Windows 上 Capsule 的“结束”语义与 macOS 对齐:inactive 后不再继续参与系统交互
- [ ] inactive Capsule 不再造成 click dead zone
- [ ] inactive Capsule 不再被截图工具选中
- [ ] inactive Capsule 不再引入 drag/compositor stutter
- [ ] 为 Windows 增加一条直接验证 inactive Capsule non-participating 的 smoke / regression check
- [ ] 修复方案明确区分 visual state 与 host-window lifecycle state,而不是继续叠加局部 workaround

## TODO / 不确定项

- 是否需要把 `capsule hidden => no hit-test / no capture / no topmost participation` 抽成统一 helper-window contract,复用于 QA panel
- 当前 `PR #140` 建议保持 draft tracking 角色,待范围与根因完全收敛后再转 ready
建议 issue 标题:`[ui][windows] Capsule 隐藏后仍参与系统交互`
52 changes: 52 additions & 0 deletions docs/github-tracking/pr-140-capsule-lifecycle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
## 摘要

Closes #139

这个 PR 现在从“问题收敛中”推进到“regression review 中”。

本轮已经完成:

- Windows helper-window lifecycle root cause 收敛
- `inactive` 路径的 native hide / non-topmost 收口
- 冷启动最新 debug 包回归
- 人工桌面症状回归:
- click dead zone:通过
- screenshot selectable:通过
- drag stutter:通过

## 修复 / 新增 / 改进

- 对齐 PR 目标:关注 Windows Capsule helper-window lifecycle,而不是单点 dead zone workaround
- 收口 Windows 上 `visible / hidden / inactive / non-participating` 的 Capsule 语义
- 在 backend 上补齐 inactive 后的 native hide 行为,避免 transparent topmost helper window lingering
- 新增 lifecycle contract / smoke 辅助脚本,帮助后续回归持续验证
- 与 [issue-139-capsule-lifecycle.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/github-tracking/issue-139-capsule-lifecycle.md) 保持同一问题口径

## 兼容

- 不包含:Capsule geometry / rounded corner / titlebar frame 纯视觉适配
- 不包含:QA hotkey / selection ask 输入源逻辑
- 对现有用户 / 本地环境 / 构建流程的影响:只聚焦 lifecycle 主线,不扩大到 UI polish 线

## 测试计划

- [x] 命令:`node openless-all/app/scripts/windows-lifecycle-contract.test.mjs`
- [x] 结果:通过
- [x] 证据路径:本地命令输出

- [x] 命令:`npm run build`
- [x] 结果:通过
- [x] 证据路径:本地命令输出

- [x] 命令:`cargo check --manifest-path openless-all/app/src-tauri/Cargo.toml`
- [x] 结果:通过
- [x] 证据路径:本地命令输出

- [x] 命令:`powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-runtime-smoke.ps1`
- [x] 结果:通过 launch / hotkey installed baseline
- [x] 证据路径:本地命令输出

- [x] 命令:人工桌面回归(latest debug cold start -> dictation start/stop)
- [x] 结果:点击 / 截图 / 拖拽三项全部通过
- [x] 证据路径:当前线程回归记录
关联 issue 建议标题:`[ui][windows] Capsule 隐藏后仍参与系统交互`
128 changes: 128 additions & 0 deletions openless-all/app/scripts/windows-capsule-lifecycle-smoke.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
param(
[string]$ExePath = "",
[int]$TimeoutSeconds = 15
)

$ErrorActionPreference = "Stop"

if ([string]::IsNullOrWhiteSpace($ExePath)) {
$appRoot = (Resolve-Path (Join-Path $PSScriptRoot "..")).Path
$ExePath = Join-Path $appRoot "src-tauri\target\debug\openless.exe"
}

if (-not (Test-Path $ExePath)) {
throw "OpenLess executable not found: $ExePath"
}

$logPath = Join-Path $env:LOCALAPPDATA "OpenLess\Logs\openless.log"
Remove-Item -LiteralPath $logPath -Force -ErrorAction SilentlyContinue
Get-Process openless -ErrorAction SilentlyContinue | Stop-Process -Force

Add-Type @"
using System;
using System.Runtime.InteropServices;

public static class OpenLessCapsuleProbe {
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
public static extern IntPtr FindWindowW(string lpClassName, string lpWindowName);

[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool IsWindowVisible(IntPtr hWnd);

[DllImport("user32.dll")]
public static extern void keybd_event(byte bVk, byte bScan, int dwFlags, UIntPtr dwExtraInfo);

public const int KEYEVENTF_EXTENDEDKEY = 0x0001;
public const int KEYEVENTF_KEYUP = 0x0002;
}
"@

function Wait-LogPattern($Pattern, $TimeoutSeconds) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
if ((Test-Path $logPath) -and ((Get-Content -Raw $logPath) -match $Pattern)) {
return $true
}
Start-Sleep -Milliseconds 200
}
return $false
}

function Send-KeyEdge([byte]$Vk, [bool]$KeyUp) {
$flags = [OpenLessCapsuleProbe]::KEYEVENTF_EXTENDEDKEY
if ($KeyUp) {
$flags = $flags -bor [OpenLessCapsuleProbe]::KEYEVENTF_KEYUP
}
[OpenLessCapsuleProbe]::keybd_event($Vk, 0x1D, $flags, [UIntPtr]::Zero)
}

function Get-CapsuleWindowState() {
$hwnd = [OpenLessCapsuleProbe]::FindWindowW($null, "OpenLess Capsule")
if ($hwnd -eq [IntPtr]::Zero) {
return [pscustomobject]@{
Exists = $false
Visible = $false
Handle = "0x0"
}
}

return [pscustomobject]@{
Exists = $true
Visible = [OpenLessCapsuleProbe]::IsWindowVisible($hwnd)
Handle = ('0x{0:X}' -f $hwnd.ToInt64())
}
}

Write-Host "== Windows capsule lifecycle smoke =="
$env:OPENLESS_ACCEPT_SYNTHETIC_HOTKEY_EVENTS = "1"
$env:OPENLESS_HOTKEY_INJECTION_DRY_RUN = "1"
$process = Start-Process -FilePath $ExePath -WorkingDirectory (Split-Path $ExePath -Parent) -PassThru
try {
if (-not (Wait-LogPattern "hotkey listener installed" $TimeoutSeconds)) {
throw "Hotkey listener did not install within $TimeoutSeconds seconds."
}

Start-Sleep -Milliseconds 500
$before = Get-CapsuleWindowState

Send-KeyEdge 0xA3 $false
Start-Sleep -Milliseconds 120
Send-KeyEdge 0xA3 $true

$startedDryRun = Wait-LogPattern "session started \(hotkey-injection dry-run\)" 5
Start-Sleep -Milliseconds 400
$afterStart = Get-CapsuleWindowState

Send-KeyEdge 0xA3 $false
Start-Sleep -Milliseconds 120
Send-KeyEdge 0xA3 $true
Start-Sleep -Seconds 3
$afterStop = Get-CapsuleWindowState

[pscustomobject]@{
StartedDryRun = $startedDryRun
Before = "$($before.Handle) visible=$($before.Visible)"
AfterStart = "$($afterStart.Handle) visible=$($afterStart.Visible)"
AfterStop = "$($afterStop.Handle) visible=$($afterStop.Visible)"
} | Format-List

if (-not $startedDryRun) {
throw "Dry-run session did not start; cannot verify capsule lifecycle."
}

if (-not $afterStart.Visible) {
throw "Capsule did not become visible during synthetic recording start."
}

if ($afterStop.Visible) {
throw "Capsule is still visible after synthetic stop."
}

Write-Host "[ok] Capsule window is not visible after synthetic stop."
}
finally {
Remove-Item Env:OPENLESS_ACCEPT_SYNTHETIC_HOTKEY_EVENTS -ErrorAction SilentlyContinue
Remove-Item Env:OPENLESS_HOTKEY_INJECTION_DRY_RUN -ErrorAction SilentlyContinue
Get-Process openless -ErrorAction SilentlyContinue | Stop-Process -Force
}
47 changes: 47 additions & 0 deletions openless-all/app/scripts/windows-capsule-watch.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
param(
[int]$DurationSeconds = 20
)

$ErrorActionPreference = "Stop"

Add-Type @"
using System;
using System.Runtime.InteropServices;

public static class OpenLessCapsuleWatch {
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
public static extern IntPtr FindWindowW(string lpClassName, string lpWindowName);

[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool IsWindowVisible(IntPtr hWnd);
}
"@

function Get-CapsuleState {
$hwnd = [OpenLessCapsuleWatch]::FindWindowW($null, "OpenLess Capsule")
if ($hwnd -eq [IntPtr]::Zero) {
return "missing"
}
if ([OpenLessCapsuleWatch]::IsWindowVisible($hwnd)) {
return "visible"
}
return "hidden"
}

Write-Host "== Windows capsule watch =="
Write-Host "Watch duration: $DurationSeconds seconds"
Write-Host "Please trigger dictation start/stop now."

$deadline = (Get-Date).AddSeconds($DurationSeconds)
$last = ""
while ((Get-Date) -lt $deadline) {
$state = Get-CapsuleState
if ($state -ne $last) {
Write-Host ("[{0:HH:mm:ss.fff}] capsule={1}" -f (Get-Date), $state)
$last = $state
}
Start-Sleep -Milliseconds 100
}

Write-Host "== Final capsule state: $last =="
54 changes: 50 additions & 4 deletions openless-all/app/scripts/windows-ui-config.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,58 @@ function assertEqual(actual, expected, name) {
}
}

function assertMatch(source, pattern, name) {
if (!pattern.test(source)) {
throw new Error(`${name}: pattern ${pattern} not found`);
}
}

const raw = await readFile(new URL('../src-tauri/tauri.conf.json', import.meta.url), 'utf-8');
const config = JSON.parse(raw);
const mainWindow = config.app.windows.find((window) => window.label === 'main');
const capsuleWindow = config.app.windows.find((window) => window.label === 'capsule');
const libRs = await readFile(new URL('../src-tauri/src/lib.rs', import.meta.url), 'utf-8');
const coordinatorRs = await readFile(new URL('../src-tauri/src/coordinator.rs', import.meta.url), 'utf-8');

if (!mainWindow) {
throw new Error('main window config missing');
if (!capsuleWindow) {
throw new Error('capsule window config missing');
}

assertEqual(mainWindow.decorations, false, 'windows main window should use only custom titlebar');
assertEqual(capsuleWindow.width, 220, 'windows capsule config keeps translation-capable width baseline');
assertEqual(capsuleWindow.height, 110, 'windows capsule config keeps translation-capable height baseline');
assertEqual(capsuleWindow.transparent, true, 'capsule window should keep transparent visuals');
assertEqual(capsuleWindow.alwaysOnTop, true, 'capsule window should stay above the focused app while recording');
assertMatch(
libRs,
/#\[cfg\(target_os = "windows"\)\][\s\S]*?\(196\.0, height\)/,
'windows runtime capsule width should collapse to the visible pill',
);
assertMatch(
libRs,
/let height = if translation_active \{ 110\.0 \} else \{ 52\.0 \};/,
'windows runtime capsule height should shrink outside translation mode',
);
assertMatch(
libRs,
/window\.set_size\(LogicalSize::new\(cap_w, cap_h\)\)\?/,
'capsule positioning should resync runtime size with the computed layout',
);
assertMatch(
coordinatorRs,
/let visible = matches!\(\s*state,\s*CapsuleState::Recording \| CapsuleState::Transcribing \| CapsuleState::Polishing\s*\);/m,
'capsule should only stay visible during active recording or processing states',
);
assertMatch(
coordinatorRs,
/fn hide_capsule_window_if_present\(\)/,
'windows capsule lifecycle should include an explicit native hide helper',
);
assertMatch(
coordinatorRs,
/ShowWindow\(hwnd, SW_HIDE\)/,
'windows capsule hide helper should force the native window hidden',
);
assertMatch(
coordinatorRs,
/SetWindowPos\([\s\S]*?HWND_NOTOPMOST[\s\S]*?SWP_HIDEWINDOW/m,
'windows capsule hide helper should drop topmost participation when inactive',
);
Loading
Loading