fix(ai): benchmark uncached vision requests - #172
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
📜 Recent review details⏰ Context from checks skipped due to timeout. (6)
🧰 Additional context used📓 Path-based instructions (5)**/*.{ts,tsx,rs}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
tests/**/*.{ts,tsx}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
**/*📄 CodeRabbit inference engine (CLAUDE.md)
Files:
**/*.md📄 CodeRabbit inference engine (CLAUDE.md)
Files:
src/**/*.{ts,tsx}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
🔇 Additional comments (33)
📝 WalkthroughWalkthroughThe AI benchmark now measures cache-cold inference requests. Stale or missing benchmarks are excluded from readiness, cadence, displayed speeds, and timeout decisions. Unbenchmarked models use a five-minute request timeout, with updated documentation and regression tests. ChangesAI benchmark freshness
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant ModelPickerContainer
participant benchmarkGate
participant sampleLoop
User->>ModelPickerContainer: Enable AI
ModelPickerContainer->>benchmarkGate: Check active model readiness
benchmarkGate-->>ModelPickerContainer: Return ready or unbenchmarked
ModelPickerContainer->>sampleLoop: Start model sampling
sampleLoop->>sampleLoop: Select current benchmark or five-minute fallback timeout
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
What changed and why
AI sampling on the reported Windows CPU setup aborted every live vision request at 90 seconds. The saved benchmark looked fast only because its warmup and three measured passes reused one byte-identical prompt; llama-server's prompt cache skipped both image encodes after the discarded warmup, recording a ~12.6 s p95 even though changing live frames needed roughly 127–148 seconds.
This change disables prompt caching for benchmark requests, fingerprints the cache-cold benchmark protocol, refuses to use stale timing data for cadence/timeout decisions, and gives unmeasured or stale models the existing five-minute upper bound. The old contaminated benchmark is preserved for display as stale and can be replaced normally by re-benchmarking.
Closes #171
Manual test
npm run tauri devlaunched and the changed surface behaves as described— n-a: the reporter's Windows/Gemma CPU environment was not available; the 130-second live-request path is covered by a fake-clock regression test
— n-a: no visual styling or motion changed
Compatibility surfaces
No wire, identity, database, updater, or persisted-store shape changed. Existing benchmark records are preserved, but records from the old benchmark protocol are intentionally treated as stale until re-measured.
src-tauri/src/db/migrations/Gates
npm run build && npm run lint && npm run testnpm run check-tokens && npm run check-strings && npm run check-contrastnpm run check-migrations && npm run check-storiesnpm run build-storybook && npm run check-a11ycd src-tauri && cargo fmt --check && cargo clippy && cargo testcd src-tauri && cargo deny checkAlso checked version lockstep and diff whitespace. Local results: 1,068 frontend tests, 316 Storybook accessibility tests, and 117 native tests passed.
Merge style
Summary by CodeRabbit
AI Model Reliability
Performance