feat(dashboard): Human Intervention metric (#34) - #70
Merged
Conversation
Add unit + e2e coverage for the interrupt / tool-reject / correction signals and the team-stats roll-up: - dashboard-collector.test.ts: countInterventions + rebuildSessions - digest-interventions.test.ts / intervention-report.test.ts - team-push-interventions.test.ts: delta + merge idempotency - intervention-e2e.test.ts: full hook -> events -> digest pipeline Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jeff-r2026
added a commit
that referenced
this pull request
Jun 30, 2026
Post-merge quality cleanups on the HTTP team-repo + status-reporter code (PRs #68/#70/#71), surfaced by /simplify. No behavior change. - path-safety: extract assertWithinRoot() and replace the hand-rolled containment check duplicated in source-http.ts and skill-command.ts. It compares resolved-but-not-symlink-followed paths, so it stays correct for not-yet-created targets and under symlinked roots (e.g. macOS /var -> /private/var) — unlike assertSafePath, which would false-reject legitimate writes there. - status-report: drop the no-op `await import('fs-extra')` in flushQueue (fs-extra is already loaded via utils/fs) and use the project remove/writeFile helpers; parallelize scanReportableSkills with Promise.all instead of awaiting per-skill reads in a loop. - team-push: reuse readJson/writeJson for the reported-interventions snapshot instead of hand-rolled JSON.parse/stringify. Verified: `tsc --noEmit` clean; full unit suite green (1662 passed). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Human Intervention metric for the team dashboard (closes #34) — tracking how often a human has to step in during a Claude Code / Cursor / CodeBuddy session. Three signals are detected from the session transcript and hook stream, then rolled up per-session, per-user and into the team digest:
[Request interrupted by user]in the transcript).The tool use was rejected/The user doesn't want to proceed with this tool use).不对/重新/redo/undo/…) rather than starting a fresh task.Wiring (no new dependencies):
src/dashboard-collector.ts—countInterventions()streams the transcript (memory-bounded, cheap pre-filter), idempotent Stop snapshot,rebuildSessions()exposes per-session badge data,aggregateSessionInterventions().src/types.ts— intervention types + detection constants (TRANSCRIPT_INTERRUPT_PREFIX,TRANSCRIPT_REJECT_MARKERS,CORRECTION_KEYWORDS,CORRECTION_WINDOW_MS, scan cap).src/team-push.ts—computeInterventionDelta()/mergeInterventionStats()(idempotent deltas intostats/<user>.yaml).src/digest.ts—summarizeInterventions()(totals, avg/session, ranked).src/dashboard-html.ts— per-session intervention badge.docs/usage-guide.md— metric documentation.Test Plan
All commands run on this branch rebased onto
main(a4a344e).Type check —
npx tsc --noEmit→ ✅ pass (exit 0)Unit suite —
npx vitest runIntervention-specific coverage (4 new unit files + amended collector test, 72 cases):
dashboard-collector.test.ts(countInterventions + rebuildSessions),digest-interventions.test.ts,intervention-report.test.ts,team-push-interventions.test.ts(delta + merge idempotency).E2E suite —
npm run test:e2e(runs undervitest.e2e.config.ts)Includes
intervention-e2e.test.ts(2 tests): full hook STDIN →events.jsonl→rebuildSessions→ delta → digest pipeline.Real end-to-end with the smallest model (
claude-haiku-4-5) — beyond the in-process tests, the full chain was exercised against a real Claude Code session and the real compiled CLI binary:claude --model claude-haiku-4-5 -p …) that used the Write tool → captured the real transcript JSONL. Confirmed our parser handles the production transcript schema (user/assistant/tool_use/tool_result+queue-operation/attachment/last-prompt) and reports 0 interventions on a clean session (no false positives).2.1.196):Request interrupted by user,The tool use was rejected,doesn't want to proceed with this tool use— all FOUND.node dist/index.js dashboard-report --stdin --tool claude) overSessionStart → PostToolUse → Stop → UserPromptSubmit, against the real transcript with the genuine interrupt + reject markers appended:Notes
package.json/ version bump (release is tag-triggered).main; full suite is 100% green (the older agent-status test-isolation flakiness is not present here).🤖 Generated with Claude Code