Skip to content

feat: Model attribution dashboard — track primary vs fallback model per turn#376

Open
vivekchand wants to merge 1 commit intomainfrom
fix/gh-300-model-attribution-dashboard
Open

feat: Model attribution dashboard — track primary vs fallback model per turn#376
vivekchand wants to merge 1 commit intomainfrom
fix/gh-300-model-attribution-dashboard

Conversation

@vivekchand
Copy link
Copy Markdown
Owner

Closes #300

What

Adds model attribution visibility to the ClawMetry dashboard — answers the question "which model actually responded to my agent's requests?" including silent fallbacks.

How

  • New /api/model-attribution endpoint: scans session JSONL files to extract per-model stats (sessions, tokens, cost) and detect primary vs fallback usage based on the configured model in openclaw.json
  • Fallback detection: any session whose observed model differs from the configured primary model is counted as a fallback
  • Alert when fallback rate exceeds 10%
  • Model Attribution panel added to the Tokens tab in both dashboard themes:
    • 🎯 Primary Model card (sessions + tokens + cost)
    • 🔀 Fallback Rate card (red highlight when >10%, amber when >5%)
    • 💸 Fallback Cost card (total tokens + cost burned on fallback models)
    • Sortable model table — all models sorted by token usage with primary/fallback/unknown badges
  • loadModelAttribution() JS wired into loadUsage() — loads automatically on every Tokens tab open
  • 5 new tests in TestModelAttribution: structure, types, range validation, session count consistency, model fields

Test

python3 -m py_compile dashboard.py  # passes
pytest tests/test_api.py::TestModelAttribution  # 5 tests

…er turn (closes #300)

- New /api/model-attribution endpoint: scans session JSONL files to build
  per-model stats (sessions, tokens, cost) and detect primary vs fallback
  usage based on configured model in openclaw.json
- Fallback rate alert when >10% of sessions used a non-primary model
- Model Attribution panel added to Tokens tab in both dashboard themes:
  Primary Model card, Fallback Rate card (red when >10%), Fallback Cost card,
  sortable model table showing all models with primary/fallback badges
- loadModelAttribution() JS wired into loadUsage() — loads on every Tokens tab open
- 5 new tests in TestModelAttribution covering structure, types, range, consistency
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Model attribution dashboard — track which model (primary vs fallback) responded per turn

1 participant