feat: generalize Usage & Spend across tools and model providers - #2322
feat: generalize Usage & Spend across tools and model providers#2322Yuxin-Qiao wants to merge 30 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0bb85adb5a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
clawsweeper
Bot
added
proof: sufficient
|
Codex review: needs real behavior proof before merge. Reviewed August 2, 2026, 12:22 PM ET / 16:22 UTC. ClawSweeper reviewWhat this changesThis branch generalizes Usage & Spend with local tool-history ingestion, provider-aware billing attribution, model and tool analysis, a token-activity heatmap, dashboard UI changes, and quota-notification persistence. Merge readiness⛔ Blocked until stronger real behavior proof is added - 8 items remain Keep this PR open for maintainer direction rather than closing it: current Priority: P2 Review scores
Verification
How this fits togetherCodexBar gathers provider API data and local tool-history snapshots, aggregates them into usage and cost records, and renders them in the Usage & Spend preferences dashboard. This PR changes both the local-data/attribution path and the dashboard surfaces that present the resulting history. flowchart LR
A[Provider APIs and local tool histories] --> B[Usage and cost snapshots]
B --> C[Scanning and billing attribution]
C --> D[Usage and Spend aggregation]
D --> E[Preferences dashboard]
D --> F[Quota notification state]
Decision needed
Why: The split PRs and the umbrella branch intentionally cover the same central capability; only maintainers can choose the preferred merge, review, and rollback boundary. Before merge
Findings
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Use one focused landing sequence: review the data layer, heatmap, and Models work in their declared split PRs, then close this umbrella branch once no unique, necessary remainder remains. Do we have a high-confidence way to reproduce the issue? Not applicable: this is a feature PR rather than a report of broken established behavior. The available screenshots show earlier UI states, but not current-head live scanner, attribution, or notification behavior. Is this the best way to solve the issue? No: the umbrella branch is not the best landing path while it duplicates the three focused, still-open PRs named in its own body. One authoritative split sequence is more reviewable and reversible. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 1fd7ec83b5d9. LabelsLabel justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (40 earlier review cycles; latest 8 shown)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 18a58d70fb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
811bc6f to
92714de
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 92714def17
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
37f7807 to
fe82c42
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 37f78075d2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe82c42ca0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fe82c42 to
ff057d5
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff057d5869
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Implements Scope B from steipete#2257. - Replace per-currency "model breakdown unavailable" panel with an embedded Models card showing daily stacked usage bars and a full model ranking. - Merge models across providers by trimmed, case-insensitive exact name matching; keep token aggregation separate from cost to avoid cross-currency sums. - Add independent 7d / 30d / All range control for the model card without affecting the existing Overview 30d spend semantics. - Add Kimi Code session scanner so Kimi token-only history can contribute to the cross-provider model analysis. - Localize new UI strings across all supported languages. - Adopt upstream completeModelSummaries filtering for models list while keeping modelAnalysis built from all summaries for partial coverage. - Regenerate CodexParserHash for merged Vendored/CostUsage sources. Tests: SpendModelsPresentationTests, SpendDashboardModelTests, SpendDashboardKimiModelTests, KimiCodeSessionScannerTests, CostUsageDailyReportMergeTests, ShareStatsTests, UserFacingLocalizationCoverageTests (63 tests). Co-authored-by: Cursor <cursoragent@cursor.com>
ff057d5 to
7321949
Compare
clawsweeper
Bot
added
status: 👀 ready for maintainer look
… activity heatmap Spend accuracy: - Expand Codex pricing fingerprint to all third-party providers routed through the Codex/Claude-compatible endpoints (deepseek, minimax, moonshotai, kimi-for-coding) so stale caches invalidate when models.dev pricing lands — fixes "Spend unavailable". - Daily-sum cost fallback: when a provider's aggregate cost figure diverges from the local per-day logs, sum per-day costs instead of voiding the whole provider. New tool coverage (by-tool / by-model): - OpenCode: read opencode.db (SQLite) in addition to JSONL messages, extracting model/tokens/cost via json_extract; cross-source dedup by id + fingerprint. - Add MiniMax, Gemini session scanners and provider registration. Token activity heatmap: - GitHub/ChatGPT-style 52-week grid with hover tooltip (date + token count). - Cache SpendActivitySeries; binary-search nearest day in models chart hover/click. Refactor & quality: - Extract LocalSnapshotSource to dedupe per-provider load blocks; ScanContext to thread shared scan state; split third-party Claude pricing into its own file. - Complete i18n: fill the new dashboard keys across all 22 locales. Co-authored-by: Cursor <cursoragent@cursor.com>
Yuxin-Qiao
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 208945060f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3d1b64f16f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cd94199770
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
cd94199 to
2cf1d84
Compare
48fdd0e to
acb9821
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: acb98219e8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
clawsweeper
Bot
added
rating: 🦐 gold shrimp
… scan-limit signaling Addresses the re-review batch: OpenCode provider routing evidence, MiniMax/Qwen/OpenCode/Kimi partial-pricing aggregates, Antigravity SQLITE_BUSY and protobuf bounds, Qwen oversized records, Kimi/OpenCode file limits, and billing-attribution merged costs.
acb9821 to
d41f830
Compare
clawsweeper
Bot
added
rating: 🦪 silver shellfish
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d41f830058
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
92ec223 to
0a487b3
Compare
There was a problem hiding this comment.
💡 Codex Review
CodexBar/Sources/CodexBarCore/CostUsageModels.swift
Lines 845 to 848 in 0a487b3
Use checked arithmetic for merged entry totals
Fresh evidence beyond the earlier model-breakdown overflow fix is that the top-level EntryAccumulator still adds accepted buckets with trapping arithmetic. An entry with inputTokens == Int.max and any positive output/cache bucket crashes immediately while evaluating entryDerivedTotalTokens—even when it already supplies an explicit totalTokens—so merging native and Pi reports can terminate the usage scan instead of marking the aggregate unavailable.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
0a487b3 to
dfc9b43
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dfc9b4301b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
dfc9b43 to
1c765c1
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c765c1b61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1c765c1 to
5478f2f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5478f2fdc9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… and quota persistence
5478f2f to
a316207
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a31620793c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| input: input, | ||
| identity: identity, | ||
| tokens: 0) | ||
| tool.cost += entryCost |
There was a problem hiding this comment.
When a preferred-currency conversion produces summary.costMultiplier != 1, the chart and provider totals are converted, but this accumulator keeps the source-currency amount; the model costs added below are likewise unscaled. Fresh evidence beyond the fixed model-analysis path is that clicking a converted chart bar therefore shows raw USD tool/model values formatted as the target currency. Apply summary.costMultiplier to both the entry and breakdown costs before accumulating the daily detail.
Useful? React with 👍 / 👎.
| else { | ||
| return false | ||
| } | ||
| let uncachedInput = max(0, promptTotal - cacheRead) |
There was a problem hiding this comment.
When a malformed or changed Qwen record reports cachedContentTokenCount > promptTokenCount, this clamp accepts the record with zero uncached input and then adds the full cache count, even though the preceding invariant states that cached tokens are a subset of the prompt. The established snapshot consequently overcounts input and estimates cost from an impossible token split; reject such a record by requiring cacheRead <= promptTotal rather than clamping the subtraction.
Useful? React with 👍 / 👎.
| } else { | ||
| self.overflowingTokenBuckets = true | ||
| } |
There was a problem hiding this comment.
When two same-day reports contain individually finite costs whose sum overflows—for example two Double.greatestFiniteMagnitude values—this branch marks the token buckets as overflowed but leaves the previous finite costUSD and sawCost intact. build then emits that first report's cost as the merged day's complete subtotal while only making its tokens unavailable; track cost overflow separately and make the merged cost nil.
Useful? React with 👍 / 👎.
| let resourceValues = try? url.resourceValues( | ||
| forKeys: [.isRegularFileKey, .fileSizeKey, .contentModificationDateKey]) | ||
| guard resourceValues?.isRegularFile == true else { continue } |
There was a problem hiding this comment.
When an enumerated Qwen transcript cannot be stat'ed because of a transient filesystem or permissions error, this try? and guard silently skip it before the now-propagating CostUsageJsonl read path is reached. Fresh evidence beyond the fixed transcript-read failure is this separate metadata-read path: remaining files can still produce an established partial snapshot, so propagate the metadata error rather than treating the transcript as non-regular.
Useful? React with 👍 / 👎.
…dels PR The daily estimated spend chart (SpendDailyChart) and its model-side DailyPoint/DailySpendDetail aggregation were split out of steipete#2322 but are not ready for review yet. Withhold them from this PR; the code remains in steipete#2322 for a follow-up PR.
The Models PR adds dashboard strings to every catalog; the Italian catalog keeps them in English (same choice as steipete#2322), so the language catalog test needs the same allowlist entries to stay green in CI.
coveredDayCount measures the coverage window, not the number of spend days, so the replacements used during the daily-card removal were wrong. The heatmap assertion now checks the 30-day window stays unwidened; the rollover and mutation cases already prove window behavior through totals. The localization coverage test follows steipete#2322 and reads the embedded model card from PreferencesSpendModelsView instead of the dashboard pane.
clawsweeper
Bot
added
rating: 🦐 gold shrimp
|
Maintainer decision on the Usage & Spend program (applies to #2322, #2527, #2548, #2569, #2575, #2577 — closing together): We want the ideas — the token-activity view and the cross-provider cost overview in particular are genuinely good, and we're adopting them. What we can't absorb is the delivery shape: 8k–20k-line PRs bundling scanners, pricing, identity, and UI can't be reviewed honestly, and live testing of the current heads surfaced the kind of defects that big surfaces hide (duplicate token-activity cards in the Models view; the unified Overview rendering no chip for an enabled provider; the account-attribution issue flagged earlier). So we're taking this work as a base in-house and rebuilding it in small, individually landable slices — first the token-activity view rebuilt on the shared scan cache (no separate annual scan), then the cost overview aligned with the #2578 design discussion. Your commits will be cherry-picked where they fit, and every slice that draws on this work will credit you in the changelog, as today's merges already do. @Yuxin-Qiao — you're this repo's most prolific contributor and much of today's release is your work. Focused PRs the size of #2574 land same-day; that's the shape that serves both of us. |
2 participants
Summary
Generalizes Usage & Spend into a tool- and provider-agnostic reporting pipeline.
This is not a MiniMax-only integration or a Claude Code-only workflow. MiniMax and Claude Code are regression fixtures for a broader requirement: users can run the same model through different IDEs, desktop apps, and CLIs, while billing still belongs to the real subscription or API provider.
What changed
Generic source architecture
Billing attribution based on evidence
Honest partial-data behavior
$0.Dashboard and interaction
Reliability
Scope and extension boundary
The dashboard automatically handles any provider that emits the shared structured token/cost snapshot. A tool with a new proprietary on-disk format still needs one bounded local adapter; until that exists, the UI reports unavailable/partial data rather than guessing. This keeps adding ZCode, Qoder, Zed, or another tool localized to its parser and descriptor instead of duplicating dashboard logic.
Visual proof
These screenshots exercise the implemented dashboard with local-history data across priced, partially priced, and fully unpriced states. They verify presentation and state handling; local estimated spend can differ from provider invoices.
Subscription ownership and tool grouping
$0.Model analysis: tokens and estimated spend
Unavailable, not zero.Daily source and model drill-down
The daily view preserves source ownership, then drills into per-model estimates. A known zero-cost model remains
$0.00, which is distinct from missing pricing.Token activity modes
Fully unpriced history
When every contributing source lacks pricing, the dashboard still shows 33K tracked tokens and both subscriptions, while the headline and chart explicitly report
Pricing unavailable/Spend unavailable.Evidence boundary
Validation
make checkmake test./Scripts/compile_and_run.shThe release app launches successfully. The visual proof above covers the implemented priced, partially priced, and fully unpriced dashboard states; it does not claim equality with external provider bills.
Split status
This umbrella PR is being split into independently mergeable pieces. Land in order, then close this PR: