Skip to content

Add China API models.dev pricing lookup - #2376

Open
joeVenner wants to merge 8 commits into
steipete:mainfrom
joeVenner:codex/modelsdev-china-api-models
Open

Add China API models.dev pricing lookup#2376
joeVenner wants to merge 8 commits into
steipete:mainfrom
joeVenner:codex/modelsdev-china-api-models

Conversation

@joeVenner

@joeVenner joeVenner commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add provider-aware models.dev pricing lookup for China API provider families.
  • Route Alibaba coding plans to alibaba-coding-plan/alibaba-coding-plan-cn, Alibaba token plans to alibaba-token-plan/alibaba-token-plan-cn, z.ai to zai/zai-coding-plan, and DeepSeek to deepseek.
  • Wire those provider-scoped lookups into the Pi session runtime cost path and expose the supported providers through codexbar cost.

Proof

  • Previous-context check: PR Improve China Kimi/GLM routes: Coding Plan 5h, Kimi Monthly, clearer labels #2351 covers Kimi/GLM quota routing, not these models.dev provider pricing mappings. PR feat(providers): add Qwen Cloud individual token-plan provider #2361 adds Qwen Cloud token-plan usage, not Qwen model pricing lookup.
  • Absence check against current base returned no matches for the IDs used here:
    git grep -n -E "glm-5\.1|glm-5v-turbo|qwen3\.7-max|deepseek-v4-pro" origin/main -- Sources Tests docs
  • Source catalog check from https://models.dev/api.json showed these live provider keys: alibaba, alibaba-cn, alibaba-coding-plan, alibaba-coding-plan-cn, alibaba-token-plan, alibaba-token-plan-cn, deepseek, zai, zai-coding-plan, zhipuai, zhipuai-coding-plan.
  • Live catalog spot checks showed:
    • alibaba-coding-plan: qwen3.7-max / input 2.5 / output 7.5 / cache_read 0.5 / cache_write 3.125
    • alibaba-token-plan: qwen3.7-plus / input 0 / output 0 / cache_read 0 / cache_write 0
    • zai: glm-5v-turbo / input 1.2 / output 4 / cache_read 0.24 / cache_write 0
    • zhipuai: glm-5v-turbo / input 5 / output 22 / cache_read 1.2 / cache_write 0, intentionally not used for z.ai routing
    • deepseek: deepseek-v4-pro / input 0.435 / output 0.87 / cache_read 0.003625 / context 1000000
  • Runtime regression test uses a cached models.dev fixture with deliberately divergent fallback entries, then verifies PiSessionCostScanner.loadDailyReport computes provider-scoped costUSD for Alibaba coding plan, Alibaba token plan, z.ai, and DeepSeek Pi JSONL rows.
  • Built CLI proof used an isolated temp home with a synthetic .pi/agent/sessions/2026-07-28T10-00-00-000Z_china-proof.jsonl containing only assistant rows. No real account files, Keychain, or browser cookies were read. Command shape:
    HOME=[temp] CFFIXED_USER_HOME=[temp] XDG_CONFIG_HOME=[temp]/.config XDG_CACHE_HOME=[temp]/.cache .build/debug/CodexBarCLI cost --provider <provider> --format json --pretty --refresh
  • Built CLI output excerpts:
    { "provider" : "alibaba", "source" : "local", "sessionTokens" : 150000, "sessionCostUSD" : 0.625,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "qwen3.7-max", "totalTokens" : 150000, "cost" : 0.625 }] }] }
    { "provider" : "alibabatokenplan", "source" : "local", "sessionTokens" : 150000, "sessionCostUSD" : 0,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "qwen3.7-plus", "totalTokens" : 150000, "cost" : 0 }] }] }
    { "provider" : "zai", "source" : "local", "sessionTokens" : 180000, "sessionCostUSD" : 0.3224,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "glm-5v-turbo", "totalTokens" : 180000, "cost" : 0.3224 }] }] }
    { "provider" : "deepseek", "source" : "local", "sessionTokens" : 160000, "sessionCostUSD" : 0.08703625,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "deepseek-v4-pro", "totalTokens" : 160000, "cost" : 0.08703625 }] }] }
    The Alibaba token-plan zero is expected from the current public models.dev catalog for alibaba-token-plan/qwen3.7-plus.

Verification

  • CLANG_MODULE_CACHE_PATH=/private/tmp/codexbar-clang-cache swift test --disable-sandbox --scratch-path /Users/mosaab/Documents/Projects/CodexBar/.build --filter PiSessionCostScannerTests: 19 tests passed.
  • CLANG_MODULE_CACHE_PATH=/private/tmp/codexbar-clang-cache swift test --disable-sandbox --scratch-path /Users/mosaab/Documents/Projects/CodexBar/.build --filter 'provider lookup resolves current China API models': 1 test passed.
  • git diff --check: clean.
  • make check completed generated parser hash, package/docs/locales checks, SwiftFormat lint mode, and SwiftLint strict with 0 violations, then exited on the sandbox-only macOS plist cache write permission error after lint shutdown.

No live provider probes, browser-cookie imports, or Keychain reads were run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac458da2f6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift Outdated
Comment thread Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift Outdated
@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Jul 27, 2026
@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codex review: found issues before merge. Reviewed August 12, 2026, 5:14 AM ET / 09:14 UTC.

ClawSweeper review

What this changes

This PR adds models.dev pricing lookup and Pi-session cost reporting for Alibaba plan, z.ai, and DeepSeek providers in CodexBar’s cost command.

Merge readiness

⚠️ Needs maintainer review before merge - 4 items remain

Keep open: the prior P1 blocker remains on the current head, which can estimate costs from the wrong regional catalog; the branch is also dirty against current main.

Priority: P2
Reviewed head: a3b98f87f0469904172f4719682d6b1cc9af33bb

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The runtime CLI proof is useful, but the unresolved regional-routing defect makes the patch not ready to merge.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The PR body provides after-fix built-CLI output from isolated synthetic Pi sessions for each added provider.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The PR body provides after-fix built-CLI output from isolated synthetic Pi sessions for each added provider.
Evidence reviewed 5 items Regional lookup remains unresolved: The new lookup accepts only UsageProvider and always tries the international Alibaba catalogs before their China variants; z.ai has no BigModel-CN catalog path.
Persisted z.ai regional behavior: Current and released source distinguish Global api.z.ai from BigModel CN open.bigmodel.cn, so provider identity alone cannot select the correct rate source.
Cost call site drops configured region: The CLI loads configuration but calls CostUsageFetcher with only the provider, leaving no region/catalog selection available to the new lookup path.
Findings 1 actionable finding [P1] Route price selection through the configured region
Security None None.

How this fits together

CodexBar reads local Pi session records and uses cached models.dev prices to estimate provider costs for CLI and dashboard output. The selected provider’s stored region must determine which vendor catalog supplies those rates.

flowchart LR
A[Pi session records] --> C[Cost scanner]
B[Provider and region settings] --> C
C --> D[Catalog selection]
E[Cached models.dev catalog] --> D
D --> F[Cost estimate]
F --> G[CLI and dashboard output]
Loading

Before merge

  • Route price selection through the configured region (P1) - This repeats the outstanding prior blocker. The lookup only receives UsageProvider, so Alibaba always checks the international catalog before -cn, while a z.ai BigModel-CN selection cannot choose its regional catalog. Overlapping model IDs can have different rates, silently producing incorrect estimates. Thread the configured region or explicit catalog through this path and test divergent regional fixtures.
  • Resolve merge risk (P1) - Alibaba China-mainland and z.ai BigModel-CN users can receive estimates from a different catalog when model IDs overlap but rates differ.
  • Resolve merge risk (P1) - GitHub reports the head as merge-dirty against current main; it needs a rebase and review refresh before landing.
  • Complete next step (P2) - The head is merge-dirty and needs contributor or maintainer handling to rebase and repair regional catalog routing before another merge review.

Findings

  • [P1] Route price selection through the configured region — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:794-799
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta production +202/-33, tests +316/-85, 1 generated line changed The pricing feature spans cost scanning, provider descriptors, CLI support, and focused regression coverage.

Merge-risk options

Maintainer options:

  1. Rebase and make catalog selection region-aware (recommended)
    Pass the configured Alibaba and z.ai region or an explicit catalog ID through the cost path, then add divergent-region regression coverage before merge.

Technical review

Best possible solution:

Carry an explicit selected pricing catalog or persisted provider region into the cost scanner, select one regional catalog rather than an ordered fallback, and cover divergent international and China fixtures.

Do we have a high-confidence way to reproduce the issue?

Yes—source-reproducible: use a fixture where the same model has different international and China-region prices, configure Alibaba CN or z.ai BigModel CN, and the current provider-only lookup selects the wrong catalog.

Is this the best way to solve the issue?

No. Ordered catalog fallback cannot represent an already persisted regional choice; the selected region or catalog must be passed into pricing resolution.

Full review comments:

  • [P1] Route price selection through the configured region — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:794-799
    This repeats the outstanding prior blocker. The lookup only receives UsageProvider, so Alibaba always checks the international catalog before -cn, while a z.ai BigModel-CN selection cannot choose its regional catalog. Overlapping model IDs can have different rates, silently producing incorrect estimates. Thread the configured region or explicit catalog through this path and test divergent regional fixtures.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a90dfed5c264.

Labels

Label changes:

  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🦐 gold shrimp, so this older rating label is no longer current.

Label justifications:

  • P2: This is a bounded provider-cost accuracy issue without evidence of an urgent service outage.
  • merge-risk: 🚨 compatibility: Merging the current fallback order can silently change cost estimates for existing regional provider configurations.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body provides after-fix built-CLI output from isolated synthetic Pi sessions for each added provider.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides after-fix built-CLI output from isolated synthetic Pi sessions for each added provider.

Evidence

What I checked:

Likely related people:

  • steipete: Reviewed this pricing series, requested its rebase/stacking, and is the recorded author of the released provenance for the z.ai region selector. (role: recent area reviewer; confidence: high; commits: 330ae4384b18; files: Sources/CodexBarCore/Providers/Zai/ZaiAPIRegion.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Rebase onto current main and resolve the dirty merge state.
  • Pass selected region/catalog through pricing resolution and add divergent-region tests.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (29 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-08T05:04:22.787Z sha 740a86b :: needs changes before merge. :: [P1] Honor the configured pricing region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T11:26:13.611Z sha 740a86b :: found issues before merge. :: [P1] Route pricing by the configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T13:48:39.921Z sha 740a86b :: found issues before merge. :: [P1] Honor the configured pricing region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T16:55:47.392Z sha 740a86b :: needs changes before merge. :: [P1] Route pricing by the configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T22:59:22.842Z sha 740a86b :: needs changes before merge. :: [P1] Select pricing catalog from configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T05:44:46.604Z sha 740a86b :: found issues before merge. :: [P1] Route price catalogs by configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T14:32:38.447Z sha 5ce6ad6 :: needs changes before merge. :: [P1] Route price catalogs by configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T22:38:22.265Z sha a3b98f8 :: needs changes before merge. :: [P1] Select the pricing catalog from the configured region

@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 28, 2026
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 68c556c to bccd82b Compare July 31, 2026 17:24
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 31, 2026
@steipete

steipete commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Thanks for the models.dev pricing series! The China API provider mappings (Alibaba, z.ai, DeepSeek) here haven't landed on main (the scanner still only maps anthropic/openai), so this is still valuable — but the branch has been merge-dirty since July 21, and #2374/#2376/#2377 all touch the same files (PiSessionCostScanner.swift, CostUsagePricing.swift, CLICostCommand.swift), so they'll conflict with each other as well as with main. Could you rebase within the next two weeks, ideally picking one PR to land first and stacking the other two on top afterwards?

@clawsweeper clawsweeper Bot added merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Aug 6, 2026
…DevCostUSD out of enum to satisfy lint limits
The hash commits were dropped during the rebase. Regenerate the final hash now that all China pricing changes are applied.
ClawSweeper flagged that .alibaba and .alibabatokenplan fell back to the direct 'alibaba' catalog when their dedicated plan catalogs lacked a model. Plan billing can differ from direct API billing, so return no estimate instead. Add regression coverage for missing plan-catalog rows.
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 740a86b to 5ce6ad6 Compare August 9, 2026 14:28
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Aug 9, 2026
@joeVenner

Copy link
Copy Markdown
Contributor Author

@steipete — this PR has been rebased onto latest main and is ready for review. Thanks!

After rebasing the China API models.dev pricing work, ProviderArchitectureGatekeeperTests failed because new provider-specific branches (Alibaba, Zai, DeepSeek) appeared in shared cost code and existing allowlist anchors shifted. Add '// Provider-specific by design:' markers at each provider-owned dispatch point, update the gatekeeper's hardcoded supportsTokenSnapshot set and shifted suppressed references, and remove obsolete CostUsageFetcher/PiSessionCostScanner allowlist entries whose anchors no longer match.

Closes provider-architecture gatekeeper failures on the China API rebase.
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants