Skip to content

feat: show remaining usage for selected model - #16

Merged
MaximilianMauroner merged 5 commits into
mainfrom
feat/selected-model-usage
Jul 28, 2026
Merged

feat: show remaining usage for selected model#16
MaximilianMauroner merged 5 commits into
mainfrom
feat/selected-model-usage

Conversation

@MaximilianMauroner

Copy link
Copy Markdown
Owner

Plan: https://html-publisher-production-a236.up.railway.app/p/wFjg3Cn1GOu_JHpcs7V4UH22YDa2qeU1

Local plan: ~/.plans/selected-model-usage-bf14ed2c-e54d-4491-98b7-63c5f8c2ab64.html

Done criteria

  • Show a compact remaining-usage status for the active Codex provider instance and selected model, with reset details.
  • Switching model/provider updates or hides the indicator without leaking stale data.
  • Use a typed, defensive normalized shape with non-fatal transient failures.
  • Preserve unsupported/provider readiness behavior.
  • Add focused backend, contract, and browser/component coverage.
  • Complete an integrated web verification pass.

Approach

Reuse targeted, proven pieces from upstream PR pingdotgg#2484, informed by pingdotgg#2155 and pingdotgg#4326. Do not cherry-pick wholesale. Adapt model/bucket resolution so the displayed quota applies only to the currently selected model.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XS labels Jul 27, 2026
@github-actions github-actions Bot added size:L and removed size:XS labels Jul 27, 2026
@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 1 — HIGH: model bucket matching is not exact

packages/shared/src/codexUsage.ts:36-42,73-76 uses substring containment for an “exact” selected-model match. This can map gpt-5.3 to gpt-5.3-codex, or gpt-5.3-codex to gpt-5.3-codex-mini, and confidently display the wrong quota.

Require normalized equality. If aliases are needed, use an explicit tested alias map. Add prefix/suffix collision regression tests.

@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 1 — HIGH: unsupported Codex auth is not gated

apps/server/src/provider/Layers/ProviderService.ts:1048-1061 and the composer mount path check only driverKind === codex. API-key/unknown accounts can still issue account/rateLimits/read every minute and potentially render returned data.

Gate this server-authoritatively on ChatGPT subscription auth (and optionally hide immediately in the UI), return null without reading for unsupported auth, and add tests proving no read/no indicator.

@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 1 — MEDIUM: fallback cache refreshes the apparent observation time

apps/server/src/provider/Layers/CodexAdapter.ts:1900-1917 caches only raw payload, while normalization generates checkedAt at fallback time. Old data can therefore look newly checked after repeated transient failures/notifications.

Cache the authoritative observed/read timestamp and source with the payload, preserve them during fallback, and add a clock-controlled transient-failure/staleness test.

@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 1 — MEDIUM: missing legacy limitId is synthesized as shared Codex

packages/shared/src/codexUsage.ts:84-89 sets a missing legacy identity to codex, ignoring a potentially model-specific limitName. A bucket for one model can then be displayed for another.

Evaluate supplied identities conservatively: only treat an explicitly identified generic Codex bucket as shared; otherwise require an exact selected-model identity or return unavailable. Add a partial/malformed payload regression test.

@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 2 — HIGH: cache can survive an account transition

apps/server/src/provider/Layers/CodexAdapter.ts:1905-1944 returns null when an authoritative account read changes from ChatGPT to API key, but does not invalidate the previous ChatGPT usage cache. A later transient account-read failure can return that old subscription snapshot, and notifications can keep mutating it.

Associate cache ownership with eligible account identity/generation, clear it on authoritative non-ChatGPT/account transitions, and merge notifications only into an eligible generation. Add ChatGPT seed → API-key transition → later account-read failure coverage.

@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 2 — MEDIUM: sparse window updates discard reset/duration metadata

apps/server/src/provider/Layers/CodexAdapter.ts:1523-1532 shallow-spreads bucket updates, replacing primary/secondary window objects. A sparse { primary: { usedPercent } } notification drops cached windowDurationMins and resetsAt.

Deep-merge primary/secondary window fields with defined null semantics for top-level and keyed buckets. Assert duration/reset survive a sparse update.

@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Review cycle 2 — MEDIUM: older reads can overwrite newer notifications

apps/server/src/provider/Layers/CodexAdapter.ts:1514-1535,1905-1924 does not order/version cache writes. A full read started before a notification can complete afterward and overwrite the newer notification while looking current.

Serialize mutations or guard full-read replacement with a cache generation captured before the request. Add controlled notification-during-read and inverse-order race tests.

@github-actions github-actions Bot added size:XL and removed size:L labels Jul 27, 2026
@MaximilianMauroner

Copy link
Copy Markdown
Owner Author

Final implementation verification

  • Adversarial review cycles: 3
  • Actionable findings fixed: 7
  • Accepted non-blocking nit: contract schemas permit broader numeric ranges than the sole normalized producer emits
  • Final correctness review: clean
  • Final plan/architecture review: clean
  • Focused implementation evidence: 99 tests passing in the final affected set; targeted format/lint and diff checks clean
  • Integrated isolated web pass: authenticated app, no-thread composer, Codex→Claude switch, desktop 1280×720 layout, and compact 390×844 layout verified
  • Integrated live-usage caveat: populated remaining percentages, reset tooltip, and A→B populated-data transition were blocked because the disposable environment returned no Codex usage payload; they were not claimed as passed
  • Typecheck caveat: the corrected package typecheck was not rerun because the repository one-heavy-check guard had already been consumed

All dev processes and disposable state were torn down.

@MaximilianMauroner
MaximilianMauroner marked this pull request as ready for review July 27, 2026 15:33
Copilot AI review requested due to automatic review settings July 27, 2026 15:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b54c58de9f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/server/src/ws.ts
}),
[WS_METHODS.serverGetCodexUsage]: (input) =>
observeRpcEffect(
WS_METHODS.serverGetCodexUsage,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Declare a scope for the Codex usage RPC

observeRpcEffect immediately calls requiredScopeForMethod, but RPC_REQUIRED_SCOPE has no serverGetCodexUsage entry. Every indicator poll therefore throws before providerService.getCodexUsage can run, so the new UI can never receive usage data; add the appropriate read scope and corresponding authorization error to the RPC contract.

AGENTS.md reference: AGENTS.md:L10-L11

Useful? React with 👍 / 👎.

Comment on lines +74 to +76
const exact = entries.filter(([key, bucket]) => matchesModel(model, key, bucket));
let selected: readonly [string, CodexUsageRawBucket] | null =
exact.length === 1 ? exact[0]! : null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve model slugs to metered limit IDs

The protocol defines rateLimitsByLimitId as keyed by metered limit IDs such as codex (packages/effect-codex-app-server/src/_generated/schema.gen.ts:37871-37875), not necessarily by model slugs. For a normal multi-bucket response such as codex plus another metered bucket, no entry exactly matches gpt-5.3-codex, and the generic fallback is explicitly disabled by the competing bucket, causing this function to return null; map the selected model to its metered limit ID before selecting the bucket.

AGENTS.md reference: AGENTS.md:L34-L36

Useful? React with 👍 / 👎.

Comment on lines +95 to +97
const windows = [selected[1].primary, selected[1].secondary]
.filter((window): window is CodexUsageRawWindow => window !== null && window !== undefined)
.flatMap((window, index) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the window index when assigning fallback labels

Filtering before flatMap renumbers the surviving windows. When the optional primary window is absent but the secondary window is present without windowDurationMins, the secondary window receives index 0 and is displayed as Session rather than Weekly; retain the original primary/secondary position while filtering invalid windows.

AGENTS.md reference: AGENTS.md:L34-L36

Useful? React with 👍 / 👎.

Comment on lines +1915 to +1917
}).pipe(
Effect.provideService(Scope.Scope, usageScope),
Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, childProcessSpawner),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Provide Crypto to the standalone Codex runtime

When no Codex session is active—the normal flow before the first turn—this path uses the live makeCodexSessionRuntime, which requires Crypto.Crypto (CodexSessionRuntime.ts:719-725), but only Scope and ChildProcessSpawner are provided here. Runtime construction therefore fails with a missing-service defect before account/read, so usage cannot be fetched until an existing session is available; provide the already captured crypto service as the session-start path does.

AGENTS.md reference: AGENTS.md:L7-L7

Useful? React with 👍 / 👎.

Comment on lines +1557 to +1559
const limitId = update?.limitId?.trim();
if (update && limitId) codexUsageGeneration += 1;
if (cachedCodexUsage && update && limitId) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refetch when a sparse update omits its limit ID

The generated protocol permits limitId to be absent or null in account/rateLimits/updated and explicitly describes the notification as a sparse update that must be merged or followed by a refetch (packages/effect-codex-app-server/src/_generated/schema.gen.ts:36398-36406). This guard silently discards every such update, leaving the displayed percentage stale until the next periodic read; when the cached bucket cannot be identified unambiguously, trigger a full account/rateLimits/read instead of ignoring the notification.

AGENTS.md reference: AGENTS.md:L34-L36

Useful? React with 👍 / 👎.

Comment on lines +20 to +21
providerInstanceId: props.providerInstanceId,
model: props.model,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invalidate usage data when the provider account changes

The query identity contains only environment, provider instance, and model, so reauthenticating the same instance from one ChatGPT account to another continues exposing the previous account's successful snapshot. The server invalidates its own cache on account/updated, but nothing invalidates this mounted client atom, and the returned snapshot echoes the same provider/model identifiers, so it remains visible until the 60-second refresh; include an account/provider revision in the query identity or explicitly clear the atom when provider authentication changes.

AGENTS.md reference: AGENTS.md:L10-L11

Useful? React with 👍 / 👎.

@MaximilianMauroner
MaximilianMauroner merged commit 55a94f0 into main Jul 28, 2026
6 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants