Skip to content

Harden unattributed Codex cost handling - #2069

Merged
steipete merged 1 commit into
mainfrom
codex/2061-unattributed-model-hardening
Jul 11, 2026
Merged

Harden unattributed Codex cost handling#2069
steipete merged 1 commit into
mainfrom
codex/2061-unattributed-model-hardening

Conversation

@steipete

Copy link
Copy Markdown
Owner

Summary

  • keep model-less Codex token events explicitly unattributed and unpriced, even if the pricing catalog contains a colliding generic entry
  • distinguish omitted model fields from explicit blank fields across the byte-fast, oversized-prefix, and Foundation fallback parsers so blank contexts clear stale attribution safely
  • avoid on-demand pricing downloads for deliberately unattributed usage
  • refine the existing changelog entry and retain contributor credit for @hhh2210

Follow-up hardening for #2061 after its contributor head landed.

Verification

  • 164 focused parser/pricing/fetcher tests across 5 suites
  • make check
  • make test — 610 selections, all 51 shards green, zero retries/timeouts
  • AutoReview clean on the exact rebased head
  • exact Developer ID-signed debug bundle: strict deep code-sign verification and Gatekeeper acceptance
  • bundled CLI, isolated synthetic Codex home, isolated pricing cache, Keychain disabled, network denied
    • 6 session fixtures produced the expected 6 model breakdowns and 2,142 total tokens
    • unattributed breakdown remained visible with 102 tokens and no cost despite a priced cache collision
    • explicit blank, incomplete/complete oversized-prefix, nested fallback, and Foundation fallback attribution all matched the regression contract
    • stderr empty; zero established TCP connections; production CodexBar PID unchanged

Risk

Low and parser-localized. The new distinction is covered through every Codex parsing path, cache collision behavior, and the packaged CLI. Claude behavior is unchanged.

Dependencies

No dependency changes; freshness not applicable.

Public model identifier gate

PASS. New test values are explicitly fictitious; the generic unattributed sentinel is not a provider model identifier. The sole real model family named in user-facing text is documented in the official OpenAI model catalog: https://platform.openai.com/docs/models

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal priority bug or improvement with limited blast radius. labels Jul 11, 2026
@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed July 11, 2026, 2:53 PM ET / 18:53 UTC.

Summary
The PR distinguishes omitted and explicitly blank Codex model fields across parser paths, keeps deliberately unattributed usage unpriced, and prevents that usage from triggering pricing-catalog refreshes.

Reproducibility: yes. at source and fixture level: explicit blank turn-context fields after a previously attributed model exercise stale attribution, while an unknown catalog collision exercises false pricing. The PR also reports after-fix packaged CLI validation for these paths.

Review metrics: 3 noteworthy metrics.

  • Patch surface: 11 files, +564/-36. The branch is a single-commit follow-up with bounded production changes and extensive regression coverage.
  • Logic and test balance: 6 logic files and 3 test files. The three affected parser/pricing behaviors receive substantially more test expansion than production-code growth.
  • Parser paths covered: 3 implementations. Byte-fast, oversized-prefix, and Foundation fallback parsing now share and test the same model-field semantics.

Root-cause cluster
Relationship: same_root_cause
Canonical: #2061
Summary: This PR hardens edge cases in the same missing-model attribution and pricing path addressed by the merged predecessor.

Members:

  • canonical: Fix unknown Codex model attribution #2061 - The merged predecessor removed false GPT-5 attribution and established the unknown-model behavior that this branch makes blank-field-aware and catalog-collision-safe.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • [P2] No repair lane is needed; this clean owner-authored implementation should proceed through routine required checks and maintainer merge handling.

Security
Cleared: The patch adds no dependencies, privileges, secret handling, executable downloads, or supply-chain changes and narrows network refresh behavior.

Review details

Best possible solution:

Keep unattributed usage visible but permanently unpriced, while omitted model fields preserve prior context and explicit blank fields clear it consistently across every Codex parser path.

Do we have a high-confidence way to reproduce the issue?

Yes at source and fixture level: explicit blank turn-context fields after a previously attributed model exercise stale attribution, while an unknown catalog collision exercises false pricing. The PR also reports after-fix packaged CLI validation for these paths.

Is this the best way to solve the issue?

Yes. The patch narrowly extends the recently merged design with one shared field-presence resolver and one shared unattributed sentinel rather than adding parallel parsing or pricing behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 0bb874930d6a.

Label changes

Label changes:

  • add P2: The PR fixes inaccurate Codex cost attribution and unnecessary refresh activity in a bounded cost-history surface.
  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply to this owner-authored PR, although its body includes unusually strong after-fix packaged CLI validation.

Label justifications:

  • P2: The PR fixes inaccurate Codex cost attribution and unnecessary refresh activity in a bounded cost-history surface.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply to this owner-authored PR, although its body includes unusually strong after-fix packaged CLI validation.
Evidence reviewed

What I checked:

Likely related people:

  • steipete: Dominant contributor across the affected cost-usage files, co-author of the merged predecessor, and author of this direct follow-up hardening. (role: feature owner; confidence: high; commits: 31cdb24880fb, 917fc7229f10, 69de57b85de7; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift, Sources/CodexBarCore/CostUsageFetcher.swift)
  • hhh2210: Authored the merged unknown-model attribution implementation and the earlier oversized turn-context attribution repair that this branch extends. (role: introduced behavior; confidence: high; commits: 300ddee98ec7, 036b49755aef; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Tests/CodexBarTests/CostUsageScannerBreakdownTests.swift)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete

Copy link
Copy Markdown
Owner Author

Maintainer proof for exact head 31cdb24880fbda31154cee0ff65e4b1546fc9c53:

  • 164 focused parser/scanner regressions passed.
  • make check passed; full make test passed all 610 selected tests in 51 groups with no retries or failures.
  • Exact-head autoreview completed with no actionable findings.
  • Developer ID-signed debug bundle passed strict code-signing and Gatekeeper checks and embedded commit 31cdb248.
  • Isolated deny-network bundled-CLI QA processed six controlled session fixtures and produced all six expected breakdowns. The unattributed sentinel retained its token count while correctly omitting cost even when the test cache deliberately assigned it a price; blank, oversized-line, and Foundation decoding paths matched.
  • Hosted CI is fully green: both macOS shards, both Linux architectures, lint/build/test, changes, lint, and security.
  • Public Model Identifier Gate: PASS.

No production CodexBar process or real provider credential was touched during QA.

@steipete
steipete merged commit 917a216 into main Jul 11, 2026
8 checks passed
@steipete
steipete deleted the codex/2061-unattributed-model-hardening branch July 11, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant