Fix Codex usage history indexing and catch-up completion

[Quicksaver]

Summary

Fix Codex Usage & Spend indexing for very large local histories. The scanner now keeps bounded passes bounded across discovery, candidate selection, file parsing, and cache reconciliation, while avoiding corpus-wide cache searches for every session file.

The catch-up state is also made durable and self-healing: completed files can leave stale lookback queues, completed caches no longer restart from zero after relaunch, appended sessions resume correctly, and refreshes cannot lose the worker that should process newly discovered tail work.

What Changed

• Cap time-limited Codex catch-up candidate work and check the byte/time budget before starting additional files or discovery work.
• Page current-window, flat-root, and older-partition directory discovery through a persisted 512-entry cursor instead of listing and sorting the entire active corpus before candidate selection.
• Build one file-identity alias index per refresh instead of scanning the entire cache for every session file.
• Use normalized path sets for discovery and deduplication, including /private/var aliases.
• Allow already-complete cached files retained in a stale lookback queue to be scheduled once and acknowledged as complete.
• Reconcile at most 512 persisted stale-lookback paths per cache load/save and defer exact full-inventory proof until no bounded work remains.
• Prepare at most 512 retained lookback paths per bounded pass, so root validation and URL materialization cannot walk the full queue before candidate selection.
• Keep post-scan progress accounting incremental during catch-up; exact byte totals, metadata validation, and sorted inventory persistence now run only after bounded work is exhausted.
• Persist an explicit completion-proof sentinel so the final non-empty work slice cannot trigger full accounting or rediscover the newest sessions; the following zero-work pass performs the exact proof and publishes a distinct completion progress key.
• Persist the exact current scan inventory and reconcile stale catch-up metadata only when every file in that inventory is complete.
• Preserve completion across relaunches when APFS exposes the same inode with a different device component, while validating size, mtime, and the indexed-content anchor before reusing cached state.
• Normalize persisted file identities against the current Codex root device and invalidate completion safely when a file was appended or changed.
• Queue one same-mode catch-up restart for both the primary Codex view and Spend Dashboard so a refresh that discovers tail work cannot race with worker teardown, while ensuring both Stop paths clear queued restarts before an active pass exits.
• Add work counters and regression coverage for bounded large-corpus scans, linear alias lookup, exact inventory completion, relaunch identity drift, appends, cancellation, and catch-up restart races.

Why

A real Codex history of approximately 6.8 GB exposed several compounding failures. Initial indexing became progressively slower as more files entered the cache, eventually appearing stalled after hours. Once the scanner was made faster, it could reach 100% but still fail while finalizing, leaving charts unavailable. Relaunching CodexBar then restarted the multi-gigabyte analysis, and later attempts repeatedly advanced only a small amount before returning to the same error state.

The symptoms were not one isolated failure: they combined unbounded work outside the parser budget, quadratic cache-identity lookup, stale retained-lookback entries, inconsistent persisted completion metadata, filesystem identity drift across relaunches, and a worker lifecycle race.

Evidence

Incident observations

• The original corpus was about 6.68 GB and continued growing with normal Codex use.
• Throughput initially advanced by megabytes every few minutes, then degraded to dozens of megabytes, then only a few megabytes; after many hours only about 1.67 GB had been processed.
• After the first performance fixes, the same history rapidly reached roughly 6.83 GB / 6.83 GB, proving that raw parsing was no longer the dominant blocker, but finalization still errored and the data/charts were not published.
• Relaunching showed the analysis beginning again near 1.7 MB / 6.84 GB, demonstrating that completion was not being restored durably.
• Subsequent runs advanced only about 20 MB before reproducing the completed-analysis error, which pointed to stale pending state and worker/cache reconciliation rather than an unreadable source record.

Root-cause findings and corresponding fixes

1. Cache alias lookup grew quadratically with the corpus. Each file could search all cached files for a matching filesystem identity. The scanner now creates one identity-to-path index and visits only the matching bucket. A 1,500-file regression corpus asserts 1,500 alias candidates visited rather than approximately 2.25 million, processes no unchanged usage rows, and stays within the timing gate.
2. A bounded pass still performed unbounded candidate work. Large retained lookback queues could be fully filtered before the 512-file cap took effect. Time-limited passes now inspect at most 512 unique candidate paths before cache-eligibility checks, reuse the URL identities already established during discovery, and consult the time/byte budget before additional files and discovery work. Regression coverage verifies that two passes each inspect and scan exactly 512 files while preserving and advancing the remainder.
3. Completed files could remain permanently pending. Older cache state could retain an already-complete file in pendingFilePaths, while candidate filtering excluded complete files. That made the UI reach 100% but prevented finalization forever. Retained paths are now admitted once so completion can acknowledge and remove them.
4. Persisted aggregate progress could disagree with the current scan inventory. catchUpPending and zero/partial counters could survive after every current file was complete. Conversely, counting every file retained from a wider history window could falsely satisfy the smaller current-scan count while current files were still absent. The scanner now persists the exact current inventory paths, and cache loading verifies every one against current metadata and indexed-content anchors before clearing pending state. Wider retained files cannot satisfy that proof.
5. Filesystem identity was not stable enough across relaunches. APFS can expose the same inode with a different st_dev component. Treating the full persisted device:inode value as permanently stable caused completed files to look new after reopening the app. Restore now maps persisted inodes to the current Codex root device and validates mtime, size, and the token-index content anchor before adopting the current identity.
6. Appended files needed precise invalidation and resume. If metadata differs after identity normalization, unchanged complete files remain reusable, genuine appends retain their identity but become incomplete, and rewrites drop reusable scan state. Tests reconstruct both stale-progress and append-after-completion caches and verify a single-file resume returns them to an exact completed state.
7. Same-mode refreshes could lose their catch-up worker. A refresh could discover new tail work while an existing automatic worker was finishing; the old early return then allowed teardown to leave pending work with no owner. Primary and Spend Dashboard catch-up now queue one restart and launch it after the current task clears its token and scope.
8. Cancel could immediately restart an active catch-up pass. Both the primary and Spend Dashboard stop paths returned while a pass was running before clearing an already queued restart. Deferred teardown could then launch another worker and reset the stop flag. Both Stop paths now clear their queued restart before that return.
9. Persisted reconciliation bypassed the bounded scanner. Cache load and save synchronously validated every retained lookback path before candidate scheduling, and exact inventory proof was evaluated before cheap pending-work guards. Reconciliation now validates at most 512 pending paths per invocation without first indexing the full cache, while full current-inventory validation runs only when no active lookback, discovery, or incomplete file work remains.
10. Post-scan progress accounting bypassed the bounded pass. After scanning at most 512 candidates, the scanner still sorted every discovered path and fetched metadata for the whole corpus to rebuild byte/file progress and inventory state. Catch-up now advances file progress from the retained lookback state without a corpus traversal, marks byte totals indeterminate while that exact denominator is unavailable, and performs the exact metadata/inventory proof once no bounded work remains.
11. Retained-lookback preparation bypassed candidate limits. Before the 512-candidate selector ran, every persisted pending path was root-validated and materialized. Bounded passes now prepare at most 512 paths and feed only that slice to selection and finalization; a no-op discovery pass also avoids reconstructing and sorting the unchanged full pending queue.
12. The final work slice could never reach a distinct proof pass. Deferring exact accounting without a durable marker allowed cache reconciliation to discard the empty lookback state, after which the newest active files were rediscovered and retried forever. The scanner now persists an explicit awaitingExactInventory sentinel, preserves it across the store round trip, clears it on the following zero-work pass, and includes pending/progress totals in the catch-up status identity so charts publish the completed snapshot.
13. Discovery itself bypassed every downstream bound. The scanner listed and sorted every JSONL file in the current date window, walked the flat root, and reconstructed cached-file candidates before applying the 512-file selector. Oversized active-day directories therefore still imposed corpus-sized work on every pass. Discovery now shares the 512-entry work budget across current-window, flat-root, and older date-partition pages, persists the logical day/directory cursor, and defers recursive legacy discovery plus the exact cached inventory to the final proof pass. Relaunch cursor rehydration is also bounded and does not reparse session JSONL.
14. The parser-hash update rejected every v0.49.2 Cost Usage database. The store now recognizes the exact released v0.49.2 parser hash as a compatible scheduling-only upgrade, updates the schema/hash atomically, and retains completed file rows. A focused upgrade regression verifies no rebuild occurs and all completed state remains available.
15. Device-drift recovery could stat the entire persisted corpus during load. Identity validation now stops at 512 files, persists the remainder in the active lookback queue, preserves that queue across scan-window resets, and removes entries only after a positively validated current snapshot. Missing or changed files stay queued for scanner handling; ordinary append/migration queues retain their existing behavior.

Automated regression evidence

• An oversized active-day directory with 1,500 JSONL files advances in bounded 512-entry discovery and scan slices without first listing, sorting, or progress-accounting over the full corpus. The first pass records first=512 discovery=512 attempts=512; after resetting the live directory handle, the bounded rehydration pass records relaunched=512 discovery=512 attempts=0; the following slice records second=1024 discovery=512 visits=512 attempts=512 accounting=0.
• Simulated relaunch rehydration proves that restoring the logical cursor remains bounded, does not reparse cached JSONL, preserves the 512 indexed files, and resumes forward progress on the following slice.
• Persisted cache save and the subsequent cache load each stop reconciliation at exactly 512 visits: save=512 load=512 pending=921.
• A 600-file finalization regression proves the final 88-file work slice performs no whole-inventory accounting, while the following zero-work pass performs exactly 600 accounting visits and clears pending state: finalWorkAttempts=88 finalWorkAccounting=0 proofAttempts=0 proofAccounting=600.
• Warm 1,500-file refreshes perform linear indexed alias lookup and parse zero unchanged usage rows.
• Retained complete files clear stale lookback state and repair processed/total byte and file counters.
• Completion reconciliation refuses to use a complete file from a wider retained window as proof for a missing current-inventory file.
• Completion survives persisted device-component drift only after content validation.
• Appended session files invalidate completion and converge again without replaying the whole corpus.
• Primary and Spend Dashboard same-mode refresh races retain a worker and finish pending tail work.
• Cancel during an active primary or Spend Dashboard pass clears any queued restart.
• Existing oversized-record, bounded-byte, wall-clock deadline, fork resume, sparse checkpoint, and exact-result convergence gates remain covered.

Validation

• CostUsageCatchUpCompletionTests: 5/5 passed, including a 600-file identity-drift corpus (512 validated, 88 deferred), missing-file retention, scan-window queue preservation, and normalized-alias completion.
• CostUsageBoundedFinalizationTests: 1 test passed, proving exact inventory work is isolated to the zero-work completion pass.
• UsageStoreCodexCostCatchUpTests: 6 tests passed, including primary active-pass cancellation with a queued restart.
• UsageStoreSpendDashboardCodexCostCatchUpTests: 6 tests passed, including active-pass cancellation with a queued restart.
• CostUsagePerformanceGateTests: 27 tests passed; oversized active-day discovery, candidate selection, JSONL scanning, and persisted reconciliation each stopped at 512 visits. Simulated relaunch rehydration performed zero JSONL scan attempts and preserved the 512 already-indexed files before the next slice advanced to 1,024.
• CostUsageBoundedProgressTests: 10/10 passed after the identity-queue repair, preserving existing append, rewrite, missing-file, cancellation, completion, and migration queue behavior.
• CostUsageStoreTests: 61/61 passed, including an exact v0.49.2 database upgrade that retains completed rows, installs the current schema/hash, and records zero rebuilds.
• ProviderArchitectureGatekeeperTests: 38 tests passed after rebasing its exact source anchors onto the latest merged upstream behavior.
• The latest 1,500-file warm-cache proof completed in 1,075 ms with 1,500 indexed lookups, 1,500 candidates visited, and zero unchanged usage rows parsed.
• make check passed, including generated-source checks, SwiftFormat, and strict SwiftLint with zero violations.
• Latest upstream/main at e5528d452 merged before final validation, including the macOS 15 Cost Usage store executor-isolation fix.
• Full make test reached 45 successful groups before the same unrelated, reproducible locale-sensitive MiniMaxMenuCardBillingTests failure on group 46 and its retry: the test expects comma-grouped 1,234/5,678, while this machine's current locale renders non-breaking-space grouping. The failing suite does not touch Codex cost scanning or any file changed by this branch.
• Manual real-data validation against the same approximately 6.8 GB Codex history that exposed the failures.

Proof

After the final fixes, the same approximately 6.83–6.84 GB local Codex dataset used to discover the performance and completion failures was quickly and successfully analyzed in only a few minutes. This validates the complete path against the original incident corpus, not merely a reduced fixture.

Redacted real-history debug log and persisted-store snapshot

The excerpt below is from the real incident dataset. Provider/account details, home paths, and spend values were removed. No conversation content was read or included.

[2026-08-10T18:54:00.362Z] [INFO] CodexBar startup [provider-state details redacted; Codex enabled]
[2026-08-10T18:54:31.626Z] [INFO] Codex cost scan applied work limits bytesConsumed=790493 deferredByBudget=0 deferredByTime=1 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=0
[2026-08-10T18:54:48.139Z] [INFO] cost usage success provider=codex duration=34.31s today=[REDACTED_SPEND] historyDays=30 windowCost=[REDACTED_SPEND]

[2026-08-10T18:57:02.992Z] [INFO] CodexBar startup [provider-state details redacted; Codex enabled]
[2026-08-10T18:57:27.755Z] [INFO] Codex cost scan applied work limits bytesConsumed=404384 deferredByBudget=0 deferredByTime=1 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=0
[2026-08-10T18:57:44.201Z] [INFO] cost usage success provider=codex duration=34.81s today=[REDACTED_SPEND] historyDays=30 windowCost=[REDACTED_SPEND]

[2026-08-10T19:57:26.551Z] [INFO] Codex cost scan applied work limits bytesConsumed=4600693 deferredByBudget=0 deferredByTime=1 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=0
[2026-08-10T19:57:43.578Z] [INFO] cost usage success provider=codex duration=31.53s today=[REDACTED_SPEND] historyDays=30 windowCost=[REDACTED_SPEND]

# Read-only query of the persisted cost-usage store after the final line above:
catch_up_pending=0
processed_bytes=6887714668
total_bytes=6887714668
completed_files=10949
total_files=10949
last_scan_utc=2026-08-10 19:57:12

This shows two distinct relaunches reusing the existing multi-gigabyte cache and scanning only 790,493 bytes and 404,384 bytes before successful publication, rather than replaying the corpus. The later 4,600,693-byte delta demonstrates appended history being incorporated. The persisted state then records exact byte/file completion with no pending catch-up.

A later read-only snapshot after normal Codex use showed the same cache still complete while the corpus had grown: catch_up_pending=0, processed_bytes=6898985724, total_bytes=6898985724, completed_files=10955, total_files=10955.

Current-head bounded traversal and completion-proof output

This current-head proof uses generated multi-thousand-file history and contains no provider identity, paths, spend, or conversation content.

[discovery-proof] first=512, discovery=512, attempts=512
[discovery-proof] relaunched=512, discovery=512, attempts=0
[discovery-proof] second=1024, discovery=512, visits=512, attempts=512, accounting=0
[alias-index-proof] warm refresh 1500 files: 1075 ms, lookups=1500, candidates=1500
[finalization-proof] finalWorkAttempts=88, finalWorkAccounting=0, proofAttempts=0, proofAccounting=600

The counters demonstrate that each pending phase is independently bounded: directory discovery, candidate selection, scanning, persisted reconciliation, and progress accounting. Relaunch rehydration visits only one bounded page, does not parse JSONL, and the next slice resumes forward progress. Exact whole-inventory accounting occurs only on the separate pass with zero file-scan attempts.

The same behaviors are also captured by synthetic regressions so the proof does not depend solely on one machine's local history.

Redacted current-head v0.49.2 upgrade and bounded identity-drift proof

The final source tree was run against an isolated copy of the real large-history database and read-only session corpus. The copy was relabeled with the exact v0.49.2 parser hash/schema and its stored device components were changed only inside that copy. Paths, identifiers, spend, identity values, and conversation content were neither printed nor retained.

[current-head-live-proof] mode=upgrade-drift-load elapsed_ms=9910 retained_files=10978 load_validation=512 pending=1
[current-head-live-proof] mode=relaunch-catch-up elapsed_ms=32170 retained_files=10978 attempts=512 discovery=512 accounting=0 pending=1 processed_bytes=0 total_bytes=0
[persisted-store] parser_hash=f1d04f67770b64cf schema=current retained_files=10978 pending_paths=9954

This proves the released database upgrades without deletion, cache load stops identity validation at exactly 512, the next relaunch performs exactly one bounded 512-file discovery/scan slice, and the remaining queue survives in the final-schema store. The temporary harness and database copy were deleted after the aggregate markers were captured; the real cache and roughly 7.1 GiB/12,400-file session corpus remained read-only.

Cache-wide migration queue proof

The latest maintainer follow-up closes a migration-specific hole in the bounded scanner. Pricing/model catalog, project metadata, turn-ID, priority metadata, and priority-turn migrations now reseed the already-discovered normalized path inventory into the existing durable active-lookback queue, even when roots and the scan window are unchanged and the old queue has drained. No new persisted schema is introduced: migration metadata can advance on the first pass because the queue itself owns every remaining revisit, and later passes drain it without reseeding.

The six-owner 600-file regression matrix proves the first bounded migration pass attempts exactly 512 files, retains 88 paths, and performs zero full progress-accounting visits. The second pass attempts exactly those remaining 88 paths and proves a path beyond the first slice was reparsed. The separate exact-validation pass reaches 600/600, and the following ordinary warm refresh performs zero candidate visits, reparses, or progress-accounting visits. Additional 513-file and partially drained 600-file tests prove migration reseeding preserves newest-first order while retaining any out-of-inventory pending work.

Current validation:

• Migration suite: 3 tests passed, including all 6 migration-owner cases.
• Existing bounded-progress suite: 10/10 passed, preserving append, rewrite, missing-file, cancellation, completion, and queue behavior.
• Requested focused scanner/store/catch-up/architecture matrix: 151 selections passed with the global reconciliation-hook suite run in isolation.
• Parser hash check and make check: passed; SwiftLint reported zero violations.
• Full make test reached 45 successful groups before the same unrelated, reproducible locale-sensitive MiniMaxMenuCardBillingTests failure on group 46 and its retry; the remaining 119 tests in that exact post-merge shard passed.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed August 11, 2026, 9:01 AM ET / 13:01 UTC.

ClawSweeper review

What this changes

The PR bounds large local Codex usage-history discovery, parsing, and cache recovery while preserving completed catch-up state across relaunches, appends, and the v0.49.2 upgrade.

Regression provenance

Possible regression — probable (reviewed change; failure trace). No predecessor PR is attributed.

Merge readiness

⚠️ Ready for maintainer review - 2 items remain

Keep this PR open for owner approval of the persisted-cache migration design; the current head has no discrete correctness finding from this review and includes sufficient real-history proof.

Priority: P1
Reviewed head: e50a58a252d4dc5742e27ef9b120ed6369b87a5c
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	Strong current-head proof and extensive focused coverage support the patch; the remaining blocker is owner approval of the persistence contract.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (logs): Redacted current-head live logs show bounded upgrade and relaunch behavior on a copied multi-gigabyte history, with persisted deferred work and no sensitive contents disclosed.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (logs): Redacted current-head live logs show bounded upgrade and relaunch behavior on a copied multi-gigabyte history, with persisted deferred work and no sensitive contents disclosed.
Evidence reviewed	5 items	Bounded catch-up implementation: The PR head creates a time-limited bounded discovery path, maintains a persisted active lookback queue, and schedules only the capped work slice. Upgrade compatibility path: The current head accepts only the explicit v0.49.2 parser-hash transition and updates schema version plus parser hash atomically. Bounded identity recovery: Cache restoration limits identity validation and defers remaining paths into the durable lookback queue; reconciliation only examines the bounded prefix.
Findings	None	None.
Security	None	None.

How this fits together

CodexBar scans local Codex session files into a persisted Cost Usage cache that supplies the menu-bar usage view and Spend Dashboard. This change governs how files are discovered, queued, reconciled, and marked complete before those views publish results.

flowchart LR
A[Local Codex session files] --> B[Bounded discovery cursor]
B --> C[Catch-up queue]
C --> D[Parser and cache update]
D --> E[Completion and migration validation]
E --> F[Menu-bar usage]
E --> G[Spend Dashboard]

Loading

Decision needed

Question	Recommendation
Does the owner approve active-lookback queue state as the durable migration continuation mechanism for existing Cost Usage stores?	Approve queue-only migration: Accept the bounded active-lookback queue as the migration continuation owner and proceed once required checks pass.

Why: The owner explicitly left this PR unmerged pending sign-off on persisted inventory/storage ownership; regression coverage cannot determine the intended long-term persistence contract.

Before merge

• Resolve merge risk (P1) - Merging commits to a compatibility-sensitive persisted Cost Usage cache migration whose queue-only ownership model still awaits owner approval.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Production and test delta	production +1,534/-126; tests +1,940/-26	The broad scanner and persisted-cache change is accompanied by more added regression coverage than production code.

Merge-risk options

Maintainer options:

1. Approve the compatibility contract (recommended)
   Confirm that existing v0.49.2 stores may retain cached rows through the parser-hash transition and use the queue-only continuation design.
2. Pause for a different migration boundary
   Keep the PR open if the persisted inventory and migration ownership need a different durable contract.

Technical review

Best possible solution:

Approve the queue-only migration contract if it is the intended durable storage boundary, then merge after required checks pass on the current head.

Do we have a high-confidence way to reproduce the issue?

Yes—focused regressions and the supplied redacted current-head live logs cover bounded discovery, migration, relaunch, and catch-up completion paths.

Is this the best way to solve the issue?

Unclear—the bounded queue implementation is well-supported, but the owner must still choose whether its persistence contract is the intended long-term design.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against e5528d452d4f.

Labels

Label changes:

• add proof: sufficient: Contributor real behavior proof is sufficient. Redacted current-head live logs show bounded upgrade and relaunch behavior on a copied multi-gigabyte history, with persisted deferred work and no sensitive contents disclosed.
• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): Redacted current-head live logs show bounded upgrade and relaunch behavior on a copied multi-gigabyte history, with persisted deferred work and no sensitive contents disclosed.
• remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.
• remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

• P1: The reported defect can leave large local Codex histories stuck, replaying, or unavailable to usage views.
• merge-risk: 🚨 compatibility: The patch changes parser-hash upgrade handling and restoration of persisted completed-file state.
• merge-risk: 🚨 availability: The patch changes catch-up worker restarts, cancellation, and bounded progress scheduling.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): Redacted current-head live logs show bounded upgrade and relaunch behavior on a copied multi-gigabyte history, with persisted deferred work and no sensitive contents disclosed.
• proof: sufficient: Contributor real behavior proof is sufficient. Redacted current-head live logs show bounded upgrade and relaunch behavior on a copied multi-gigabyte history, with persisted deferred work and no sensitive contents disclosed.

Evidence

What I checked:

• Bounded catch-up implementation: The PR head creates a time-limited bounded discovery path, maintains a persisted active lookback queue, and schedules only the capped work slice. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:5274, e50a58a252d4)
• Upgrade compatibility path: The current head accepts only the explicit v0.49.2 parser-hash transition and updates schema version plus parser hash atomically. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageStore.swift:336, e50a58a252d4)
• Bounded identity recovery: Cache restoration limits identity validation and defers remaining paths into the durable lookback queue; reconciliation only examines the bounded prefix. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageStore+CodexCache.swift:198, e50a58a252d4)
• Current-head provenance: The reviewed head is the merge commit containing the v0.49.2-store and identity-validation follow-ups; it is not contained by checked-out main or a local release tag. (e50a58a252d4)
• Owner review context: The repository owner stated that the queue-only migration design remains intentionally unmerged pending owner sign-off on persisted inventory/storage ownership. (11b9cd7d9847)

Likely related people:

• steipete: The repository owner authored the bounded queue and migration follow-ups on this PR and explicitly reserved final sign-off on storage ownership. (role: likely follow-up owner; confidence: high; commits: ff442402c65f, 11b9cd7d9847; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsageStore+CodexCache.swift)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Obtain explicit owner sign-off on the queue-only persisted-cache migration design.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (9 earlier review cycles; latest 8 shown)

• reviewed 2026-08-10T20:18:49.469Z sha e0e1f8b :: needs changes before merge. :: [P2] Clear the primary queued restart before returning from Stop
• reviewed 2026-08-10T21:12:48.649Z sha a1aab85 :: needs changes before merge. :: [P2] Bound candidate selection before applying the cap
• reviewed 2026-08-10T21:55:51.930Z sha 3f7c44d :: needs changes before merge. :: [P2] Bound persisted lookback reconciliation
• reviewed 2026-08-10T22:15:59.809Z sha 295987c :: found issues before merge. :: [P2] Keep post-scan accounting inside the bounded pass
• reviewed 2026-08-10T22:39:05.157Z sha 40ff7ee :: needs real behavior proof before merge. :: [P2] Bound retained-lookback preparation before selection
• reviewed 2026-08-11T08:16:46.797Z sha 31c1c2b :: needs changes before merge. :: [P2] Bound discovery before building the candidate corpus
• reviewed 2026-08-11T08:30:31.564Z sha 11b9cd7 :: found issues before merge. :: [P2] Bound discovery before constructing the candidate corpus
• reviewed 2026-08-11T10:40:18.510Z sha 5cdc017 :: needs real behavior proof before merge. :: [P1] Preserve existing Cost Usage stores across upgrade | [P2] Bound identity-drift validation during cache load

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a976c44e4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Quicksaver]

Responding to the request for inspectable real-behavior proof: the PR body now contains this same redacted artifact under Proof.

Redacted real-history debug log and persisted-store snapshot

Provider/account details, home paths, and spend values were removed. No conversation content was read or included.

[2026-08-10T18:54:00.362Z] [INFO] CodexBar startup [provider-state details redacted; Codex enabled]
[2026-08-10T18:54:31.626Z] [INFO] Codex cost scan applied work limits bytesConsumed=790493 deferredByBudget=0 deferredByTime=1 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=0
[2026-08-10T18:54:48.139Z] [INFO] cost usage success provider=codex duration=34.31s today=[REDACTED_SPEND] historyDays=30 windowCost=[REDACTED_SPEND]

[2026-08-10T18:57:02.992Z] [INFO] CodexBar startup [provider-state details redacted; Codex enabled]
[2026-08-10T18:57:27.755Z] [INFO] Codex cost scan applied work limits bytesConsumed=404384 deferredByBudget=0 deferredByTime=1 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=0
[2026-08-10T18:57:44.201Z] [INFO] cost usage success provider=codex duration=34.81s today=[REDACTED_SPEND] historyDays=30 windowCost=[REDACTED_SPEND]

[2026-08-10T19:57:26.551Z] [INFO] Codex cost scan applied work limits bytesConsumed=4600693 deferredByBudget=0 deferredByTime=1 maxBytesPerRefresh=536870912 maxFileBytes=268435456 partialFiles=0
[2026-08-10T19:57:43.578Z] [INFO] cost usage success provider=codex duration=31.53s today=[REDACTED_SPEND] historyDays=30 windowCost=[REDACTED_SPEND]

# Read-only query of the persisted cost-usage store after the final line above:
catch_up_pending=0
processed_bytes=6887714668
total_bytes=6887714668
completed_files=10949
total_files=10949
last_scan_utc=2026-08-10 19:57:12

The two startup markers show relaunches reusing the existing multi-gigabyte cache: only 790,493 bytes and 404,384 bytes were consumed before successful Codex publication, rather than replaying the corpus. The later 4,600,693-byte delta shows appended history being incorporated. The persisted store then records equal processed/total bytes and completed/total files with no pending catch-up.

The two line-review findings were also fixed on top of merged upstream behavior:

• Completion now persists and validates the exact current scan inventory, so wider retained files cannot falsely satisfy completion; a regression removes a current file while retaining a complete wider file and verifies catch-up remains pending.
• Spend Dashboard Stop clears a queued restart before returning from an active pass; a regression covers that cancellation race.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31428524762
• Updated: 2026-08-10T20:20:36.555Z

[Quicksaver]

Resolved the updated ClawSweeper finding in a1aab85fa.

The primary stopCodexCostCatchUp() path now clears codexCostCatchUpRestartRequested before returning from an active pass, so deferred teardown cannot launch a replacement worker after Stop. I added the symmetric primary stopping an active pass clears a queued restart regression.

Validation after merging latest upstream/main:

• UsageStoreCodexCostCatchUpTests: 6 passed
• UsageStoreSpendDashboardCodexCostCatchUpTests: 6 passed
• CostUsageCatchUpCompletionTests: 2 passed
• ProviderArchitectureGatekeeperTests: 38 passed
• make check: passed with zero lint violations

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31433905955
• Updated: 2026-08-10T21:27:23.270Z

[Quicksaver]

Fixed the updated candidate-selection finding in 3f7c44d28.

The bounded path now stops after inspecting 512 unique candidate path keys, before cache-eligibility checks. Pending lookback paths are consumed directly, and the selector reuses the URL identity already established during discovery so cache accumulation remains stable across passes.

Concrete 1,500-file regression proof from CostUsagePerformanceGateTests:

[candidate-selection-proof] first=512, second=1024, overlap=512, pending=476, visits=512, attempts=512

This proves that the second pass inspected only 512 candidates (not the full retained queue), scanned 512 files, retained all 512 first-pass cache entries, and advanced the cache to 1,024 files with 476 remaining.

Validation:

• CostUsagePerformanceGateTests: 27/27 passed
• CostUsageCatchUpCompletionTests: 2/2 passed
• UsageStoreCodexCostCatchUpTests: 6/6 passed
• UsageStoreSpendDashboardCodexCostCatchUpTests: 6/6 passed
• make check: passed, zero SwiftLint violations

make test also reached 45 successful groups before the existing locale-sensitive MiniMaxMenuCardBillingTests failure in group 46 and its retry (1,234 expected versus this machine’s locale rendering 1 234); that suite does not touch these scanner changes.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31436203940
• Updated: 2026-08-10T21:57:50.219Z

[Quicksaver]

Fixed the persisted reconciliation finding in 295987c71.

Changes:

• Each cache load/save now validates at most 512 retained lookback paths
• The reconciliation path no longer builds a full-cache identity index first; it uses bounded direct/normalized path aliases
• Exact current-inventory validation is deferred until catch-up is pending but there is no active lookback, discovery work, or incomplete/buffered file work

Concrete 1,500-file regression output:

[reconcile-proof] save=512 load=512 pending=988
[candidate-selection-proof] first=512, second=1024, overlap=512, pending=476, visits=512, attempts=512

The first line proves that both save and the subsequent load stop after 512 metadata/completion visits even though the retained queue contains 988 paths. The second proves the scanner then remains bounded and advances another 512 files without losing the first-pass cache.

Validation:

• CostUsagePerformanceGateTests: 27/27 passed
• CostUsageCatchUpCompletionTests: 2/2 passed
• make check: passed, zero SwiftLint violations

The full local make test remains blocked only by the unrelated locale-sensitive MiniMaxMenuCardBillingTests expectation documented in the PR body; the branch CI macOS shards exercise the complete suite in their standard environment.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31438569828
• Updated: 2026-08-10T22:30:44.768Z

[Quicksaver]

Fixed the latest bounded-pass finding in 40ff7ee. Pending catch-up passes now maintain file progress incrementally and deliberately leave exact byte totals/inventory indeterminate, so they no longer sort or stat the full discovered corpus after the 512-file scan. Exact metadata accounting and sorted inventory persistence run only once bounded work is exhausted.

The counted 1,500-file regression proves two consecutive bounded passes advance 512 files each while performing zero whole-corpus progress-accounting visits: first=512, second=1024, overlap=512, pending=476, visits=512, attempts=512, accounting=0. CostUsagePerformanceGateTests (27), CostUsageCatchUpCompletionTests (2), both catch-up orchestration suites (6 + 6), and make check pass. The PR Evidence/Validation sections now include this proof.

@clawsweeper review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31439270017
• Updated: 2026-08-10T22:40:41.625Z

[steipete]

Maintainer migration follow-up is now pushed at exact head 11b9cd7d98477c2b0d30a73f6498d29ea3f30af1.

The active-lookback queue now owns cache-wide migration continuation without adding persisted schema. A completed 600-file cache with a drained queue is covered across pricing metadata, pricing-key, project-metadata, turn-ID, priority-metadata, and priority-turn migrations: pass one attempts exactly 512 files and persists 88; pass two attempts the remaining 88 and proves a path beyond the first slice was reparsed; bounded passes perform zero whole-inventory progress-accounting visits; exact completion reaches 600/600; the next warm refresh does no reseed or reparse. Separate coverage proves newest-first order both from a drained queue and when a migration interrupts a partially drained queue.

Validation at this head: migration suite 3 tests including 6 parameterized owner cases; bounded-progress 10/10; requested focused matrix 151 selections with the global-hook suite isolated; parser hash and make check clean; full make test 842/842 selections across 71/71 groups on the first attempt, with zero retries, failures, or timeouts.

The branch preserves the contributor head that advanced during validation, then applies the maintainer queue-only design afterward. It remains intentionally unmerged pending owner sign-off on persisted inventory/storage ownership.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31473602953
• Updated: 2026-08-11T08:32:01.937Z

[Quicksaver]

Addressed the latest finding from #2849 (comment) in 5cdc017.

Time-limited Codex catch-up now pages current-window date partitions, flat-root sessions, and the older active-session lookback through one shared 512-entry discovery budget before candidate construction. Per-root day/directory cursors persist in the catch-up state, cached-file reconstruction and recursive legacy discovery are deferred during bounded passes, and the exact full inventory runs only after bounded discovery and queued work are exhausted. Cache-wide migration queues retain ownership of cached paths without duplicating them into discovery.

Current-head oversized active-day proof:

[discovery-proof] first=512, discovery=512, attempts=512
[discovery-proof] relaunched=512, discovery=512, attempts=0
[discovery-proof] second=1024, discovery=512, visits=512, attempts=512, accounting=0

Validation:

• CostUsagePerformanceGateTests: 27 passed
• CostUsageBoundedProgressTests: 10 passed
• CostUsageCacheWideMigrationTests: 3 passed, including 6 migration-owner cases
• CostUsageCatchUpCompletionTests: 2 passed
• CostUsageStoreTests: 60 passed
• ProviderArchitectureGatekeeperTests: 38 passed
• make check: passed with zero lint violations
• Full make test: 45 groups passed; group 46 and its retry fail only the unrelated locale-sensitive MiniMax billing assertions (1,234 expected versus this Lisbon locale rendering 1 234). The other 121 tests in that exact shard pass.

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31483522636
• Updated: 2026-08-11T10:44:33.382Z

[Quicksaver]

Addressed the current durable ClawSweeper requirements at merged head e50a58a25 (6cf344300, 9279695ac; latest upstream/main e5528d452 merged).

• P1 — preserve v0.49.2 stores: added an exact compatible parser-hash/schema upgrade. The focused regression opens a completed v0.49.2 store, retains its files, installs the current schema/hash atomically, and records zero rebuilds.
• P2 — bound identity drift: cache load validates at most 512 drifted identities, persists the remainder in the active lookback queue, carries that queue across scan-window resets, retains missing/changed candidates for scanner handling, and drains only positively validated identity-drift entries. The 600-file regression validates 512 and defers 88.
• Current-head live proof: an isolated copy of the real >12,400-file, ~7.1 GiB history/store was relabeled with the exact v0.49.2 schema/hash and had only the copy's identity metadata changed. Real sessions and the active cache stayed read-only. Paths, identifiers, identities, spend, and content were not printed or retained.

[current-head-live-proof] mode=upgrade-drift-load elapsed_ms=9910 retained_files=10978 load_validation=512 pending=1
[current-head-live-proof] mode=relaunch-catch-up elapsed_ms=32170 retained_files=10978 attempts=512 discovery=512 accounting=0 pending=1 processed_bytes=0 total_bytes=0
[persisted-store] parser_hash=f1d04f67770b64cf schema=current retained_files=10978 pending_paths=9954

The temporary proof harness/database were deleted afterward. The earlier redacted real-history evidence in the PR body still demonstrates normal appended history being incorporated; this new run proves the final merged head's upgrade/relaunch bound.

Validation on the merged head:

• CostUsageCatchUpCompletionTests: 5/5
• CostUsageBoundedProgressTests: 10/10
• CostUsagePerformanceGateTests: 27/27
• CostUsageStoreTests: 61/61
• CostUsageStoreExecutorIsolationTests: 2/2
• make check: pass, current parser hash, formatting clean, 0 lint violations
• Full make test: 45 groups pass; group 46 and its retry fail only the pre-existing locale-sensitive MiniMaxMenuCardBillingTests assertions (1,234/5,678 expected, Lisbon locale renders nonbreaking-space grouping). The other 119 tests in that shard pass.

The prior CI failures were checked before this push: Linux could not resolve DIR, and SwiftLint rejected an over-length test file. Both are fixed in the pushed commits; the old macOS shards were already green.

[Quicksaver]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/31494213170
• Updated: 2026-08-11T13:03:39.747Z