[bug] 503 Service Unavailable

[haf]

Summary

❯ codex-multi-auth doctor
Doctor diagnostics
Storage: /Users/h/.codex/openai-codex-accounts.json
Summary: 17 ok, 0 warnings, 0 errors

✓ storage-file: Account storage file found
  /Users/h/.codex/openai-codex-accounts.json
✓ storage-readable: Storage file is readable
  5197 bytes
✓ codex-auth-file: Codex auth file found
  /Users/h/.codex/auth.json
✓ codex-auth-readable: Codex auth file is readable
  email=, accountId=org-***Uv8
✓ codex-config-file: Codex config file found
  /Users/h/.codex/config.toml
✓ codex-auth-store: Codex auth storage is set to file
  mode=file
✓ codex-cli-state: Codex CLI state loaded
  /Users/h/.codex/auth.json
✓ accounts: Loaded 2 account(s)
✓ active-index: Active index is valid (1)
✓ enabled-accounts: 2 enabled / 0 disabled
✓ duplicate-refresh-token: No duplicate refresh tokens detected
✓ duplicate-email: No duplicate emails detected
✓ placeholder-email: No placeholder emails detected
✓ refresh-token-shape: Refresh token format looks normal
✓ recommended-switch: Current account aligns with forecast recommendation
✓ forecast-runtime-alignment: Forecast and runtime availability are aligned
✓ active-selection-sync: Manager active account and Codex active account are aligned
  manager=email=, accountId=org-*** | codex=email=, accountId=org-***

Reproduction

1. Launch Codex (app)
2. Write 'Hello world'

Expected Behavior

For the proxy to work

Actual Behavior

Getting 503 Service Unavailable

Environment

• codex --version: codex-cli 0.134.0 and Version 26.519.81530 (3178)
• codex auth status output summary: invalid command (does not exist) — without, starts:

  └ Unexpected status 503 Service Unavailable: Pinned account 1 is currently unavailable; run `codex-multi-auth
    status` for details, or `codex-multi-auth unpin` to allow rotation., url: http://127.0.0.1:63884/responses

• codex auth rotation status output summary, if runtime rotation is involved:
• npm ls -g codex-multi-auth:
• OS: macOS
• Node.js: v24.13.1

Diagnostic Outputs

Include relevant outputs from:

• codex auth check
• codex auth report --json
• codex auth doctor --json

Logs (Optional)

If needed, include sanitized logs from ~/.codex/multi-auth/logs/codex-plugin/: ls: /Users/h/.codex/multi-auth: No such file or directory
Only enable logging temporarily: ENABLE_PLUGIN_REQUEST_LOGGING=1.

Compliance Confirmation

• I am using this project for personal development workflows.
• This report does not request policy bypasses or prohibited usage.
• I removed any secrets/tokens from this report.

[coderabbitai]

Coding Plan

Summary

• the fix adds the specific skip reason to the pinned account 503 response, matching the existing writePoolExhausted pattern that already includes account_skip_reasons
• both the human-readable message and a structured reason field are updated for maximum diagnostic value
• regression tests ensure each skip reason type is properly surfaced when a pinned account is unavailable

Design Choices

Design Choice 1: should the skip reason be included in the message string or as a separate field?

Options Considered:

1. add reason field to response body only (machine-readable, matches writePoolExhausted pattern)
2. include reason in both the message string and as a separate field (human-readable + machine-readable)
3. only modify the message string (human-readable but breaks api consistency)

Chosen Option: 2

Rationale: option 2: include in both message and as reason field. users reading the error in logs get immediate context, and programmatic consumers get structured data. matches existing patterns while improving ux.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

include skip reason in pinned account 503 response

this phase modifies the pinned account 503 response to include the specific runtime skip reason, helping users diagnose why their pinned account is unavailable.

Task 1: add skip reason to 503 response body

include the skip reason in the pinned account 503 response at lib/runtime-rotation-proxy.ts:~1742-1755.

• extract skip reason from accountSkipReasons.get(pinnedIndex) before writing response
• add reason field to the error object (e.g., "rate-limited", "cooling-down:auth-failure", "circuit-open")
• update message string to include the reason: "Pinned account ${n} is currently unavailable (${reason}); run..."
• fallback to "unknown" if no skip reason captured (defensive, shouldn't happen)

Task 2: add regression tests for skip reason inclusion

add test coverage in test/issue-474-pin-end-to-end.test.ts or a new test file to verify skip reasons are surfaced.

• add test case: pinned account rate-limited → 503 includes reason: "rate-limited"
• add test case: pinned account cooling-down → 503 includes reason: "cooling-down:auth-failure" (or similar)
• add test case: pinned account circuit-open → 503 includes reason: "circuit-open"
• verify message string contains the reason substring
• note: existing test at test/issue-474-pin-end-to-end.test.ts:254-283 tests disabled account; extend coverage for other skip reasons

Task 3: update unit tests for chooseAccount skip reason tracking

ensure test/issue-474-pin-honored.test.ts tests verify skipReasons map is populated correctly.

• verify existing tests at test/issue-474-pin-honored.test.ts:294-462 check skipReasons output
• if not covered, add assertions that skipReasons.get(pinnedIndex) returns expected string for each unavailability case
• covers: "rate-limited", "cooling-down", "disabled", "policy-blocked", "missing", "circuit-open"

🤖 Prompt for AI agents

modify the pinned account 503 response in
`lib/runtime-rotation-proxy.ts:~1742-1755` to surface the skip reason, and
add/extend tests to verify each skip reason type is correctly returned.

**`lib/runtime-rotation-proxy.ts:~1742-1755` — response body change:**
- before writing the 503 response for a failed pinned account, extract the skip
reason via `accountSkipReasons.get(pinnedIndex)`
- assign to a local `reason` variable; fallback to `"unknown"` if the map
returns nothing (defensive — shouldn't happen in practice, but flag it with a
comment)
- add a `reason` field to the error response object (e.g., `reason:
"rate-limited"`, `reason: "circuit-open"`)
- update the human-readable `message` string to: `"Pinned account ${n} is
currently unavailable (${reason}); run..."`
- the shape should match the existing `writePoolExhausted` pattern that already
includes `account_skip_reasons`

**`test/issue-474-pin-end-to-end.test.ts` — regression tests:**
- the existing test at `test/issue-474-pin-end-to-end.test.ts:254-283` covers
the disabled-account case — extend, don't replace
- add three new test cases covering: `reason: "rate-limited"`, `reason:
"cooling-down:auth-failure"` (or similar variant), and `reason: "circuit-open"`
- each test must assert both the structured `reason` field in the response body
and that the message string contains the reason as a substring
- flag: missing regression tests for concurrent pin + skip-reason-change races
should be called out as a known gap in a code comment

**`test/issue-474-pin-honored.test.ts:294-462` — unit tests:**
- check whether existing tests assert on `skipReasons.get(pinnedIndex)` output —
if not, add assertions
- cover all known skip reason strings: `"rate-limited"`, `"cooling-down"`,
`"disabled"`, `"policy-blocked"`, `"missing"`, `"circuit-open"`
- flag: windows edge cases (e.g., path separators in error messages) are not
expected here but note if any string interpolation could be
environment-sensitive

Research

the runtime rotation proxy in lib/runtime-rotation-proxy.ts handles request routing. when a manual pin is active and the pinned account fails runtime availability checks, it returns 503 with code codex_pinned_account_unavailable. the skip reason is tracked in accountSkipReasons map but never included in the response. chooseAccount() calls getAccountRuntimeSkipReason() which returns strings like "rate-limited", "cooling-down:auth-failure", "circuit-open", "workspace-disabled", etc. the non-pinned path (writePoolExhausted) already includes account_skip_reasons in its 503 response.

---

🚀 Next Steps

🤖 All AI agent prompts combined

modify the pinned account 503 response in
`lib/runtime-rotation-proxy.ts:~1742-1755` to surface the skip reason, and
add/extend tests to verify each skip reason type is correctly returned.

**`lib/runtime-rotation-proxy.ts:~1742-1755` — response body change:**
- before writing the 503 response for a failed pinned account, extract the skip
reason via `accountSkipReasons.get(pinnedIndex)`
- assign to a local `reason` variable; fallback to `"unknown"` if the map
returns nothing (defensive — shouldn't happen in practice, but flag it with a
comment)
- add a `reason` field to the error response object (e.g., `reason:
"rate-limited"`, `reason: "circuit-open"`)
- update the human-readable `message` string to: `"Pinned account ${n} is
currently unavailable (${reason}); run..."`
- the shape should match the existing `writePoolExhausted` pattern that already
includes `account_skip_reasons`

**`test/issue-474-pin-end-to-end.test.ts` — regression tests:**
- the existing test at `test/issue-474-pin-end-to-end.test.ts:254-283` covers
the disabled-account case — extend, don't replace
- add three new test cases covering: `reason: "rate-limited"`, `reason:
"cooling-down:auth-failure"` (or similar variant), and `reason: "circuit-open"`
- each test must assert both the structured `reason` field in the response body
and that the message string contains the reason as a substring
- flag: missing regression tests for concurrent pin + skip-reason-change races
should be called out as a known gap in a code comment

**`test/issue-474-pin-honored.test.ts:294-462` — unit tests:**
- check whether existing tests assert on `skipReasons.get(pinnedIndex)` output —
if not, add assertions
- cover all known skip reason strings: `"rate-limited"`, `"cooling-down"`,
`"disabled"`, `"policy-blocked"`, `"missing"`, `"circuit-open"`
- flag: windows edge cases (e.g., path separators in error messages) are not
expected here but note if any string interpolation could be
environment-sensitive

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[haf]

codex-multi-auth uninstall, then npm i -g codex-multi-auth then double-clicking ~/Applications/Codex Multi Auth.app then starting /Applications/Codex.app again cleared the error without other changes or extra log-outs, log-ins.

[coderabbitai]

🔗 Related PRs

#134 - add proxy-compatible upstream transport [merged]
#458 - Fix npm bin collision with official Codex [merged]
#460 - Fix app bind history repair [merged]
#462 - Fix Codex TUI goal proxy handling [merged]
#480 - Fix runtime rotation pool diagnostics [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[haf]

OK, then it went down again. 503 unavailable.

[haf]

It also disabled plugins

So a lot of the automations don't work

[haf]

Also, all history per-project disappears, unless they're tagged

[ndycode]

Hey @haf — pushed a beta build with extra diagnostics so we can tell exactly why the pin is failing.

npm i -g codex-multi-auth@beta

That gives you 2.1.13-beta.0. Your latest is untouched, only this opt-in pulls the new build.

When you hit the 503 again, the body now includes a reason field (rate-limited, cooling-down:auth-failure, circuit-open, disabled, workspace-disabled, policy-blocked, missing — or null if it's the state desync we suspect). Paste that body here.

Also helpful if you can run with logging on for one repro:

ENABLE_PLUGIN_REQUEST_LOGGING=1 codex

Then attach the latest file from ~/.codex/multi-auth/logs/codex-plugin/.

Stable v2.1.13 lands once we see those logs and patch the root cause.

(Plugins-disabled and history-loss you mentioned are now tracked in #488 and #489 so they don't get lost.)