fix(kosong): fail fast on quota-exhausted 429 instead of retrying

[vinlee19]

Related Issue

Part of #1860 — this PR fixes the engine-side fail-fast half. Companion UI PR: #1861 (live retry progress in the TUI), which closes the issue together with this one.

Problem

When an account's quota/balance is exhausted, Moonshot returns HTTP 429 with a structured body:

HTTP/2 429            (no retry-after header)
{"error":{"message":"Your account org-… <ak-…> is suspended due to insufficient balance,
  please recharge your account or check your plan and billing details",
  "type":"exceeded_current_quota_error"}}

normalizeAPIStatusError classifies every 429 as a retryable APIProviderRateLimitError — the structured error.type/error.code that the OpenAI SDK parses onto its APIError is dropped at the convertOpenAIError seam. A quota-exhausted 429 can never succeed on retry, yet it burns the full retry budget: 10 attempts with 0.5→32s exponential backoff, ~3 minutes per turn, with no feedback in the interactive TUI. Users read it as a frozen app.

Captured session log (kimi-code 0.26.0, provider type = "kimi" against api.moonshot.cn, one turn ≈ 189s):

11:51:30.355 INFO  llm request  turnStep=15.1
11:51:35.358 INFO  llm request  turnStep=15.1 attempt=2/10
...
11:54:34.426 INFO  llm request  turnStep=15.1 attempt=10/10
11:54:39.021 WARN  llm request failed  ... errorName=APIProviderRateLimitError statusCode=429
11:54:39.037 ERROR turn failed  turnId=15

The same account produced two different message wordings for the same error.type ("You exceeded your current token quota: … please check your account balance" earlier, the "suspended due to insufficient balance" variant later) — so message text alone is not a reliable discriminator, but the structured type is.

What changed

Layering (reworked per review — the error type is generic, the classification is vendor-owned):

• New APIProviderQuotaExhaustedError (extends APIStatusError; deliberately not a subclass of APIProviderRateLimitError), excluded from isRetryableGenerateError (fails fast even when a retry-after header is present) and from isProviderRateLimitError (no swarm requeue/suspend). This type and those consumers are the only quota knowledge left in the contract layer — normalizeAPIStatusError stays vendor-free.
• New convertError vendor hook: ProtocolTrait.convertError in agent-core-v2 (single-value, last-declarer-wins; bound by composeOpenAIChatHooks / composeAnthropicHooks / traitConvertError), and an equivalent optional hook parameter on convertOpenAIError / convertAnthropicError in kosong. Bases consult it — right after the abort guard — with the raw failure: the SDK error on HTTP paths, the raw event on the Responses in-stream error path (the streamed message threads the hook through).
• Moonshot's quota signals live Kimi-side in kimi-errors.ts (classifyKimiQuotaError): the structured exceeded_current_quota_error type/code plus the observed billing wordings as a fallback for gateways that flatten the body — deliberately no bare /quota/ or /balance/ patterns, so transient "token quota per minute" throttles keep retrying. Declared on kimiOpenAITrait and kimiAnthropicTrait (v2), and at the KimiChatProvider / KimiFiles catch sites (kosong, which has no trait seam).
• The OpenAI bases recognize only OpenAI's own documented insufficient_quota code (HTTP and Responses stream events) as protocol knowledge of that wire.
• toKimiErrorPayload (v1) and translateProviderError (v2) map the type to the existing provider.api_error code (retryable: false) instead of provider.rate_limit (retryable: true) — no new protocol code needed; the provider's actionable "please recharge" message is preserved on the payload. classifyApiError reports quota_exhausted in telemetry.
• Docs: the loop_control section (en/zh) now states retries only apply to transient failures.

Behavior notes for reviewers:

• Provider type = "kimi" (and openai with OpenAI's own quota code) fails fast exactly as before; quota-failed swarm subagents fail instead of suspending indefinitely as "Rate limited...".
• Quota errors cross the wire as provider.api_error (retryable: false) rather than provider.rate_limit.
• An unregistered vendor speaking Moonshot-style billing wordings through a plain openai transport is no longer classified by the shared base (vendor-neutral by design, per review) — it keeps the standard 429 retry ladder, and tests assert that.
• Transient rate-limit 429 behavior is unchanged. The captured quota body was verified end-to-end against a mock provider on the original classification seam (fails at attempt 1/10 with the provider's "please recharge" message; a rate_limit_reached_error body with retry-after: 1 still walks the full 10-attempt ladder honoring the header); the relocated classification is covered by unit/integration tests at every consult seam — trait declaration, hook composition, HTTP catch, Responses in-stream event, and the vendor-neutral base.

Checklist

• I have read the CONTRIBUTING document.
• I have linked a related issue, or explained the problem above.
• I have added tests that prove my feature works.
• Ran gen-changesets skill, or this PR needs no changeset. (changeset added: patch for @moonshot-ai/kimi-code and @moonshot-ai/kimi-code-sdk)
• Ran gen-docs skill, or this PR needs no doc update. (en/zh config-files.md updated)

[changeset-bot]

🦋 Changeset detected

Latest commit: 2aa7918

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@moonshot-ai/kimi-code	Patch
@moonshot-ai/kimi-code-sdk	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f585cbb4f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[vinlee19]

Pushed two follow-up commits after a local review pass:

• 5c60c91 — the openai_responses provider path was still burning the retry budget on quota exhaustion: Responses response.failed/error SSE events carry no HTTP status and were minted as a base ChatProviderError (whose unclassified fallback is retryable) or, with an embedded status_code=429, as a retryable rate limit. errorFromOpenAIResponsesEvent now routes the event code/message through the same quota-exhausted check before the rate-limit branch (kosong + agent-core-v2 mirror; all three entry paths share that converter). Tests added for both engines.
• 7ceabcc — dropped the inline comments my first commit added in agent-core-v2 sources, per its AGENTS.md header-only comment convention; the kosong twins keep the full rationale.

[vinlee19]

Hi @wbxl2000 , PTAL 🙏 — this one bit us for real: our quota ran out and every prompt turned into what looked like a ~3-minute freeze. I attached the Node inspector to the
stuck process and dumped JS stacks — the event loop was idle apart from the render tick, which led me to the silent 429 retry ladder. Analysis + captured 429 body:
#1860

Fixes: #1857 (fail fast on quota-exhausted 429) and #1861 (show retry progress in
the TUI). Feedback welcome!

[vinlee19]

Hi @RealKai42 @sailist — merged latest main and resolved the conflicts from the #1970 restructure: the v2 changes now live under src/kosong/ (contract/errors.ts, provider/bases/openai/*, protocol/errors.ts), and the quota tests from the deleted provider-errors.test.ts moved into test/kosong/provider/errors.test.ts. kosong and agent-core-v2 suites are green locally, and the agent-core tests covering this change pass as well. PTAL when you have a chance 🙏

[sailist]

Suggestion: move the quota-exhausted classification into the kimi trait via a convertError hook

Thanks for the thorough fix — the fail-fast semantics and the structured-code-first classification are clearly the right behavior. I'd like to suggest a different placement for the classification logic, though.

The knowledge this PR encodes — Moonshot's exceeded_current_quota_error type, the observed billing message wordings — is vendor-specific: it describes how the Kimi backend signals quota exhaustion. Right now that knowledge lands in the shared OpenAI base / contract layer, which means it runs for every OpenAI-compatible provider (custom base_url endpoints, unregistered vendors, gateways in front of other backends). A 429 carrying these signals means "quota exhausted, fail fast" on Kimi — the shared base shouldn't be the place that decides what a vendor's 429 means.

The split I'd propose follows the existing layering: the error type is generic, the error conversion is not.

• APIProviderQuotaExhaustedError itself, plus its consumers (isRetryableGenerateError, isProviderRateLimitError, the provider.api_error wire mapping) stay in the contract layer exactly as this PR has them — they're vendor-neutral and consumed via instanceof. That part is placed correctly.
• The classification — which raw backend error mints that type — is a vendor deviation, and vendor deviations are what traits are for. Concretely:
  1. Add a convertError?(error: unknown, ctx: TraitContext): ChatProviderError | undefined hook to ProtocolTrait (single-value, last-declarer-wins; undefined = keep the base classification). It must receive the raw SDK error, since the base conversion drops the parsed error.code / error.type.
  2. Bind it in composeOpenAIChatHooks and consult it at the catch sites in openai-legacy / openai-responses (including the Responses in-stream error-event path) before falling back to convertOpenAIError.
  3. Declare the Moonshot codes + billing message patterns on kimiOpenAITrait via that hook.
  4. The new options parameter on normalizeAPIStatusError can then be dropped — the base stays free of vendor knowledge.

Two places need explicit wiring so nothing regresses: the (kimi, anthropic) registration (the Anthropic base calls normalizeAPIStatusError directly, so it needs the same hook point) and kimi-files.ts, which calls convertOpenAIError without a hooks context today.

This keeps openai-common vendor-neutral and gives every other vendor the same extension point when they need to express their own error semantics.

[vinlee19]

@sailist — hardened the hook wiring per further review, in three follow-ups (db499a0, 79b840b, 2aa7918), PTAL when you have a chance:

• legacy Kimi-over-Anthropic now classifies quota too: AnthropicOptions.convertError threaded through the provider and stream wrapper, ProviderManager passes classifyKimiQuotaError on that route, and the classifier reads the Anthropic SDK's nested body shape;
• v2 telemetry reports quota_exhausted instead of rate_limit, matching the legacy engine;
• already-converted errors pass through before the hook, so it sees each raw failure exactly once (asserted in tests);
• regression tests now cover the real assembly paths: registry-composed (kimi, anthropic), ProviderManager routing, and KimiFiles uploads — each fails fast on a Moonshot quota 429 while a plain anthropic composition keeps the same 429 retryable.

kosong / agent-core-v2 suites green; PR description is up to date.

[pkg-pr-new]

pnpm dlx https://pkg.pr.new/@moonshot-ai/kimi-code@2aa7918

npx https://pkg.pr.new/@moonshot-ai/kimi-code@2aa7918

commit: 2aa7918

[sailist]

Thanks a lot for your work on this, @vinlee19! 🙏