fix(proxy): consume UsageBypass on OpenAI chat completions

[rohith500]

Summary

runTurnLoop already sets UsageBypass=true for both wire formats when the per-installation gate engages, but only ProxyMessages consumed it via bypassToAnthropic (strict subscription pass-through, skip ledger, and on a retryable 429 fall through to routeFor). ProxyOpenAIChatCompletion ignored routeRes.UsageBypass and continued normal dispatch.

• Happy path: often looked fine by coincidence (same turn-loop decision + OAuth).
• 429 fallthrough: Messages recovered via scorer reroute; OpenAI returned a raw 429 to the client.

Confirmed unintentional incomplete port in #775 (same class as #771). No overlap with #762 (preemptive exhaustion suppress) or #773 (post-dispatch baseline / subscription-credit retry).

Expected vs actual (pre-fix → post-fix)

Happy path (gate on, util 0.20, requested claude-sonnet-4-6)

	Messages	OpenAI pre-fix	OpenAI post-fix
Client	200	200 (coincidence)	200 via bypassAnthropicOpenAI
x-router-decision	usage_bypass	usage_bypass	usage_bypass
Scorer	0	0	0
Path	bypassToAnthropic	normal OpenAI dispatch	bypassAnthropicOpenAI (skip billing)
Response shape	Anthropic Messages	chat.completion	chat.completion

Weekly-limit 429 on the bypass attempt

	Messages	OpenAI pre-fix (#775)	OpenAI post-fix
Client	200	raw 429	200
Decision after	cluster:v0.2	stuck usage_bypass	cluster:v0.2
Model after	scorer pick (claude-haiku-4-5)	requested	scorer pick
Scorer calls	1	0	1
Dispatches	2 (bypass 429 → routed OK)	3× OAuth 429	2 (bypass 429 → routed OK)

Subscription-only + bypass 429

	Messages	OpenAI pre-fix	OpenAI post-fix
Result	ErrCreditsExhaustedSubscriptionUnavailable	raw 429 (no refuse)	ErrCreditsExhaustedSubscriptionUnavailable
Scorer	0	0	0
Dispatches	1 (bypass only)	3× OAuth	1

Fix

1. bypassAnthropicOpenAI (usage_bypass.go) — OpenAI-wire counterpart to bypassToAnthropic:

   • PrepareAnthropic + NewSSETranslator → OpenAI client response
   • resolve subscription credentials; skip ledger (early return)
   • errBypassRetryable on providers.IsRetryable pre-commit
   • non-retryable buffered upstream → flushBufferedIfPresent
   • emit router.usage_bypass span (same attrs as Messages)

2. ProxyOpenAIChatCompletion after runTurnLoop — mirror Messages (service.go ~2168):

   if routeRes.UsageBypass && routeRes.Decision.Provider == providers.ProviderAnthropic {
       err := s.bypassAnthropicOpenAI(...)
       if !errors.Is(err, errBypassRetryable) { return err }
       if billing.SubscriptionOnlyFromContext(ctx) {
           return ErrCreditsExhaustedSubscriptionUnavailable
       }
       // clear UsageBypass, refresh subsidy, routeFor, continue
   }

3. turnLoopResult.UsageBypass godoc — names both surfaces.

Design: second helper (not a shared Messages/OpenAI abstraction) because emit/flush differ (flushUpstreamErrorAsAnthropic vs flushBufferedIfPresent + SSE translator). Eligibility / fallthrough control flow matches Messages exactly. ProxyMessages untouched (diff only in OpenAI entry + new helper).

What changed

File	Change
internal/proxy/usage_bypass.go	Add bypassAnthropicOpenAI
internal/proxy/service.go	Consume UsageBypass in ProxyOpenAIChatCompletion
internal/proxy/turnloop.go	Godoc: both surfaces
internal/proxy/openai_usage_bypass_test.go	Happy / 429 fallthrough / subscription-only refuse

Live verification (committed tip 7994e67)

Re-ran the same bypassFixture shapes used in #775 against the committed branch (throwaway probe; not committed):

POST-FIX 429 Messages: status=200 decision="cluster:v0.2" model="claude-haiku-4-5" scorer=1 dispatches=2
POST-FIX 429 OpenAI:   status=200 decision="cluster:v0.2" model="claude-haiku-4-5" scorer=1 dispatches=2
POST-FIX SUB-ONLY 429: err=credits exhausted… scorer=0 calls=1

Pre-fix OpenAI 429 (from #775): status=429, scorer=0, anthCalls=3, decision stuck on usage_bypass.

Billing

Successful OpenAI bypass now skips the ledger (early return), matching Messages bypassToAnthropic — not the prior coincidence path’s emitBilling delta=0 notional row.

Not in scope / not overlapping

• ProxyOpenAIChatCompletion never suppresses an exhausted Claude subscription — spent OAuth token dispatched, no failover to deployment key #761 / fix(proxy): suppress exhausted Claude subscription on OpenAI path #762 — preemptive exhaustion suppress before first inject
• ProxyOpenAIChatCompletion has no post-dispatch failover: OSS-outage baseline retry and subscription-credit retry both absent #771 / fix(proxy): port post-dispatch failover to OpenAI chat completions #773 — post-dispatch baseline + subscription-credit retry (different region; Weave-key retry ≠ UsageBypass routeFor)
• proxy: usage-bypass turns skip fireTelemetry and invisible to dashboard metrics and cost charts #516 / fix(proxy): fire telemetry row for usage-bypass turns #729 — telemetry visibility for bypass turns

Test plan

• TestProxyOpenAI_UsageBypass_WeeklyLimit_FallsBackToRoutedDispatch + subscription-only refuse fail on main, pass here
• Happy-path OpenAI UsageBypass regression kept
• Full TestUsageBypass_* / TestBypass_* / Messages weekly-limit suite green
• make check green
• go test ./internal/proxy/... -race clean except known TestFireTelemetryRecoversFromPanic flake
• ProxyMessages region unchanged in diff
• New comments ≤2 consecutive lines (comment-length review)

Fixes #775

Made with Cursor

[greptile-apps]

PR author is not in the allowed authors list.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 7994e67. Configure here.}

[rohith500]

Addressed Bugbot: bypassAnthropicOpenAI now wraps the client writer in preludeBuffer, runs subscription-only OpenAIRoutingMarkerWriter.Prelude into that buffer, Seals before upstream, and Discards on errBypassRetryable so a streaming 429→402 refuse never leaves a committed SSE preamble.

Added TestSubscriptionOnly_OpenAI_BypassRetryable_Stream_NoPartialCommit.

[devin-ai-integration]

Thanks @rohith500 — this one follows the repo conventions nicely and needs no changes. 👍 The UsageBypass consumer sits on proxy.Service (right layer), reuses the ErrCreditsExhaustedSubscriptionUnavailable sentinel and errBypassRetryable, uses providers.Provider* constants and slog throughout, and the tests are non-tautological (scorer-skip, 429 reroute, subscription-only refusal, and the buffered-prelude no-partial-commit stream case are all real behavior guards). The prelude-buffer why-comment is exactly the kind AGENTS.md wants. Nice work!