Skip to content

fix(proxy): port post-dispatch failover to OpenAI chat completions#773

Open
rohith500 wants to merge 5 commits into
workweave:mainfrom
rohith500:fix/openai-post-dispatch-failover
Open

fix(proxy): port post-dispatch failover to OpenAI chat completions#773
rohith500 wants to merge 5 commits into
workweave:mainfrom
rohith500:fix/openai-post-dispatch-failover

Conversation

@rohith500

Copy link
Copy Markdown
Contributor

Summary

ProxyOpenAIChatCompletion had no post-dispatch failover: neither baseline
retry (OSS outage → requested Anthropic model) nor subscription-credit
retry (OAuth 429/timeout or 401/403 → Weave/BYOK key). Both already work in
ProxyMessages. Live pre-fix: identical OSS-outage fixture returned raw
503 with anthropicCount=0 on OpenAI while Messages recovered via Anthropic.

Full root-cause and evidence: #771

Fix

Port both mechanisms into ProxyOpenAIChatCompletion, same eligibility and
pre-commit gates as Messages. Design choice: second full port (not a shared
helper) because emit/flush paths differ (OpenAI uses PrepareAnthropic +
SSE translator; Messages uses Anthropic-native SSE). Local
openaiAnthropicAttempt is reused by both retry sites on the OpenAI path
only — ProxyMessages is untouched.

  1. Baseline failover when cost-routed OSS bindings fail retryably /
    model-not-found and the caller requested a known Anthropic catalog model.
  2. Subscription-credit retry when an Anthropic+subscription turn fails
    with IsRetryable or anthropicOAuthCredentialRejected (401/
    authentication_error, 403/permission_error).

Out of scope (called out in #771)

  • UsageBypass on the OpenAI path — still absent; separate issue. This PR
    only covers the reactive subscription-retry half of that composite case.
  • failover_used / fallback_attempts telemetry accuracy when a cross-
    binding retry ran but the request still failed — same Messages semantics;
    left alone to avoid OpenAI-only asymmetry.

Tests

  • TestProxyOpenAI_OSSOutageFailsOverToBaselineAnthropic (+ negatives:
    forced model, requested OSS, Anthropic excluded)
  • TestProxyOpenAI_SubscriptionRetry_Live429FailsOverToDeployKey
  • TestProxyOpenAI_SubscriptionRetry_OAuth401FailsOverToDeployKey
  • TestProxyOpenAI_SubscriptionRetry_OAuth403FailsOverToDeployKey

Live verification (committed tip)

Check Pre-fix (#771) Post-fix
OSS outage baseline fw=1, or=1, anth=0 → 503 fw=1, or=1, anth=1 model=claude-opus-4-8 → 200
Sub 429 dispatch[0..2] OAuth → client 429 dispatch[0..2] OAuth → dispatch[3] deploy key → 200
Sub 401/403 1× OAuth → client raw OAuth reject → deploy key → 200

Test plan

  • New tests fail on main, pass on this branch
  • make check green
  • go test ./internal/proxy/... -race clean except known TestFireTelemetryRecoversFromPanic flake
  • ProxyMessages region unchanged (diff only in ProxyOpenAIChatCompletion)
  • New comments kept ≤2 consecutive lines (comment-length review)

Fixes #771

Made with Cursor

rohith500 and others added 5 commits July 17, 2026 13:05
Lock in ProxyOpenAIChatCompletion parity with Messages for OSS-outage
baseline failover (happy path + negative cases) before the port.

Signed-off-by: N Rohith Reddy <rohithreddy2202@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
When a cost-routed OSS binding fails on the OpenAI wire, retry the
caller-requested Anthropic model pre-commit — same eligibility and
commit gates as ProxyMessages.

Signed-off-by: N Rohith Reddy <rohithreddy2202@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Cover live 429 and OAuth 401/403 arms on ProxyOpenAIChatCompletion before
porting the Messages subscriptionRetryEligible path.

Signed-off-by: N Rohith Reddy <rohithreddy2202@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Retry Anthropic-routed OpenAI turns on the Weave/BYOK key after
subscription 429 or OAuth 401/403, matching ProxyMessages.

Signed-off-by: N Rohith Reddy <rohithreddy2202@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Keep new comment blocks under the comment-length review threshold.

Signed-off-by: N Rohith Reddy <rohithreddy2202@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@greptile-apps

greptile-apps Bot commented Jul 17, 2026

Copy link
Copy Markdown

PR author is not in the allowed authors list.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Thanks @rohith500 — conforms to the repo conventions, no rewrite needed. 👍 The baseline and subscription failover ports mirror the ProxyMessages predicates on proxy.Service, reuse the existing dispatchWithFallback/failoverInputs machinery with a shared openaiAnthropicAttempt closure, keep pre-commit prelude semantics intact, and use sentinel errors + slog correctly. The tests are non-tautological end-to-end guards (real httptest upstreams for the OSS-outage baseline path, forced-model non-substitution, exclusion, and OAuth 429/401/403 deploy-key retries). Great work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ProxyOpenAIChatCompletion has no post-dispatch failover: OSS-outage baseline retry and subscription-credit retry both absent

1 participant