Skip to content

Extend circuit breaker functionality to support per-model isolation#111

Merged
kacpersaw merged 6 commits into
mainfrom
kacpersaw/aibridge-circuit-breaker-per-model
Jan 16, 2026
Merged

Extend circuit breaker functionality to support per-model isolation#111
kacpersaw merged 6 commits into
mainfrom
kacpersaw/aibridge-circuit-breaker-per-model

Conversation

@kacpersaw
Copy link
Copy Markdown
Contributor

@kacpersaw kacpersaw commented Jan 13, 2026

Summary

Extends the circuit breaker implementation to isolate failures at the model level, not just per-endpoint. This prevents a failing model (e.g., claude-sonnet-4-20250514 being rate-limited) from blocking requests to other models on the same endpoint.

Problem

With endpoint-only circuit breakers, if one model starts failing (e.g., due to rate limiting), all requests to that endpoint get blocked - even for healthy models.

Solution

Circuit breakers are now keyed by endpoint:model tuple instead of just endpoint. Each model gets independent failure tracking and circuit state.

Changes

  • Circuit breaker key: Changed from endpoint to endpoint:model
  • Metrics labels: Added model label to all circuit breaker metrics
  • Architecture: Replaced middleware pattern with Execute() method to access model after request parsing
  • Tests: Added per-model isolation tests

Example

/v1/messages:claude-sonnet-4-20250514  → OPEN (rate limited)
/v1/messages:claude-3-5-haiku-20241022 → CLOSED (healthy)

Requests to haiku continue working even when sonnet is blocked.

Files Changed

File Change
circuitbreaker/circuitbreaker.go Get(endpoint, model), Execute() method
bridge.go Pass model to circuit breaker after interceptor creation
metrics/metrics.go Added model label to CB metrics
circuitbreaker/circuitbreaker_test.go Per-model isolation tests
circuit_breaker_integration_test.go Integration test for model isolation

@kacpersaw kacpersaw force-pushed the kacpersaw/aibridge-circuit-breaker-per-model branch from 7ad6549 to 45a2c45 Compare January 14, 2026 14:29
@kacpersaw kacpersaw marked this pull request as ready for review January 14, 2026 15:06
@kacpersaw kacpersaw requested a review from pawbana January 14, 2026 15:08
Comment thread circuitbreaker/circuitbreaker.go Outdated
Comment thread bridge.go Outdated
Comment thread circuitbreaker/circuitbreaker.go Outdated
Comment thread circuit_breaker_integration_test.go
Comment thread circuit_breaker_integration_test.go Outdated
@kacpersaw kacpersaw force-pushed the kacpersaw/aibridge-circuit-breaker-per-model branch 2 times, most recently from 30f30ec to 5cc2635 Compare January 15, 2026 14:55
Comment thread circuitbreaker/circuitbreaker.go Outdated
Comment thread bridge.go Outdated
@kacpersaw kacpersaw force-pushed the kacpersaw/aibridge-circuit-breaker-per-model branch from fec402b to 0b0a29e Compare January 15, 2026 15:29
Comment thread bridge.go Outdated
Comment thread circuitbreaker/circuitbreaker.go Outdated
@kacpersaw kacpersaw changed the base branch from kacpersaw/aibridge-circuit-breaker to main January 16, 2026 13:44
@kacpersaw kacpersaw force-pushed the kacpersaw/aibridge-circuit-breaker-per-model branch from 44bb888 to dd13bd5 Compare January 16, 2026 13:47
@kacpersaw kacpersaw merged commit 5269560 into main Jan 16, 2026
1 check passed
@kacpersaw kacpersaw deleted the kacpersaw/aibridge-circuit-breaker-per-model branch January 16, 2026 13:55
SasSwart added a commit to coder/coder that referenced this pull request Jan 20, 2026
Update our dependency on coder/aibridge. This allows us to benefit from
the following additions to bridge:

feat: inner agentic loop for openai responses requests (blocking only)
(coder/aibridge#127)
feat: req/resp logging middleware (coder/aibridge#105)
perf: eliminate unnecessary json marshalling for anthropic requests to
bridge (coder/aibridge#102)
feat: add token usage recording for responses streaming interceptor
(coder/aibridge#125)
feat: add token usage recording for responses blocking interceptor
(coder/aibridge#124)
feat: add tool usage recording to streaming responses interceptor
(coder/aibridge#123)
feat: add tool usage recording for blocking responses interceptor
(coder/aibridge#122)
Extend circuit breaker functionality to support per-model isolation
(coder/aibridge#111)
feat: add circuit breaker for upstream provider overload protection
(coder/aibridge#75)
chore: change blocking request timeouts to 10m (coder/aibridge#118)
feat: add prompt recording for responses API (coder/aibridge#109)
feat: add basic responses API interceptor  (coder/aibridge#107)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants