|
| 1 | +--- |
| 2 | +title: 'Circuit Breaker' |
| 3 | +description: 'Configure per-strategy circuit protection and failure handling' |
| 4 | +--- |
| 5 | + |
| 6 | +<Info> |
| 7 | +This feature is available on all Portkey [plans](https://portkey.ai/pricing). |
| 8 | +</Info> |
| 9 | + |
| 10 | +## Circuit Breaker Config Schema |
| 11 | + |
| 12 | +Each `strategy` in a config may define a `cb_config` with the following fields: |
| 13 | + |
| 14 | +- **failure_threshold**: Number of failures after which the circuit opens. |
| 15 | +- **failure_threshold_percentage** *(optional)*: Percentage failure rate to trip the circuit. |
| 16 | +- **cooldown_interval**: Time (in milliseconds) to wait before allowing retries. A minimum of 30 seconds is enforced. |
| 17 | +- **failure_status_codes** *(optional)*: Specific HTTP status codes considered as failures. If not provided, all status codes >500 are considered as failures. |
| 18 | +- **minimum_requests** *(optional)*: Minimum number of requests required before failure rate is evaluated. |
| 19 | + |
| 20 | +<Info> |
| 21 | +- Strategies can inherit `cb_config` from a parent if not set at their level. |
| 22 | +- Targets inherit `cb_config` from their parent strategy. |
| 23 | +</Info> |
| 24 | + |
| 25 | +<Warning> |
| 26 | +Strategies using the `conditional` mode are **not considered** in circuit breaker logic. |
| 27 | +</Warning> |
| 28 | + |
| 29 | +## Example Config |
| 30 | + |
| 31 | +```json |
| 32 | +{ |
| 33 | + "retry": { |
| 34 | + "attempts": 3 |
| 35 | + }, |
| 36 | + "strategy": { |
| 37 | + "mode": "fallback", |
| 38 | + "cb_config": { |
| 39 | + "failure_threshold_percentage": 20, |
| 40 | + "minimum_requests": 10, |
| 41 | + "cooldown_interval": 60000, |
| 42 | + "failure_status_codes": [ |
| 43 | + 401, |
| 44 | + 429, |
| 45 | + 500, |
| 46 | + ] |
| 47 | + } |
| 48 | + }, |
| 49 | + "targets": [ |
| 50 | + { |
| 51 | + "virtual_key": "virtual-key-1" |
| 52 | + }, |
| 53 | + { |
| 54 | + "virtual_key": "virtual-key-2" |
| 55 | + } |
| 56 | + ] |
| 57 | +} |
| 58 | +``` |
| 59 | + |
| 60 | +## Circuit State Evaluation |
| 61 | + |
| 62 | +Circuit breaker logic is evaluated per strategy path. It tracks: |
| 63 | + |
| 64 | +- Number of failures and successes. |
| 65 | +- The time of the first failure. |
| 66 | +- Whether the total requests meet the `minimum_requests` threshold and calculates failure rate as percentage |
| 67 | + |
| 68 | +The circuit is **OPEN**(unhealthy) if: |
| 69 | +- The failure count exceeds the `failure_threshold`, or |
| 70 | +- The failure rate exceeds `failure_threshold_percentage`. |
| 71 | + |
| 72 | +Once **OPEN**, requests to the affected targets are blocked until the `cooldown_interval` has passed. |
| 73 | + |
| 74 | +When the cooldown period ends, the circuit is **CLOSED** automatically, and failure counters are reset. |
| 75 | + |
| 76 | +## Runtime Behavior |
| 77 | + |
| 78 | +- At each strategy level, the circuit breaker evaluates the status of all sibling targets. |
| 79 | +- Unhealthy targets (circuit OPEN) are removed from the target list before strategy execution. |
| 80 | +- If no healthy targets remain, circuit breaker is bypassed and all targets are used for routing. |
| 81 | + |
| 82 | +```mermaid |
| 83 | +flowchart TD |
| 84 | + A[User sends request] --> B[Gateway selects a strategy] |
| 85 | + B --> C[Strategy has multiple targets] |
| 86 | + C --> D{All targets OPEN?} |
| 87 | + D -->|Yes| E[Ignore circuit breakers] |
| 88 | + E --> F[Use all targets for routing] |
| 89 | + D -->|No| G["Use only CLOSED (healthy) targets"] |
| 90 | + F --> H[Route to best target] |
| 91 | + G --> H |
| 92 | + H --> I[Process target] |
| 93 | +``` |
0 commit comments