Add a new adaptive circuit breaker that:

1. Works without need for configuration
2. Has the ability to open partially/fully depending on the severity of the incident

The circuit breaker has two main components:

Ideal Error Rate Estimator

This estimator tries to find out the expected error rate from a healthy dependency. It does that through a method of simple exponential smoothing, which basically relies on calculating a weighted average, with a few extra domain-specific hints:

1. Ignore any observations that are obviously too high
2. Converge slowly towards higher values, and quickly towards lower values
3. Be more receptive to signals for the first 30 minutes after boot (because we start with a random guess for the value), and less receptive afterwards

PID Controller (Process, Integral, Derivative) Controller

See wikipedia link

This controller increases/decreases the the rejection rate of the circuit breaker based on the value of:

kI * Integral + kP * P + kD * derivative

Where P is:

ErrorDelta - (1 - ErrorDelta) * RejectionRate

The intuition of the value of P:

• Increases when error rate increases
• Decreases when rejection rate increases
• The rejection rate has a (1 - ErrorDelta) multiplier, that we call the "defensiveness" multiplier. This allows rejection to increase more aggressively if the error rate is high

Note: The derivate component is currently set to 0. The Integral component mostly contributes as a history to prevent the circuit breaker from fluctuating too much.