Microservices · Lesson 9

Circuit Breakers and Resilience

Handling Partial Failure

In a distributed system, a dependency can be slow or unavailable while the caller is healthy. A circuit breaker stops repeatedly sending requests to a failing dependency and gives it time to recover. Resilience also includes timeouts, bounded retries, fallbacks, and bulkheads.

Circuit States

The breaker tracks failures and moves between closed, open, and half-open states. The example captures the decision without connecting to a real network.

public class CircuitBreaker {
    private final int failureLimit;
    private int failures;
    private boolean open;

    public CircuitBreaker(int failureLimit) {
        this.failureLimit = failureLimit;
    }

    public void recordFailure() {
        failures++;
        if (failures >= failureLimit) open = true;
    }

    public boolean allowRequest() {
        return !open;
    }

    public static void main(String[] arguments) {
        CircuitBreaker breaker = new CircuitBreaker(2);
        breaker.recordFailure();
        breaker.recordFailure();
        System.out.println(breaker.allowRequest());
    }
}

Resilience Tools

Common Mistake and Practices

Common Mistake: Retrying every failure immediately and without a limit. A retry storm can overload a dependency that is already struggling.

• Set timeouts before retries.
• Use exponential backoff and jitter.
• Retry only idempotent or safely repeatable operations.
• Monitor breaker state transitions.
• Make fallback behavior explicit to users.

Real-World Use Case

When the quiz service cannot reach an analytics service, it records the event for later processing instead of blocking quiz delivery. A circuit breaker prevents analytics failure from taking down the learner workflow.