Retry With Exponential Backoff and Jitter in Python

TL;DR: Retry transient failures only, cap the attempts at three, draw each delay from random.uniform(0, min(cap, base * 2 ** attempt)), and honour Retry-After when the provider sends one. Bound the whole sequence with a deadline so a slow provider cannot stretch one record across a minute. The wider stack is described in resilience patterns for geocoding APIs.

The Loop

from __future__ import annotations

import asyncio
import random
import time
from typing import Awaitable, Callable, TypeVar

T = TypeVar("T")


class Transient(Exception):
    """Raised by the caller for failures a second attempt might fix."""


async def with_retry(
    call: Callable[[], Awaitable[T]],
    *,
    attempts: int = 3,
    base_s: float = 0.5,
    cap_s: float = 8.0,
    deadline_s: float = 20.0,
) -> T:
    """Run `call`, retrying transient failures with full jitter and a deadline."""
    started = time.monotonic()
    last: Exception | None = None

    for attempt in range(attempts):
        try:
            return await call()
        except Transient as exc:
            last = exc
            if attempt == attempts - 1:
                break

            ceiling = min(cap_s, base_s * (2 ** attempt))
            delay = random.uniform(0.0, ceiling)          # full jitter

            hint = getattr(exc, "retry_after_s", None)    # provider's own advice
            if hint is not None:
                delay = max(delay, float(hint))

            remaining = deadline_s - (time.monotonic() - started)
            if delay >= remaining:
                break                                     # no time left to try again
            await asyncio.sleep(delay)

    raise last if last is not None else RuntimeError("retry loop exited unexpectedly")

Two details make this different from the usual version. The delay is drawn from the whole interval rather than added to it, and the deadline check happens before the sleep — there is no point waiting four seconds when only two remain in the budget.

Why Full Jitter Rather Than Equal Jitter

Three jitter strategies appear in the wild, and they differ in how tightly retries cluster. Under concurrency, the clustering is the whole problem: the provider that just failed is about to receive every worker’s retry at once.

Spread of twelve retries under three jitter strategies Three timelines each showing twelve worker retries after a shared failure. Without jitter, all twelve land on the same instant. With equal jitter, they occupy the upper half of the window in a tight group. With full jitter, they are spread evenly across the whole window. no jitter all 12 at once equal jitter half the window is never used, so the peak is still high full jitter

Equal jitter — half the ceiling plus a random half — is a common compromise and still concentrates every retry in the upper half of the window. Full jitter uses the whole interval, which halves the peak instantaneous load for no extra complexity and one fewer arithmetic term.

Honouring Retry-After

When a provider sends Retry-After, it is telling you exactly when it will accept traffic again, and that beats any computed guess. Parse both forms — delta seconds and an HTTP date — and take the larger of the hint and the jittered delay.

from email.utils import parsedate_to_datetime
from datetime import datetime, timezone


def retry_after_seconds(header: str | None) -> float | None:
    """Parse a Retry-After header in either supported form."""
    if not header:
        return None
    header = header.strip()
    if header.isdigit():
        return float(header)
    try:
        when = parsedate_to_datetime(header)
    except (TypeError, ValueError):
        return None
    if when.tzinfo is None:
        when = when.replace(tzinfo=timezone.utc)
    return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())

Taking the maximum rather than replacing the computed delay matters when many workers receive the same header: they would all wake at the identical instant the provider named, which reintroduces the synchronised burst that jitter exists to prevent. Jitter above the hint keeps both properties.

The Ceiling Grows, the Draw Does Not

Growing ceiling, uniform draw beneath it Three attempts with ceilings of half a second, one second and two seconds. Above each ceiling a shaded band shows the interval from zero to the ceiling, and a marker shows one sampled delay drawn uniformly from that interval. The ceiling grows geometrically while each individual delay is random. attempt 1 · ceiling 0.5 s attempt 2 · ceiling 1.0 s attempt 3 · ceiling 2.0 s Shaded band = the interval sampled from · bar = the delay actually drawn The ceiling doubles so a struggling provider is called less often; the uniform draw means twelve workers occupying the same band land on twelve different instants rather than one. Both properties are needed: growth alone still synchronises, and jitter alone never backs off.

Attempts, Deadlines and What They Cost

An attempt cap alone does not bound time, and a deadline alone does not bound spend. Both are needed, and they answer different questions.

Bound Protects Symptom when missing
attempt cap provider quota and your bill spend rising with no increase in resolved addresses
deadline batch duration and worker occupancy a nightly job finishing hours late with no errors logged
per-attempt timeout one hung socket workers idle while a provider holds connections
breaker sustained failure thousands of doomed retries during an outage

The second row is the one that surprises teams, because the symptom looks nothing like a retry problem. A batch that used to finish at 03:00 and now finishes at 06:00, with a normal error rate and no alerts, is very often a provider whose latency drifted upward enough that retries and backoff now dominate the run.

Choosing the Base and the Cap

The base delay sets how quickly the first retry follows a failure, and the cap sets how far the sequence can stretch. Both should come from the provider’s recovery behaviour rather than from habit, and the two most useful reference points are its typical response time and the length of its shortest observed incident.

A base close to the provider’s median latency is a reasonable default. Retrying much sooner tends to arrive while the underlying problem is still present, which produces a second failure and wastes an attempt; retrying much later adds latency to records that would have succeeded immediately. Half a second suits providers whose median response is in the low hundreds of milliseconds.

The cap matters mainly during incidents. With a base of half a second and three attempts, the ceiling never exceeds two seconds, so the cap is inactive — it starts to matter when the attempt count is raised or when Retry-After hints are long. Setting it in the region of eight seconds keeps a single record from stalling a worker while still allowing a meaningful pause when a provider explicitly asks for one.

One further consideration applies to batch work specifically: the retry sequence competes with the rest of the batch for concurrency slots. A worker sleeping between attempts is a worker not dispatching new addresses, so long backoff sequences reduce throughput even when every individual record eventually succeeds. Where the batch has a fallback provider available, escalating after two attempts is frequently faster overall than persisting with three against a struggling primary.

Measure rather than assume. The attempt-count histogram per provider shows immediately whether the second and third attempts are earning their keep — if almost all successes happen on the first attempt and the rest fail regardless, the retry budget is better spent on the next provider in the chain.

Attempt Histograms Tell You What to Change

Do the second and third attempts earn their keep? Two histograms of attempts per successful lookup. In the first, a meaningful share of successes occur on the second attempt, so the retry budget is productive. In the second, almost all successes occur on the first attempt and the rest fail regardless, so the budget is better spent escalating to another provider. Attempts per successful lookup provider A retries work provider B retries do not first attempt · second · third For provider B, escalating after one attempt is both faster and cheaper — the retry budget buys nothing there.

Edge Cases and Failure Modes

Retrying a deterministic failure. A zero-result response is not transient, and three attempts produce three identical answers at triple the price. This is why the loop takes a dedicated Transient exception rather than catching broadly — the classification decision belongs to the caller, where the status code is visible.

Retrying a non-idempotent operation. Geocoding reads are safe to repeat, but a pipeline that writes as part of the same call is not. Attach the stable lookup key described in resilience patterns so a repeated request is recognisable on both sides.

Sleeping past the deadline. Checking the remaining budget before sleeping — rather than after waking — is what keeps the deadline meaningful. The version that sleeps first routinely overshoots by the length of the final delay.

Backoff inside and outside a library. Some SDKs retry internally by default. Stacking your loop on top multiplies the attempts, so a “three attempt” policy silently becomes nine. Disable the client’s own retries explicitly rather than assuming they are off.

Integration Note

This loop is the innermost of the resilience layers that make repeated calls, and it must sit inside the circuit breaker rather than around it. Inside, the breaker observes each attempt and opens promptly; outside, it sees one failure per logical lookup and takes three times as long to react — during which the retries it should have suppressed have already been sent. The per-attempt timeouts bound each individual call inside this loop.