Setting Timeouts for Geocoding HTTP Clients in Python

TL;DR: Set three bounds, not one — connect at about two seconds, read a little above the provider’s slow-response tail, and a hard total derived from its 99th percentile plus margin. Configure them explicitly in aiohttp or httpx, because neither imposes a useful default. The surrounding patterns live in resilience patterns for geocoding APIs.

The Configuration

import aiohttp
import httpx

# aiohttp — one object, applied to every request on the session
AIOHTTP_TIMEOUT = aiohttp.ClientTimeout(
    total=8.0,          # hard ceiling for the whole request
    connect=2.0,        # time to obtain a connection, including pool wait
    sock_connect=1.5,   # time for the TCP handshake alone
    sock_read=4.0,      # maximum gap between received chunks
)

# httpx — the same three ideas, named slightly differently
HTTPX_TIMEOUT = httpx.Timeout(
    timeout=8.0,        # default for anything not named below
    connect=2.0,
    read=4.0,
    write=2.0,
    pool=2.0,           # wait for a free connection from the pool
)

Neither library imposes a helpful default. aiohttp applies a five-minute total timeout unless told otherwise, and httpx defaults to five seconds for everything — the first is far too long for a batch, and the second is a single number applied to four different waits.

What Each Bound Actually Protects

The three bounds catch three distinct failures, which is why one number cannot replace them. Each is set from a different part of the latency distribution.

Three bounds, three failures Three rows. The connect bound catches an unreachable or saturated host and is set low because a healthy connection completes in well under a second. The read bound catches a stalled response stream where bytes stop arriving. The total bound catches a response that keeps progressing but is too slow to be useful, and is derived from the 99th percentile. connect catches: host unreachable, DNS stall, saturated connection pool set from: a healthy handshake — well under a second, so 2 s is generous read catches: a connection that opens then goes silent mid-response set from: the gap between chunks, not the whole response — p95 plus margin total catches: a slow but progressing response you can no longer afford to wait for set from: this provider's p99 plus roughly 50% — never a round number

The read bound is the one most often set wrong, because it is intuitively read as “how long the response may take”. It is not: it bounds the gap between successive chunks, so a large response arriving steadily over ten seconds never trips a four-second read timeout. That is what the total bound is for.

Deriving the Numbers

Timeouts copied from another service are guesses. The five-minute exercise below produces defensible values from traffic you already have, and it is worth repeating whenever a provider’s performance visibly shifts.

import numpy as np
import pandas as pd


def derive_timeouts(latencies_ms: pd.Series, margin: float = 1.5) -> dict[str, float]:
    """Derive connect/read/total bounds from observed successful-call latencies."""
    p50, p95, p99 = np.percentile(latencies_ms.dropna(), [50, 95, 99])
    return {
        "connect": 2.0,                          # network-bound, not provider-bound
        "read": round(max(2.0, p95 / 1000 * margin), 1),
        "total": round(max(4.0, p99 / 1000 * margin), 1),
        "observed_p50_ms": round(float(p50), 1),
        "observed_p99_ms": round(float(p99), 1),
    }

Feed it only successful calls. Including failures pulls the percentiles toward whatever the current timeout is, which produces a slow ratchet: each recalculation raises the ceiling a little, and after a few rounds the timeout is set by its own history rather than by the provider.

Timeouts and Retries Interact

A total timeout of eight seconds with three retries is a worst case of well over twenty-four seconds per record, before backoff delays. That arithmetic is what determines whether a batch finishes in its window, and it is routinely overlooked because the two settings live in different files.

Setting Value Worst case per record
total timeout 8 s 8 s
retries 3 attempts 24 s of request time
backoff delays 1 s, 2 s, 4 s + 7 s
chain fallback 1 further provider ≈ 62 s

Sixty-two seconds for one record is fine for an interactive lookup that almost never hits the worst case, and catastrophic for a batch where a provider incident pushes a large fraction of records onto that path. Bound the whole logical lookup as well as each attempt — a deadline that spans the retries and the fallback keeps the batch predictable.

Per-attempt bounds versus one overall deadline The upper timeline shows three attempts against the primary provider plus one against the fallback, each bounded individually, accumulating to over a minute. The lower timeline shows the same attempts under a single twenty-second deadline that cuts the sequence short and returns a partial result. per-attempt bounds only ≈ 62 s worst case attempt 1 · attempt 2 · attempt 3 · fallback provider — each individually bounded, collectively unbounded with an overall deadline deadline — return what we have The record is marked unresolved and queued, rather than holding a worker for a minute during an incident. A deadline is one number and it is the one that makes batch duration predictable.

Choosing a Margin

The multiplier applied to the observed percentile is a judgement call, and it is worth making deliberately rather than defaulting to a round number. A margin near 1.2 keeps the pipeline tight and will abandon a meaningful share of genuinely healthy slow responses whenever the provider has a mildly bad hour. A margin near 2.0 tolerates those hours and lets a degraded provider hold workers for twice as long as it should.

The right value depends on what happens after a timeout. If the next step is a cheap fallback provider with good coverage, a tight margin is fine — abandoning a slow response costs one extra call to a provider that will probably answer. If the next step is an expensive provider, or a dead-letter queue and a human, then a generous margin is better, because the cost of giving up early is far higher than the cost of waiting.

That reasoning also explains why the margin should differ between an interactive path and a batch path against the same provider. A checkout form has a human waiting and should give up quickly and degrade gracefully; a nightly batch has nobody waiting and should wait longer rather than pay for a second call. Two configurations, one provider, and the only difference is what a timeout costs in each context.

Recompute the numbers quarterly, and whenever a provider announces infrastructure changes. Latency distributions drift as providers add regions and change routing, and a timeout set from last year’s percentiles is a timeout set from a system that no longer exists. Keeping the derivation in code rather than in a runbook means the recomputation is a job you can schedule instead of a task somebody has to remember.

One more practical note on the derivation: exclude the first few minutes after a deployment from the sample. Cold connection pools, empty DNS caches and unwarmed provider-side routing all inflate early latencies, and a percentile computed across a restart window produces bounds set by conditions that last for seconds and never recur.

Where the Margin Lands

Choosing the ceiling against the real distribution A right-skewed latency distribution with markers at the median, the 95th and the 99th percentiles. A tight candidate ceiling sits inside the tail and would abandon a visible share of healthy responses. The derived ceiling sits beyond the 99th percentile plus a margin and abandons almost none. p50 p95 p99 a tight ceiling abandons healthy responses derived fast slow Every response to the right of a ceiling becomes a retry — billed again, and no faster.

Edge Cases and Failure Modes

Pool wait counted as connect time. In aiohttp, connect includes time spent waiting for a free connection from the pool, so a saturated pool presents as connect timeouts on a perfectly healthy provider. Use sock_connect to separate the handshake from the queue, and check the bulkhead sizing before touching the timeout.

DNS resolution outside the timeout. Some resolver configurations block outside the event loop, so a DNS outage can stall past any client-level bound. Enable the client’s own DNS caching and set a resolver timeout at the system level; the HTTP timeout cannot save you from a synchronous resolver.

Timeouts that fire during the response body. A provider returning a large batch response may legitimately take longer than a single-address call. Configure per-endpoint timeouts rather than per-session ones where the same client serves both shapes.

Retry storms triggered by a too-tight total. Lowering the total timeout looks like it speeds up a batch and often slows it down, because responses that would have arrived become retries. Watch the success rate when tightening, not just the latency.

Verifying the Settings Actually Apply

Timeout configuration is unusually easy to get wrong in a way that leaves no trace: the object is constructed, passed somewhere that ignores it, and the client quietly uses its default. Two checks catch that before production does.

The first is a deliberate failure test. Point the client at a host that accepts connections and never responds — a local socket that accepts and sleeps is enough — and assert that the call raises within the expected window. If it raises after five minutes instead of eight seconds, the timeout object never reached the request. This test runs in seconds and is the only reliable way to know the configuration is live.

The second is an assertion at start-up that the session carries the expected values. Reading them back from the client object and logging them once per process costs nothing and turns a silent misconfiguration into a line in the startup log that somebody will notice during an incident. It also documents the effective values for whoever is on call, which is more useful than a constant buried three modules deep.

Both checks matter more in a codebase where several clients exist. A worker that constructs its own session for a health check, or a library that creates an internal client, will not inherit your carefully derived bounds — and those are precisely the code paths that hang during an incident, because nobody thought to configure something that was only ever meant to be a quick call.

Integration Note

The bounds set here are the innermost layer of the resilience stack: they bound one attempt, the retry loop bounds the attempts, and the breaker decides whether to attempt at all. They also interact directly with async concurrency settings, because a timeout is what returns a worker slot to the pool — with no total bound, a stalled provider consumes concurrency indefinitely and the whole dispatcher grinds down.