Caching Geocoding Results With Redis TTL in Python

Cache a geocoding result in Redis with SETEX key ttl value, where the TTL is chosen by how stable the address is — long for mature rooftop results, short for volatile or low-confidence ones — and always add random jitter so a batch loaded together does not expire in one spike; this page drills into that TTL strategy within Redis and Postgres caching patterns.

Choosing a TTL by Address Stability

There is no single correct TTL. The right value trades cost against freshness: too long and you pin a stale coordinate, too short and you re-bill the provider for an address that never changed. The deciding factor is how likely the underlying result is to change, which correlates tightly with result precision and confidence.

The mechanism is SETEX (set with expiry) or the equivalent SET key value EX ttl. Redis also exposes EXPIRE key ttl to set or refresh a TTL on an existing key, and TTL key to inspect remaining seconds. Prefer setting the TTL atomically at write time with SETEX rather than a SET followed by a separate EXPIRE, which can leave a key permanent if the second call fails.

Anatomy of a Safe Cache Key

The key carries more than the address. It has to encode everything that could change the answer — otherwise a change in provider or normalisation logic silently serves results derived under different rules. Four segments cover it, and each one earns its place by naming a thing that can change independently.

Twelve hex characters is 48 bits, which keeps collisions far below the level where they matter for a cache of a few hundred million entries. If that reasoning makes you uneasy, store the canonical address in the value and compare on read — a collision then costs one extra provider call rather than a wrong coordinate.

Key and TTL Component Breakdown

Compile-Once Key Builder

Build the key deterministically and once, at module load, so every call site produces identical keys. The normalization here must match the fuller canonical form used across the pipeline; see Unicode and character normalization in Python.

from __future__ import annotations

import hashlib
import re
import unicodedata

# Compile once at module level.
_MULTI_WS = re.compile(r"\s+")
_PUNCT = re.compile(r"[.,#]+")
_KEY_PREFIX = "geo:"


def build_key(raw: str, country_code: str = "US") -> str:
    """Return a stable Redis key for a raw address string.

    Normalizes to a canonical form, then hashes with SHA-256 so the
    same logical address always maps to the same key.
    """
    nfc = unicodedata.normalize("NFC", raw).strip().lower()
    canonical = _MULTI_WS.sub(" ", _PUNCT.sub(" ", nfc)).strip()
    pre_image = f"{canonical}|{country_code.upper()}"
    digest = hashlib.sha256(pre_image.encode("utf-8")).hexdigest()
    return f"{_KEY_PREFIX}{digest}"

The Stampede a Fixed TTL Creates

Cache entries written together expire together. A nightly batch that resolves 400 000 addresses at 02:00 with a 30-day TTL creates 400 000 entries that all become invalid at 02:00 thirty days later — and the batch that runs at that moment pays full price for every one of them, in one burst, against a rate limiter that was never sized for it.

Ten percent of jitter is usually enough and costs nothing: ttl + random.randint(-p, p) at write time. The same trick protects the write side of a read-through cache, where a popular key expiring can otherwise send every concurrent worker to the provider for the same address simultaneously.

Jittered TTL and get_or_geocode

Jitter matters more than it looks. When a nightly job loads 100k addresses into cache with an identical 30-day TTL, all 100k keys expire in the same second 30 days later — and the next run stampedes the provider. Spreading each TTL by a random ±15% turns that spike into a smooth distribution.

from __future__ import annotations

import json
import random
from typing import Awaitable, Callable, Optional

import redis.asyncio as aioredis

_NEGATIVE = "\x00NEG"

# TTL bands in seconds, keyed by a coarse stability label.
_TTL_BANDS = {
    "stable": 60 * 60 * 24 * 90,   # 90 days
    "normal": 60 * 60 * 24 * 30,   # 30 days
    "volatile": 60 * 60 * 24 * 7,  # 7 days
    "negative": 60 * 60 * 4,       # 4 hours
}

Fetcher = Callable[[str, str], Awaitable[Optional[dict]]]


def jittered(ttl: int, spread: float = 0.15) -> int:
    """Spread a TTL by +/- `spread` so batch entries expire gradually."""
    delta = int(ttl * spread)
    return max(1, ttl + random.randint(-delta, delta))


def _band_for(result: dict) -> str:
    """Pick a TTL band from result precision/confidence."""
    precision = result.get("precision", "range")
    confidence = float(result.get("confidence", 0.0))
    if precision == "rooftop" and confidence >= 0.9:
        return "stable"
    if precision in ("centroid", "postal") or confidence < 0.6:
        return "volatile"
    return "normal"


async def get_or_geocode(
    r: aioredis.Redis,
    raw: str,
    fetcher: Fetcher,
    country_code: str = "US",
) -> Optional[dict]:
    """Return a cached geocode or resolve it once and cache it.

    Caches genuine no-results negatively with a short TTL. Transient
    provider errors are re-raised, never cached, so retries stay valid.
    """
    key = build_key(raw, country_code)

    cached = await r.get(key)
    if cached is not None:
        return None if cached == _NEGATIVE else json.loads(cached)

    try:
        result = await fetcher(raw, country_code)
    except Exception:
        # Transient failure (429/503/timeout): do not cache, let caller retry.
        raise

    if result is None:
        await r.set(key, _NEGATIVE, ex=jittered(_TTL_BANDS["negative"]))
        return None

    ttl = _TTL_BANDS[_band_for(result)]
    await r.set(key, json.dumps(result), ex=jittered(ttl))
    return result

Note the error discipline: a genuine None no-result is cached negatively, but an exception (rate limit, timeout, provider outage) propagates untouched so the caller can retry or cascade. Caching a transient error would suppress a valid address for hours — the same classification rule used when implementing fallback chains for failed lookups.

Vectorized pandas Example

For a DataFrame, resolve the whole hot tier in one MGET, then route only the misses to the provider. This keeps Redis round trips to one per batch instead of one per row.

from __future__ import annotations

import json
from typing import Callable, Optional

import pandas as pd
import redis

_NEGATIVE = "\x00NEG"


def cache_lookup_df(
    df: pd.DataFrame,
    r: redis.Redis,
    resolve_miss: Callable[[str, str], Optional[dict]],
    address_col: str = "address",
    country_col: str = "country_code",
) -> pd.DataFrame:
    """Attach cached geocodes to a DataFrame with a single MGET.

    Rows still missing after the cache read are resolved via
    `resolve_miss` and written back with a jittered TTL.
    """
    df = df.copy()
    df["key"] = [
        build_key(a, c)
        for a, c in zip(df[address_col], df[country_col].fillna("US"))
    ]
    hits = r.mget(df["key"].tolist())
    df["result"] = [
        json.loads(v) if v and v != _NEGATIVE else None for v in hits
    ]

    miss_mask = df["result"].isna()
    for idx in df.index[miss_mask]:
        raw = df.at[idx, address_col]
        country = df.at[idx, country_col] or "US"
        result = resolve_miss(raw, country)
        if result is not None:
            df.at[idx, "result"] = result
            r.set(
                df.at[idx, "key"],
                json.dumps(result),
                ex=jittered(_TTL_BANDS[_band_for(result)]),
            )
    return df

Serialising the Value Without Regret

What goes in the value determines what you can do later without re-calling the provider. The temptation is to store a bare latitude and longitude; the version that ages well stores the precision tier, the answering provider and the time it was obtained, because every one of those becomes a question within the first quarter of running the cache.

The third row has a legal dimension as well as an operational one. If a contract ends and its terms require deleting stored results, a cache that does not record which provider produced each entry leaves you flushing everything — throwing away years of results from providers you are still entitled to keep.

Edge Cases

Mass expiry from un-jittered TTLs

A batch written with identical TTLs expires as a block, producing a provider spike exactly one TTL later. Always route TTLs through jittered(). If you inherited a cache already loaded without jitter, run a one-off pass that re-EXPIREs a sample of keys with staggered values to break up the block before it expires.

Redis eviction is not the same as TTL expiry

Under maxmemory pressure Redis evicts keys before their TTL if the eviction policy allows it. That is acceptable only when a durable tier can backfill the loss; never treat a Redis-only geocode as authoritative. Persist paid results to the durable store described in Postgres materialized view geocode cache so an eviction costs latency, not money.

Refreshing TTL on read (sliding expiry)

If you want frequently-requested addresses to stay hot, call EXPIRE key ttl on a cache hit to slide the window forward. Use this sparingly for geocoding: an address that is queried often is not necessarily more likely to have changed, so sliding expiry can pin a stale result. Prefer fixed TTLs tied to stability unless request frequency is a genuine freshness signal for your data.

Integration Note

This Redis TTL layer is the hot half of the two-tier design in Redis and Postgres caching patterns: Redis absorbs repeat lookups at sub-millisecond latency while a durable Postgres tier survives eviction and restarts. Pair the miss-routing step with the budget controls in API quota tracking and cost management so that even after caching, the residual provider traffic stays within quota.

---

Related

• Redis and Postgres caching patterns — the full two-tier cache design this TTL strategy plugs into.
• Postgres materialized view geocode cache — the durable tier that backfills Redis after eviction or restart.
• Unicode and character normalization in Python — the canonical normalization the key builder depends on.
• API quota tracking and cost management — cap the provider spend that remains after cache hits.