Optimizing Batch Geocoding Throughput

Batch geocoding throughput is governed less by how fast you can call an API and more by how much work you can avoid. This guide, part of the Caching, Deduplication & Spatial Indexing section, is the performance capstone: it shows how to combine deduplication, cache reuse, bounded concurrency, and chunked I/O into one pipeline that converts a raw address file into geocoded rows at the highest sustainable records-per-second rate. The core idea is an ordering discipline — remove redundant work first, then spend concurrency only on what genuinely remains.

A naive loop that geocodes every row one at a time will process a few addresses per second and re-pay for every duplicate. A correctly ordered pipeline on the same hardware and the same API quota routinely runs one to two orders of magnitude faster, because the expensive network stage only ever sees the distinct, uncached subset of the input.

Batch geocoding throughput pipeline A left-to-right pipeline: a chunked CSV reader feeds a deduplication stage that collapses rows to distinct canonical keys. Distinct keys enter a cache lookup that splits into cache hits and cache misses. Misses flow into a bounded concurrent dispatcher calling the geocoding API. Hits and freshly geocoded misses merge into a bulk write that also populates the cache, then results are broadcast back onto duplicate rows. Chunked Read chunksize rows Deduplicate canonical key Cache Lookup hit / miss split hit miss Concurrent Dispatch semaphore + pool geocoding API Bulk Write populate cache broadcast to rows stream to storage backpressure

Prerequisites

The Pipeline Order That Matters

Throughput collapses when stages run in the wrong order. The correct sequence removes the most work as early and as cheaply as possible:

  1. Chunked read — pull a bounded number of rows so memory stays flat.
  2. Deduplicate first — collapse the chunk to distinct canonical keys. This is pure in-process work and is the cheapest possible way to eliminate an API call.
  3. Cache lookup — batch-check every distinct key against the cache and split the set into hits and misses. Hits cost one cache round trip, not one API call.
  4. Geocode only misses — dispatch the surviving distinct, uncached keys concurrently. This is the only stage that touches the network, and it now sees the smallest possible input.
  5. Bulk write — persist the freshly geocoded misses to the cache in one round trip, then broadcast all resolved coordinates back onto the original (duplicate-inclusive) rows.

Every reordering that moves geocoding earlier — geocoding before dedup, or before the cache check — pays for work that a downstream stage would have made unnecessary. Deduplication before the cache check also shrinks the number of cache lookups, which matters when the cache is a network hop away.

Step 1 — Stream the Input in Bounded Chunks

Loading a multi-gigabyte address file into a single DataFrame is the most common cause of out-of-memory failures in batch geocoding. Use pandas.read_csv with chunksize to get an iterator of fixed-size frames:

import pandas as pd
from typing import Iterator

def read_chunks(path: str, chunksize: int = 50_000) -> Iterator[pd.DataFrame]:
    """Yield fixed-size row chunks so peak memory is bounded by chunksize.

    Args:
        path: Path to the source CSV of raw addresses.
        chunksize: Rows held resident per iteration.

    Yields:
        DataFrame chunks of at most chunksize rows.
    """
    for chunk in pd.read_csv(path, chunksize=chunksize, dtype=str):
        yield chunk.fillna("")

Peak memory now scales with chunksize, not with the total file size. A 20-million-row file processes with the same footprint as a 50-thousand-row file — it simply takes more iterations.

Step 2 — Deduplicate, Then Resolve the Cache

Within each chunk, collapse rows to their canonical key, keep the distinct set, and check the cache in one batched call. The generating canonical address keys in Python guide covers the key function; here it is imported as canonical_key.

import pandas as pd
from typing import Callable

def dedup_and_split(
    chunk: pd.DataFrame,
    address_col: str,
    canonical_key: Callable[[str], str],
    cache_mget: Callable[[list[str]], dict[str, dict]],
) -> tuple[pd.DataFrame, list[str], dict[str, dict]]:
    """Add a canonical key, deduplicate, and split distinct keys into hits/misses.

    Args:
        chunk: Raw row chunk.
        address_col: Column holding the raw address string.
        canonical_key: Function mapping a raw address to its canonical key.
        cache_mget: Batched cache read returning {key: result} for keys that hit.

    Returns:
        (chunk with 'ckey' column, list of miss keys, dict of cache hits).
    """
    chunk = chunk.copy()
    chunk["ckey"] = chunk[address_col].map(canonical_key)
    distinct_keys = chunk["ckey"].unique().tolist()
    hits = cache_mget(distinct_keys)          # one round trip for the whole chunk
    misses = [k for k in distinct_keys if k not in hits]
    return chunk, misses, hits

Note the two-level saving: unique() removes intra-chunk duplicates before any lookup, and cache_mget removes cross-run duplicates in a single batched read rather than one call per key.

Step 3 — Dispatch Only the Misses Concurrently

The miss list is the only input that reaches the network. Because geocoding an address over HTTP is I/O-bound, asyncio with a bounded semaphore delivers the highest single-core request concurrency. The pattern below mirrors building async geocoding requests in Python and reuses a single connection-pooled client.

import asyncio
import httpx
from typing import Optional

async def geocode_one(
    client: httpx.AsyncClient,
    key: str,
    address: str,
    sem: asyncio.Semaphore,
) -> tuple[str, Optional[dict]]:
    """Geocode a single canonical key, bounded by the shared semaphore."""
    async with sem:
        try:
            resp = await client.get("/v1/geocode", params={"q": address})
            resp.raise_for_status()
            return key, resp.json()
        except (httpx.HTTPStatusError, httpx.RequestError):
            return key, None

async def geocode_misses(
    miss_addrs: dict[str, str],
    concurrency: int = 40,
    pool: int = 100,
) -> dict[str, dict]:
    """Concurrently geocode a {key: address} map of cache misses.

    Args:
        miss_addrs: Distinct, uncached canonical keys mapped to a representative address.
        concurrency: Upper bound on simultaneous in-flight requests (backpressure).
        pool: Max pooled connections; keep >= concurrency to avoid socket queueing.

    Returns:
        {key: geocode_result} for every miss that resolved successfully.
    """
    sem = asyncio.Semaphore(concurrency)
    limits = httpx.Limits(max_connections=pool, max_keepalive_connections=pool)
    async with httpx.AsyncClient(
        base_url="https://api.provider.example", limits=limits, timeout=8.0
    ) as client:
        tasks = [
            geocode_one(client, k, addr, sem)
            for k, addr in miss_addrs.items()
        ]
        results = await asyncio.gather(*tasks)
    return {k: v for k, v in results if v is not None}

The Semaphore is the backpressure control: it caps in-flight requests so the event loop never launches more work than the provider tolerates or the connection pool can serve. Keep the pool size at or above the concurrency ceiling — if the pool is smaller, requests silently queue on connection acquisition and your effective concurrency drops below the semaphore limit.

When the miss set instead needs heavy CPU-bound normalization or parsing before it can be geocoded, dispatch that stage to a process pool rather than the event loop; parallel geocoding with multiprocessing in Python covers that variant.

Step 4 — Bulk-Write and Broadcast Back to Rows

Freshly geocoded misses are written to the cache in one batched call, then all resolved coordinates — hits plus new misses — are broadcast back onto the original rows via a key join. Writing per row would serialize the fast part of the pipeline behind network latency.

import pandas as pd
from typing import Callable

def merge_and_write(
    chunk: pd.DataFrame,
    hits: dict[str, dict],
    new_results: dict[str, dict],
    cache_mset: Callable[[dict[str, dict]], None],
) -> pd.DataFrame:
    """Persist new results to cache once, then map all results back to rows.

    Args:
        chunk: Chunk carrying the 'ckey' column from dedup_and_split.
        hits: Results already present in the cache.
        new_results: Freshly geocoded miss results.
        cache_mset: Batched cache write for new results.

    Returns:
        Chunk with a 'geocode_result' column populated for every row.
    """
    if new_results:
        cache_mset(new_results)               # single bulk write
    resolved = {**hits, **new_results}
    chunk = chunk.copy()
    chunk["geocode_result"] = chunk["ckey"].map(resolved)  # broadcast to duplicates
    return chunk

The map on ckey is where deduplication pays off a second time: one geocoded result populates every duplicate row for free, with no additional call.

Throughput Levers Reference

Lever What it controls Typical setting Failure if misapplied
Dedup ratio Distinct keys ÷ input rows Collapse aggressively via canonical key Weak keys leave duplicates that re-geocode
Cache hit rate Fraction of distinct keys served from cache Persist results across runs; long TTL for stable addresses Cold cache forces every key to the API
Concurrency (semaphore) Simultaneous in-flight requests Lowest value that saturates the rate limit Too high → 429 storms; too low → idle pool
Connection pool size Reusable sockets >= concurrency Smaller than concurrency → hidden queueing
Chunk size Rows resident per iteration 25k–100k rows Too large → OOM; too small → per-chunk overhead
Process-pool workers CPU cores for normalization os.cpu_count() for CPU-bound stages Over-subscription → context-switch thrash
Bulk write batch Rows per cache/db write 1k–10k per write Per-row writes serialize the pipeline

Edge Cases

Duplicate-Heavy vs Unique-Heavy Batches

A duplicate-heavy batch (e.g. transaction logs where the same store appears thousands of times) is dominated by dedup and cache broadcast — the API stage is tiny, so raising concurrency barely helps. A unique-heavy batch (a fresh national address list) barely benefits from dedup and is bound entirely by the geocoding stage; here concurrency, connection pooling, and rate-limit headroom are what move the needle. Profile the dedup ratio first, because it tells you which stage to tune.

Memory Blowups From Hidden Materialization

chunksize bounds the read, but a downstream pd.concat of all chunks, an unbounded results dict, or asyncio.gather over a million tasks at once re-materializes the whole job in memory. Keep the async fan-out scoped to one chunk’s miss set, stream each finished chunk to disk, and drop references before the next iteration.

GIL Contention on CPU-Bound Normalization

If each address needs expensive parsing (libpostal, spaCy, heavy regex) before geocoding, running that under asyncio pins it to one core because the GIL serializes Python bytecode. Throughput plateaus no matter how high the semaphore goes. Move the CPU-bound stage to a ProcessPoolExecutor and keep only the network stage on the event loop.

Skewed Chunk Latency

One chunk full of rural or international addresses can take far longer than a chunk of clean urban ones, because those records trigger deeper provider-side matching. Fixed-size chunks then produce uneven wall-clock times. Size chunks by expected cost, not just row count, or process chunks through a bounded worker queue so a slow chunk does not stall the whole run.

Performance and Scaling

Scenario Bottleneck Primary lever Expected effect
60% duplicate transaction file In-process dedup Canonical key + cache broadcast 2–3x fewer API calls before any tuning
Warm cache, repeated nightly run Cache read latency Batched mget, local pipelining Most keys served without a call
Cold unique national list API concurrency Semaphore + pool sizing to rate limit Near-linear up to the rate ceiling
CPU-heavy normalization first GIL ProcessPoolExecutor for parse stage Scales with core count
20M-row file, 16GB host Memory chunksize + streamed writes Flat footprint regardless of size

Report throughput as records per second end to end — total input rows divided by wall-clock time — not API calls per second, because dedup and cache hits resolve rows with no call at all. A pipeline can show 40 calls/sec while delivering 4,000 records/sec if the cache and dedup layers are doing their job.

Troubleshooting

Throughput Falls When Concurrency Rises

You have crossed the provider’s rate limit. Extra tasks now generate HTTP 429s, retries, and backoff, so net records/sec drops while spend climbs. Cap the semaphore at the rate limit derived in rate limiting strategies for batch processing and stop raising it once 429s appear.

Cache Hit Rate Is Near Zero on the Second Run

The cache key is not deterministic. If the canonical key depends on dict ordering, locale, or an un-normalized string, identical addresses hash differently between runs. Verify the key function is stable and idempotent before blaming the cache — the generating canonical address keys guide covers determinism requirements.

Event Loop Idles While the Pool Is Full

Effective concurrency is capped by the connection pool, not the semaphore, because max_connections is smaller than the semaphore limit. Requests queue on connection acquisition and the loop sits idle waiting for a free socket. Raise httpx.Limits(max_connections=...) to at least the semaphore ceiling.

Memory Climbs Across Chunks Instead of Staying Flat

Results are accumulating in a list or dict that is never released, or finished chunks are being concatenated instead of streamed out. Write each chunk’s output to durable storage immediately and drop the reference. Peak memory should return to baseline between chunks.

Bulk Write Becomes the Bottleneck

If the geocoding stage is fast but records/sec is still poor, the cache or database write is serialized. Batch writes into groups of 1k–10k rows, use a pipeline or COPY-style bulk insert, and write asynchronously so the next chunk’s dedup and lookup proceed while the previous chunk persists.

FAQ

What is the single biggest throughput lever for batch geocoding?

Deduplication almost always wins. Real-world address files are 20–60% duplicates once collapsed to a canonical key, and every duplicate you remove is an API call you never pay for or wait on. Cache reuse across runs compounds this. Only after dedup and caching have removed redundant work does raising concurrency meaningfully increase records per second.

Should I use asyncio or multiprocessing for batch geocoding?

Match the tool to the bottleneck. Network calls to a geocoding API are I/O-bound, so asyncio with a bounded semaphore gives the highest request concurrency on a single core. CPU-bound work — libpostal parsing, Unicode normalization, fuzzy key generation — is throttled by the GIL under asyncio and belongs in a process pool. Large pipelines use both: a process pool normalizes and keys records, then an async client dispatches the surviving cache misses.

How do I keep a 10-million-row geocoding job from exhausting memory?

Never materialize the whole file. Read with pandas read_csv chunksize (or an iterator) so only one chunk is resident at a time, deduplicate and cache-resolve within each chunk, stream results to durable storage, and release the chunk before reading the next. Peak memory then scales with chunk size, not with total row count.

Why does adding more concurrency sometimes lower throughput?

Past the provider’s rate limit, extra concurrency only generates HTTP 429 responses, retries, and backoff — net throughput falls while cost rises. Concurrency also has to stay below your connection-pool size or requests queue on socket acquisition. The optimal point is the lowest concurrency that saturates either the rate limit or the pool, whichever binds first.

How should I measure batch geocoding throughput?

Report records/sec end to end (total input rows divided by wall-clock time), not just API calls per second, because dedup and cache hits process rows without any call. Track cache hit rate, dedup ratio, in-flight request count, and 429 rate alongside it. A job can show a healthy calls/sec figure while overall records/sec is poor if the cache layer is slow or the bulk write is serialized.