Distributed Rate Limiting With Redis and Lua

TL;DR: When several processes share one provider quota, the bucket has to live outside them. A Lua script that refills, checks and decrements in a single atomic Redis call gives every worker a consistent view; splitting those three steps into separate commands re-introduces exactly the over-issuing the limiter exists to prevent. The single-process version is in rate limiting strategies for batch processing.

Why Local Limiters Multiply

An in-process limiter enforces its rate against the traffic it can see. Run four workers, each configured at the full provider rate, and the provider receives four times the agreed rate — which it will notice long before you do.

Four local buckets versus one shared bucket On the left, four workers each hold their own limiter configured at 300 requests per minute, and the provider receives 1200. On the right, the same four workers draw from a single Redis-held bucket of 300, and the provider receives 300 regardless of worker count. four local buckets 300/min 300/min 300/min 300/min provider receives 1 200/min symptom: sporadic 429s under no obvious load, invisible in any single worker's logs one shared bucket redis: 300/min provider receives 300/min worker count becomes an operational detail rather than a term in the rate calculation

Dividing the rate by the worker count works and is brittle: it breaks the moment a worker is added, an autoscaler reacts, or one process restarts and briefly overlaps its predecessor. A shared bucket removes worker count from the arithmetic entirely, which is worth the round trip it costs.

The Script

-- KEYS[1] = bucket key
-- ARGV[1] = capacity, ARGV[2] = refill per second
-- ARGV[3] = now (seconds, float), ARGV[4] = tokens requested
local capacity  = tonumber(ARGV[1])
local rate      = tonumber(ARGV[2])
local now       = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])

local state = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(state[1])
local ts     = tonumber(state[2])
if tokens == nil then
  tokens = capacity
  ts = now
end

-- Lazy refill: credit whatever has accrued since the last call.
local elapsed = math.max(0, now - ts)
tokens = math.min(capacity, tokens + elapsed * rate)

local allowed = tokens >= requested
local wait = 0
if allowed then
  tokens = tokens - requested
else
  wait = (requested - tokens) / rate
end

redis.call('HMSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', KEYS[1], math.ceil(capacity / rate) * 2)
return { allowed and 1 or 0, tostring(wait) }

Redis executes a script atomically, which is the entire reason for using one. The refill, the comparison and the decrement happen with no other client interleaved, so two workers arriving at the same instant cannot both observe the same last token.

Calling It From Python

from __future__ import annotations

import asyncio
import time

import redis.asyncio as aioredis


class DistributedLimiter:
    """Token bucket shared across processes via a Redis Lua script."""

    def __init__(self, r: aioredis.Redis, key: str, rate_per_s: float,
                 capacity: float | None = None) -> None:
        self._r = r
        self._key = key
        self._rate = rate_per_s
        self._capacity = capacity if capacity is not None else max(1.0, rate_per_s * 3)
        self._script = r.register_script(LUA_SOURCE)

    async def acquire(self, tokens: float = 1.0) -> None:
        """Block until a token is available. Sleeps the exact computed wait."""
        while True:
            allowed, wait = await self._script(
                keys=[self._key],
                args=[self._capacity, self._rate, time.time(), tokens],
            )
            if int(allowed) == 1:
                return
            await asyncio.sleep(min(float(wait), 5.0) or 0.01)

Returning the computed wait rather than a bare rejection is what keeps this efficient. A caller that knows it must wait 0.4 seconds sleeps once; a caller that only learns “no” polls, and a fleet of polling workers generates more Redis traffic than the geocoding itself.

Capacity, Not Just Rate

Capacity decides how much burst is allowed Three configurations at the same sustained rate. A capacity equal to one second of rate stalls on any scheduling hiccup. A capacity of three seconds absorbs normal jitter. A capacity of sixty seconds lets an idle pipeline release a minute of traffic at once and trip the provider's own limiter. capacity = 1 s no burst tolerance every scheduling hiccup becomes a stall too tight capacity = 3 s absorbs ordinary jitter without a visible burst at the provider the usual default capacity = 60 s an idle hour accrues a full minute of tokens released in one instant too loose

The middle setting is the one to start from, and the reason is that a shared bucket accumulates tokens whenever all workers are idle — which happens between batches, during deploys, and whenever the upstream queue empties. Capacity is the cap on how much of that idleness can be spent at once.

When Redis Is Unavailable

This is the design decision that distinguishes a limiter you can operate from one that turns a cache incident into a provider incident. Three options exist, and the right one depends on what a breach actually costs.

Failing open at the full rate is almost never right: it converts a Redis outage into a quota breach and possibly a temporary ban. Failing closed is safe and stops the pipeline entirely, which is a large blast radius for a dependency that is not on the critical path of correctness. The middle option — fall back to a conservative local limiter at the rate divided by the expected worker count — keeps the batch moving at reduced throughput without risking the contractual limit, and is what most pipelines settle on.

Whichever you choose, log the transition loudly. A pipeline silently running on its local fallback for a week will look healthy on every dashboard while quietly under-using its quota, and the only evidence is a throughput figure nobody was watching.

The Cost of the Round Trip

A shared bucket adds one Redis call per acquisition, and it is worth knowing what that costs against what it prevents.

One Redis round trip against the call it gates Three bars on a shared scale. A Redis script call takes about 0.4 milliseconds. A geocoding provider call takes about 200 milliseconds. The cool-down after a rate-limit breach can take tens of seconds. The Redis call is negligible against either. Time cost, same scale Redis script ~0.4 ms provider call ~200 ms — the thing being gated breach cool-down tens of seconds, plus whatever the provider decides — which is what the round trip buys insurance against

Pipelined against the request that follows it, the acquisition is invisible in end-to-end latency. The only workload where it is worth optimising away is one whose per-item work is itself sub-millisecond, which a geocoding call never is.

Testing a Distributed Limiter

The property worth testing is the one that a local limiter cannot violate and a distributed one can: that the total issued across all workers never exceeds the configured rate.

Write the test as a swarm. Start twenty coroutines against a single Redis instance with a bucket configured at a low rate, have each record the timestamp at which it was granted a token, and assert that no one-second window in the collected timestamps contains more grants than the rate allows. That single assertion covers the race the Lua script exists to prevent, and it fails reliably against a naive get-then-set implementation.

Use a real Redis rather than a fake. The whole correctness argument rests on script atomicity, and a mock that executes the script logic in Python provides no evidence about the property being tested. A container started for the test suite is inexpensive and tests the thing that ships.

Add a second test for the returned wait time: after exhausting the bucket, the reported wait should be within a small tolerance of the true refill interval. That number is what callers sleep on, and a wait that is systematically too short turns every blocked caller into a polling loop, which is a performance bug that no correctness test would catch.

Edge Cases and Failure Modes

Clock skew between workers. The script takes now from the caller, so a worker whose clock is ahead credits itself extra tokens. Pass Redis’s own time instead — TIME inside the script — when hosts cannot be trusted to stay in sync.

Script reloaded on every call. register_script uses EVALSHA and falls back to EVAL on a cache miss, which is correct; constructing the script object per call defeats it and sends the source every time.

Bucket keys per provider and per region. One key per rate limit, not one per process. If a provider meters separately by region or by endpoint, the key must include that dimension or the buckets will interfere.

Unbounded wait. A caller that sleeps the returned wait without a cap can block for a long time if the bucket is badly misconfigured. Cap the sleep and re-check, as the implementation above does.

Integration Note

The distributed limiter replaces the in-process one described in rate limiting strategies for batch processing and sits beside, not inside, the resilience stack: the limiter decides when a call may start, the breaker decides whether it is worth making, and the retry loop lives inside both. Sharing the Redis instance with the spend counters is normal and keeps the operational surface small.