Fuzzy Address Matching with rapidfuzz in Python

TL;DR: For address strings, token_sort_ratio is the best default rapidfuzz scorer because it neutralizes token reordering; use token_set_ratio when one string has extra tokens, reserve WRatio for messy free-text, and threshold around 88-90. This page supports deduplicating addresses by fuzzy canonical key, where fuzzy scoring recovers the residual near-misses an exact key cannot merge.

Why rapidfuzz for Addresses

rapidfuzz is a C+±backed reimplementation of the fuzzy-matching API popularized by fuzzywuzzy, returning similarity scores on a 0-100 scale. It is an order of magnitude faster and has no license friction, which matters when you are scoring millions of address pairs inside blocking buckets. The value it adds over an exact canonical key is tolerance for typos, OCR noise, and partial tokens — the exact key merges 123 Main St with 123 Main Street, but only fuzzy scoring will merge 123 Main St with 123 Maine St.

Scorer Breakdown

Each scorer treats tokens and ordering differently. Choosing the wrong one is the most common cause of both false merges and missed duplicates.

For addresses that have already passed through canonicalization, token order is unstable (people write the unit before or after the street), so an order-insensitive scorer is essential. token_sort_ratio is the workhorse. Reach for token_set_ratio when comparing a bare street address against one that also carries a city and ZIP — the set logic ignores the extra tokens instead of penalizing them.

Choosing a Threshold

Thresholds are data-dependent; calibrate against a labelled sample rather than guessing.

A robust pattern is a two-tier cutoff: auto-merge above the high threshold, queue for human review in a middle band, and reject below. This keeps precision high on automatic decisions while not silently discarding ambiguous pairs.

Parse-and-Score Function

The function below scores a query against a candidate, guarding against the null and empty inputs that are endemic in real address data. Both strings are assumed to be canonicalized already — pushing normalization upstream keeps the scorer focused and fast.

from __future__ import annotations

from dataclasses import dataclass
from typing import Optional

from rapidfuzz import fuzz, utils


@dataclass
class MatchResult:
    """Outcome of comparing two address strings."""
    score: float
    is_match: bool
    scorer: str


def score_pair(
    left: Optional[str],
    right: Optional[str],
    threshold: float = 90.0,
) -> MatchResult:
    """Score two address strings with token_sort_ratio.

    Args:
        left: First canonicalized address string (may be None/empty).
        right: Second canonicalized address string (may be None/empty).
        threshold: Minimum score (0-100) to count as a match.

    Returns:
        A MatchResult with the score, match flag, and scorer name.
    """
    if not left or not right:
        return MatchResult(score=0.0, is_match=False, scorer="token_sort_ratio")
    score = fuzz.token_sort_ratio(left, right, processor=utils.default_process)
    return MatchResult(
        score=score,
        is_match=score >= threshold,
        scorer="token_sort_ratio",
    )

Passing processor=utils.default_process applies rapidfuzz’s built-in lowercasing and non-alphanumeric trimming, a cheap safety net even when inputs are already canonicalized.

Scoring Blocked Candidates with process.cdist

Never score the full Cartesian product. Within a blocking bucket, process.cdist computes a full score matrix in one vectorized C++ call — far faster than a Python double loop and the right tool once a block is small.

import numpy as np
from rapidfuzz import fuzz, process, utils


def block_score_matrix(
    queries: list[str],
    candidates: list[str],
    score_cutoff: float = 90.0,
) -> np.ndarray:
    """Compute a query x candidate score matrix for one blocking bucket.

    Scores below score_cutoff are returned as 0 to keep the matrix sparse.

    Args:
        queries: Canonical address strings on the row axis.
        candidates: Canonical address strings on the column axis.
        score_cutoff: Scores under this become 0.

    Returns:
        A 2D numpy array of shape (len(queries), len(candidates)).
    """
    return process.cdist(
        queries,
        candidates,
        scorer=fuzz.token_sort_ratio,
        processor=utils.default_process,
        score_cutoff=score_cutoff,
        workers=-1,  # use all cores
    )

workers=-1 parallelizes the matrix across every available core with no GIL contention, since the heavy lifting happens in C++. For a symmetric self-join within a block, pass the same list as both arguments and read the upper triangle to avoid double-counting each pair.

Vectorized pandas Example

To score a DataFrame of candidate pairs already produced by a blocking join, apply the scorer column-wise. This assumes an upstream step has emitted left_addr / right_addr pairs (for example, a self-join within each blocking bucket).

import pandas as pd
from rapidfuzz import fuzz, utils


def score_pairs_frame(pairs: pd.DataFrame) -> pd.DataFrame:
    """Add a fuzzy score column to a DataFrame of candidate address pairs.

    Args:
        pairs: DataFrame with 'left_addr' and 'right_addr' columns.

    Returns:
        The DataFrame with an added 'score' column (0-100).
    """
    out = pairs.copy()
    left = out["left_addr"].fillna("").astype(str)
    right = out["right_addr"].fillna("").astype(str)
    out["score"] = [
        fuzz.token_sort_ratio(a, b, processor=utils.default_process)
        for a, b in zip(left, right)
    ]
    return out

For blocks large enough that even the pair list is expensive to materialize, prefer process.cdist on the raw block arrays and threshold the matrix before extracting pairs — it avoids building a wide intermediate DataFrame.

Edge Cases

Numeric Tokens Dominate the Score

House numbers and ZIP codes are short numeric tokens, and token_sort_ratio treats 123 and 132 as fairly similar. Two different buildings on the same street can score above threshold. Compare the street number as a separate exact field and only fuzzy-score the remaining street tokens, so a house-number mismatch vetoes the merge regardless of overall similarity.

Abbreviation Noise Inflates Distance

If one string reads st and the other street and abbreviations were not expanded upstream, ratio penalizes the four-character gap. Expand abbreviations during canonical key generation before scoring so the scorer never sees the variance in the first place.

One String Is a Strict Superset

main st versus main st springfield il 62704 scores low on token_sort_ratio because of the extra tokens, but token_set_ratio returns 100 by scoring the shared token set. Switch scorers when comparing records of deliberately different completeness levels.

Integration Note

Fuzzy scoring is the residual pass of the workflow in deduplicating addresses by fuzzy canonical key: the exact key collapses the bulk of duplicates cheaply, then rapidfuzz recovers the typo-level near-misses inside each blocking bucket. Because scoring is only correct on normalized input, it depends on the transforms documented in Unicode and character normalization in Python running first — accented and full-width characters must be folded before any Levenshtein comparison, or the edit distance measures encoding differences rather than real spelling variance.

---

Related

• Deduplicating addresses by fuzzy canonical key — the full deduplication workflow this scoring pass plugs into, including blocking and survivorship.
• Generating canonical address keys in Python — the deterministic normalization that makes fuzzy scores meaningful.
• Unicode and character normalization in Python — fold characters before scoring so edit distance reflects spelling, not encoding.