Validating Geocoding Accuracy and Confidence Scoring

Every geocoding result that reaches your spatial database should have survived a validation gate, because a single wrong coordinate silently corrupts downstream joins, route calculations, and territory analytics. This guide, part of Accuracy Validation & CI/CD Sync, explains how to validate geocoding results across three tiers and collapse every provider’s heterogeneous confidence field into one comparable 0-1 score you can threshold consistently — the discipline that turns a pile of raw API responses into a trustworthy dataset.

The problem is that geocoding providers disagree on almost everything: coordinate order, precision vocabulary, and how they express confidence. Google Maps emits a location_type enum (ROOFTOP, RANGE_INTERPOLATED, GEOMETRIC_CENTER, APPROXIMATE); Mapbox emits a relevance float; HERE emits scoring.queryScore plus a resultType; Nominatim emits an importance value on an entirely different scale. Without a normalization layer, you cannot apply one accept/reject rule, and you cannot compare providers when comparing geocoding accuracy across providers. The validator below fixes that.

Prerequisites

The Validation Pipeline

The three tiers run in order, cheapest first, so an impossible coordinate never reaches an expensive point-in-polygon test. Only records that clear all three tiers are scored and committed.

Three-tier geocoding validation and confidence scoring pipeline A raw geocoding result enters a syntactic gate (range and null checks), then a semantic gate (point-in-country and bounding box), then a precision gate (result_type). Surviving results are normalized into a single zero-to-one confidence score, which routes each record to accept, fallback, or reject. Failures at any gate branch to a rejection log. Raw Result Syntactic range · null result count Semantic in-country bbox · distance Precision result_type rooftop ·centroid Unified Score 0-1 normalize Accept Fallback re-route Rejection Log reason codes TIER 1 TIER 2 TIER 3

1. Syntactic validation

The cheapest checks run first: latitude in [-90, 90], longitude in [-180, 180], neither coordinate null or NaN, and at least one result present. The (0.0, 0.0) “null island” coordinate is a special case — it passes a naive range check but almost always signals a parsing failure or a provider default, so treat it as invalid unless your data genuinely includes the Gulf of Guinea.

2. Semantic validation

Syntactically valid coordinates can still be geographically wrong. Semantic validation confirms the point falls where the input says it should: inside the expected country polygon, inside a supplied bounding box, and within a sane distance of a reference centroid. A French address that resolves to Argentina is a provider error, and only semantic checks catch it. For distance-based outlier detection specifically, see detecting geocoding outliers with haversine distance.

3. Precision validation

Finally, compare the provider’s precision field against the minimum your use case requires. A rooftop point and a postal-code centroid can be metres or kilometres apart; a delivery pipeline must reject the centroid where an analytics rollup can accept it. The mapping from each provider’s precision vocabulary to a normalized rank is given in the reference table below.

Provider Precision and Confidence Reference

Precision fields and confidence fields are two different things, and both must be normalized. This table maps each provider’s precision vocabulary to a common rank (higher is sharper) and names the native confidence field to normalize.

Provider Precision field Sharpest → coarsest values Normalized precision rank Native confidence field Confidence range
Google Maps geometry.location_type ROOFTOP → RANGE_INTERPOLATED → GEOMETRIC_CENTER → APPROXIMATE 1.0 → 0.75 → 0.4 → 0.15 none (derive from location_type) n/a
Mapbox place_type[0] address → poi → postcode → place → region 1.0 → 0.7 → 0.5 → 0.3 → 0.1 relevance 0.0–1.0
HERE resultType houseNumber → street → postalCodePoint → locality 1.0 → 0.6 → 0.4 → 0.2 scoring.queryScore 0.0–1.0
Nominatim (OSM) class / type + addresstype building/house → road → postcode → suburb 1.0 → 0.6 → 0.4 → 0.2 importance ~0.0–1.0 (skewed low)
Pelias layer + match_type address+exact → street → postalcode → locality 1.0 → 0.6 → 0.4 → 0.2 confidence 0.0–1.0

Nominatim’s importance is the trap: it reflects OSM prominence, not match quality, and clusters low (a correct house match may score 0.3). Rescale it rather than thresholding it directly.

The Validator and Confidence Scorer

The row-level implementation below runs the three tiers, then computes a unified 0-1 confidence by blending the normalized provider score with the precision rank. It returns a structured verdict — accept, fallback, or reject — plus a reason code that goes straight into the rejection log.

from __future__ import annotations

import logging
import math
from dataclasses import dataclass, field
from enum import Enum
from typing import Optional

logger = logging.getLogger("geocode.validation")

# Normalized precision rank per provider result_type (higher = sharper).
_PRECISION_RANK: dict[str, dict[str, float]] = {
    "google": {"ROOFTOP": 1.0, "RANGE_INTERPOLATED": 0.75,
               "GEOMETRIC_CENTER": 0.4, "APPROXIMATE": 0.15},
    "mapbox": {"address": 1.0, "poi": 0.7, "postcode": 0.5,
               "place": 0.3, "region": 0.1},
    "here": {"houseNumber": 1.0, "street": 0.6,
             "postalCodePoint": 0.4, "locality": 0.2},
    "nominatim": {"house": 1.0, "building": 1.0, "road": 0.6,
                  "postcode": 0.4, "suburb": 0.2},
    "pelias": {"address": 1.0, "street": 0.6, "postalcode": 0.4,
               "locality": 0.2},
}

# Weight split between the normalized provider score and precision rank.
_W_PROVIDER = 0.45
_W_PRECISION = 0.55

ACCEPT_THRESHOLD = 0.85
REJECT_THRESHOLD = 0.55


class Verdict(str, Enum):
    ACCEPT = "accept"
    FALLBACK = "fallback"
    REJECT = "reject"


@dataclass
class GeocodeResult:
    """A single provider result normalized into common fields."""
    provider: str
    lat: Optional[float]
    lon: Optional[float]
    result_type: Optional[str]
    raw_confidence: Optional[float]   # provider-native 0-1, or None
    result_count: int = 1


@dataclass
class Validation:
    verdict: Verdict
    score: float
    reason: str
    tier: str
    extras: dict = field(default_factory=dict)


def _normalize_provider_confidence(r: GeocodeResult) -> float:
    """Map a provider's native confidence onto a calibrated 0-1 float."""
    if r.raw_confidence is None:
        # Google exposes no numeric score; fall back to precision rank.
        return _precision_rank(r)
    c = max(0.0, min(1.0, float(r.raw_confidence)))
    if r.provider == "nominatim":
        # importance clusters low; rescale so a typical match lands mid-range.
        return min(1.0, c / 0.6)
    return c


def _precision_rank(r: GeocodeResult) -> float:
    ranks = _PRECISION_RANK.get(r.provider, {})
    return ranks.get(r.result_type or "", 0.0)


def unified_confidence(r: GeocodeResult) -> float:
    """Blend normalized provider score and precision rank into one 0-1 score."""
    provider_score = _normalize_provider_confidence(r)
    precision = _precision_rank(r)
    score = _W_PROVIDER * provider_score + _W_PRECISION * precision
    return round(max(0.0, min(1.0, score)), 4)


def validate(
    r: GeocodeResult,
    country_bbox: Optional[tuple[float, float, float, float]] = None,
    min_precision_rank: float = 0.6,
) -> Validation:
    """Run the three validation tiers and return a routed verdict.

    Args:
        r:                 Normalized provider result.
        country_bbox:      (min_lon, min_lat, max_lon, max_lat) for the expected
                           region, or None to skip the semantic bbox check.
        min_precision_rank: Minimum acceptable precision rank (0.6 ~ street).

    Returns:
        A Validation with verdict, unified score, and a reason code.
    """
    # --- Tier 1: syntactic ---
    if r.result_count < 1:
        return Validation(Verdict.FALLBACK, 0.0, "empty_result_set", "syntactic")
    if r.lat is None or r.lon is None or math.isnan(r.lat) or math.isnan(r.lon):
        return Validation(Verdict.REJECT, 0.0, "null_coordinate", "syntactic")
    if not (-90.0 <= r.lat <= 90.0) or not (-180.0 <= r.lon <= 180.0):
        return Validation(Verdict.REJECT, 0.0, "coord_out_of_range", "syntactic")
    if r.lat == 0.0 and r.lon == 0.0:
        return Validation(Verdict.REJECT, 0.0, "null_island", "syntactic")

    # --- Tier 2: semantic ---
    if country_bbox is not None:
        min_lon, min_lat, max_lon, max_lat = country_bbox
        if not (min_lon <= r.lon <= max_lon and min_lat <= r.lat <= max_lat):
            return Validation(Verdict.REJECT, 0.0, "outside_expected_bbox",
                              "semantic")

    # --- Tier 3: precision ---
    score = unified_confidence(r)
    if _precision_rank(r) < min_precision_rank:
        return Validation(Verdict.FALLBACK, score, "precision_below_min",
                          "precision")

    # --- Route by unified score ---
    if score >= ACCEPT_THRESHOLD:
        return Validation(Verdict.ACCEPT, score, "ok", "scored")
    if score >= REJECT_THRESHOLD:
        return Validation(Verdict.FALLBACK, score, "low_confidence", "scored")
    return Validation(Verdict.REJECT, score, "below_reject_threshold", "scored")


def log_rejection(r: GeocodeResult, v: Validation) -> None:
    """Emit a structured rejection record for audit and regression capture."""
    logger.warning(
        "geocode_rejected",
        extra={
            "provider": r.provider,
            "lat": r.lat,
            "lon": r.lon,
            "result_type": r.result_type,
            "raw_confidence": r.raw_confidence,
            "unified_score": v.score,
            "tier": v.tier,
            "reason": v.reason,
        },
    )

Call it per record and act on the verdict, logging anything that is not accepted:

result = GeocodeResult(
    provider="google", lat=48.8584, lon=2.2945,
    result_type="ROOFTOP", raw_confidence=None,
)
# France bbox roughly (min_lon, min_lat, max_lon, max_lat)
verdict = validate(result, country_bbox=(-5.2, 41.3, 9.6, 51.1))
if verdict.verdict is not Verdict.ACCEPT:
    log_rejection(result, verdict)

Vectorized pandas Variant

Row-at-a-time validation is fine for streaming, but a nightly batch of millions of records needs a vectorized pass. The version below scores an entire DataFrame with boolean masks and numpy.select, keeping the same tiers and thresholds.

import numpy as np
import pandas as pd


def validate_frame(
    df: pd.DataFrame,
    provider_col: str = "provider",
    lat_col: str = "lat",
    lon_col: str = "lon",
    type_col: str = "result_type",
    conf_col: str = "raw_confidence",
    min_precision_rank: float = 0.6,
) -> pd.DataFrame:
    """Validate and score a DataFrame of geocoding results in one vectorized pass.

    Adds 'unified_score', 'verdict', and 'reason' columns. Assumes a single
    country bbox is applied per-row via optional bbox_* columns when present.
    """
    out = df.copy()

    # Precision rank lookup, vectorized via a mapping Series per provider.
    def _rank_row(prov: str, rtype: object) -> float:
        return _PRECISION_RANK.get(prov, {}).get(str(rtype), 0.0)

    precision = np.array([
        _rank_row(p, t) for p, t in zip(out[provider_col], out[type_col])
    ])

    raw = pd.to_numeric(out[conf_col], errors="coerce")
    prov_norm = raw.clip(0.0, 1.0)
    is_nom = out[provider_col].eq("nominatim")
    prov_norm = prov_norm.mask(is_nom, (prov_norm / 0.6).clip(upper=1.0))
    # Where no numeric confidence exists, fall back to the precision rank.
    prov_norm = prov_norm.fillna(pd.Series(precision, index=out.index))

    score = (_W_PROVIDER * prov_norm + _W_PRECISION * precision).clip(0.0, 1.0)
    out["unified_score"] = score.round(4)

    lat = pd.to_numeric(out[lat_col], errors="coerce")
    lon = pd.to_numeric(out[lon_col], errors="coerce")

    bad_null = lat.isna() | lon.isna()
    bad_range = ~lat.between(-90, 90) | ~lon.between(-180, 180)
    null_island = (lat == 0.0) & (lon == 0.0)
    low_precision = precision < min_precision_rank

    conditions = [
        bad_null | bad_range | null_island,
        low_precision,
        score >= ACCEPT_THRESHOLD,
        score >= REJECT_THRESHOLD,
    ]
    verdicts = ["reject", "fallback", "accept", "fallback"]
    reasons = ["syntactic_fail", "precision_below_min", "ok", "low_confidence"]

    out["verdict"] = np.select(conditions, verdicts, default="reject")
    out["reason"] = np.select(conditions, reasons, default="below_reject_threshold")
    return out

The vectorized pass omits the point-in-polygon country test because shapely operations do not vectorize with numpy.select; run bounding-box masks inline as shown and reserve full polygon containment for the smaller set of records that pass everything else.

Edge Cases

Provider returns 200 with an empty result set

A coverage miss frequently arrives as HTTP 200 with results: [] rather than a 404. The syntactic tier catches this via result_count < 1 and routes to a fallback with reason empty_result_set. If you read coordinates before checking the count, you commit None or a provider default and corrupt the row. This is the single most common silent failure — the same pattern the fallback chain orchestrator must guard against before cascading.

Plausible-but-wrong coordinates

A valid-looking point in the wrong country passes every range check. Only the semantic bounding-box test — and, for finer resolution, a haversine distance from a postal centroid — flags it. Always run at least a country bbox check when the input carries a country code.

Confidence ties at the threshold

When two candidate results score identically at the boundary (say both 0.85), break the tie deterministically: prefer the sharper result_type, then the provider with the higher historical accuracy for that region, then a stable provider ordering. Never break ties randomly — nondeterministic acceptance makes regressions impossible to reproduce.

Nominatim importance underflow

Because importance clusters low, a correct match can score 0.3 raw. The _normalize_provider_confidence rescale (c / 0.6) lifts a typical good match into a usable range; without it, every Nominatim result would fall to the reject tier.

Performance and Vectorization

Approach Throughput (approx.) When to use
Row-level validate() 30k–80k records/sec Streaming ingestion, per-record fallback decisions
validate_frame() bbox only 2M–5M records/sec Nightly batch validation over a single region
Vectorized + shapely polygon on survivors 50k–200k records/sec When precise country containment matters
Full haversine outlier pass (numpy) 3M+ pairs/sec Distance-to-reference screening in bulk

The dominant cost is point-in-polygon containment, not scoring. Filter with bounding boxes first so shapely only runs on the fraction of records that survive the cheap tiers.

Troubleshooting

Every result from one provider lands in the fallback tier

The provider’s confidence field is not being read into raw_confidence, so the scorer falls back to precision rank alone and never clears the accept threshold. Confirm the parse step maps the correct native field (relevance, queryScore, importance) before scoring.

Accept rate collapses after a provider schema change

Providers rename precision values without notice. If _PRECISION_RANK has no entry for a returned result_type, its rank is 0.0 and everything drops to reject. Log unknown result_type values and alert when the unknown rate exceeds a small threshold.

Semantic check rejects valid border addresses

A hard country bounding box clips legitimate points just across an administrative line. Buffer the bbox by a few kilometres, or switch to polygon containment with a small tolerance for records near borders.

Unified scores cluster at exactly two values

This means one input to the blend is constant — usually raw_confidence is missing for the whole batch, so every score reduces to _W_PRECISION * precision. Verify the confidence column is populated and numeric.

FAQ

Why not just trust the provider’s own confidence field?

Provider scores are not comparable across vendors and are not calibrated to your use case. Google returns a location_type enum, Mapbox a 0-1 relevance float, HERE a queryScore, and Nominatim an importance value on a different scale. A raw 0.8 from one provider does not mean the same thing as 0.8 from another. Normalizing every provider onto one scale and layering your own syntactic, semantic, and precision checks on top is the only way to apply a single threshold consistently.

What threshold should I use to accept a geocode automatically?

For delivery, routing, and rooftop-dependent workloads, accept only unified scores at or above 0.85 with a rooftop or range-interpolation result_type. Route scores between roughly 0.55 and 0.85 to a fallback provider or manual review, and reject anything below 0.55 or failing semantic validation outright. Aggregate analytics that tolerate postal-centroid precision can lower the accept threshold to around 0.6.

How do I handle a provider that returns HTTP 200 with an empty result set?

Treat an empty results array as a coverage miss, not a success. Check the result count in the syntactic tier before reading any coordinates, assign a zero confidence, and route the record to a fallback rather than committing null coordinates. Committing a 200-with-no-results as a valid geocode is one of the most common silent data-corruption bugs in production pipelines.

How do I detect plausible-but-wrong coordinates that pass range checks?

Range checks only catch impossible coordinates. Plausible-but-wrong results, such as a valid point in the wrong country, need semantic validation: test the point against the expected country polygon or bounding box, and measure the haversine distance from a known reference such as a postal centroid or a prior stored value. A result that lands 40 km from the postal centroid it claims to belong to is almost certainly wrong even though its latitude and longitude are individually valid.

Should rejected geocodes be logged, and what should the log contain?

Always log rejections. Each rejection record should carry the normalized input, the provider, the raw provider score, the computed unified score, the failing tier and reason code, the returned coordinates, and a timestamp. This audit trail is what lets you distinguish a normalization bug from a genuine coverage gap and feeds directly into a regression corpus so the same failure is caught automatically next time.