GitHub Actions Workflow for Geocoding Pipelines

TL;DR: A GitHub Actions workflow for a geocoding pipeline should install dependencies, restore a geocode response cache with actions/cache, run a Python accuracy-regression check against a stored baseline (exiting non-zero on a match-rate or precision drop), matrix the check across providers, and upload the JSON report with actions/upload-artifact. This page is part of automating CI/CD sync for spatial enrichment.

The workflow

The workflow below runs on pull requests against cached fixtures for fast, deterministic feedback, and nightly on a schedule to refresh fixtures from live providers. Provider API keys are read from repository secrets and never printed. The accuracy check is fanned across a provider matrix so a regression in one provider fails only that leg.

name: geocode-accuracy-gate

on:
  pull_request:
    paths:
      - "src/**"
      - "config/**"
      - "fixtures/**"
  schedule:
    - cron: "0 3 * * *"   # nightly full run against live providers
  workflow_dispatch: {}

permissions:
  contents: read

concurrency:
  group: geocode-gate-${{ github.ref }}
  cancel-in-progress: true

jobs:
  accuracy-check:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    strategy:
      fail-fast: false
      matrix:
        provider: [here, mapbox, nominatim]
    env:
      # PR runs stay offline against cached fixtures; scheduled runs go live.
      GEOCODE_MODE: ${{ github.event_name == 'schedule' && 'live' || 'cached' }}
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: pip

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[ci]"

      - name: Restore geocode response cache
        uses: actions/cache@v4
        with:
          path: .geocode_cache
          key: geocode-${{ matrix.provider }}-${{ hashFiles('fixtures/reference_sample.jsonl') }}
          restore-keys: |
            geocode-${{ matrix.provider }}-

      - name: Run accuracy-regression check
        env:
          HERE_API_KEY: ${{ secrets.HERE_API_KEY }}
          MAPBOX_API_KEY: ${{ secrets.MAPBOX_API_KEY }}
        run: |
          python -m pipeline.ci_check \
            --provider "${{ matrix.provider }}" \
            --samples fixtures/reference_sample.jsonl \
            --baseline "fixtures/baseline.${{ matrix.provider }}.json" \
            --report "report.${{ matrix.provider }}.json" \
            --mode "${GEOCODE_MODE}"

      - name: Upload accuracy report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: accuracy-report-${{ matrix.provider }}
          path: report.${{ matrix.provider }}.json
          retention-days: 30

Which Trigger Runs Which Job

One workflow file usually serves three purposes, and conflating them wastes provider quota on every push. Separate the fast checks that run on every commit from the corpus run that gates a merge and the scheduled job that watches for provider drift.

Trigger, scope and provider cost per job Three rows. A push trigger runs linting and unit tests against recorded fixtures with zero provider calls. A pull request trigger runs the gating corpus subset, costing about 900 calls. A scheduled nightly trigger runs the full corpus, costing about 3500 calls and detecting provider drift. Trigger Runs Provider calls on: push lint, unit tests, recorded fixtures none on: pull_request gating corpus subset ~900 · budgeted per PR on: schedule full corpus, drift detection ~3 500 · once a night Concurrency groups matter here: cancel superseded pull-request runs, or a busy branch spends its quota on commits that were replaced minutes later.

The concurrency note is a real budget line. A pull request updated five times in an hour runs the gate five times unless superseded runs are cancelled, and at nine hundred calls apiece that is most of a day’s testing budget spent on commits nobody will ever merge.

Step breakdown

Step / key Purpose Failure behaviour
on.pull_request Fast gate on code, config, or fixture changes Blocks merge if the check exits non-zero
on.schedule (cron) Nightly live-provider run to catch upstream drift Alerts via a failed run; does not block merges
concurrency Cancels superseded runs on the same ref Saves runner minutes and quota
strategy.matrix.provider Isolates each provider’s accuracy check fail-fast: false lets other providers finish
GEOCODE_MODE env Switches between cached fixtures and live calls Cached on PR, live on schedule
setup-python + cache: pip Reproducible interpreter and fast installs Fails early if the version is unavailable
actions/cache (.geocode_cache) Restores recorded provider responses Cache miss falls back to restore-keys prefix
ci_check step Runs the regression gate against the baseline Non-zero exit fails the job
secrets.*_API_KEY Injects provider keys for live runs only Absent secret is an empty string, harmless in cached mode
upload-artifact (if: always()) Persists the report even on failure Report is available for post-mortem and trend charts

Secrets handling for API keys

Store each provider key as a repository or organization secret (Settings → Secrets and variables → Actions) and reference it only through the env block of the step that needs it. Three rules keep keys safe:

  • Never echo a secret. GitHub masks known secret values in logs, but constructing a URL with the key inline can still leak it through error messages. Pass keys as environment variables the Python process reads with os.environ, not as command-line arguments that show up in ps or logs.
  • Scope the token. Set permissions: contents: read at the workflow level so a compromised step cannot push commits or open releases.
  • Guard forks. Secrets are not exposed to workflows triggered by pull requests from forks. Because the pull-request gate runs in cached mode, it needs no keys — the offline fixture cache is exactly what makes the fork case safe.

Secrets That Do Not Leak

API credentials in CI have two failure modes: being printed into a log that is retained for months, and being available to workflows triggered from forks. Both are configuration problems with well-defined answers, and both are worth verifying rather than assuming.

Three rules for API credentials in CI Three rules. Scope credentials to a protected environment with required reviewers so only approved workflows can read them. Pass secrets as environment variables rather than command arguments so they never appear in process listings or command echoes. Ensure fork-triggered pull requests run without access to secrets, using recorded fixtures instead. 1 · scope to a protected environment geocoding keys live in an environment with required reviewers, not in repository-wide secrets 2 · pass as env, never as an argument command arguments appear in process listings and in echoed shell traces; environment variables do not 3 · forks get fixtures, not credentials a fork-triggered run must fall back to recorded responses and say so in its summary

Rule three has a corollary worth designing for: the corpus runner should work without credentials, replaying recorded provider responses. That makes external contributions testable, keeps the unit-test path fast for everyone, and removes the incentive to relax rule one when a contributor’s build fails.

The Python entrypoint it calls

The workflow calls a small argparse wrapper around the regression logic from the parent guide. It selects cached or live mode, runs the check, writes the report, and exits with the check’s code so the job status reflects the result.

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

from pipeline.gate import Sample, check_regression  # from the parent guide
from pipeline.geocoders import build_cached_geocoder, build_live_geocoder


def load_samples(path: Path) -> list[Sample]:
    """Load newline-delimited JSON reference records into Sample objects."""
    samples: list[Sample] = []
    with path.open(encoding="utf-8") as fh:
        for line in fh:
            line = line.strip()
            if not line:
                continue
            record = json.loads(line)
            samples.append(
                Sample(
                    address=record["address"],
                    expected_precision=record["expected_precision"],
                )
            )
    return samples


def main(argv: list[str] | None = None) -> int:
    """Parse args, run the accuracy gate, and return its exit code."""
    parser = argparse.ArgumentParser(description="Geocoding accuracy regression gate")
    parser.add_argument("--provider", required=True)
    parser.add_argument("--samples", type=Path, required=True)
    parser.add_argument("--baseline", type=Path, required=True)
    parser.add_argument("--report", type=Path, required=True)
    parser.add_argument("--mode", choices=("cached", "live"), default="cached")
    args = parser.parse_args(argv)

    if args.mode == "cached":
        geocode = build_cached_geocoder(args.provider, cache_dir=Path(".geocode_cache"))
    else:
        geocode = build_live_geocoder(args.provider, cache_dir=Path(".geocode_cache"))

    samples = load_samples(args.samples)
    code = check_regression(
        samples=samples,
        geocode=geocode,
        baseline_path=args.baseline,
    )

    # Persist a machine-readable report regardless of pass/fail.
    args.report.write_text(
        json.dumps({"provider": args.provider, "mode": args.mode, "exit_code": code}),
        encoding="utf-8",
    )
    return code


if __name__ == "__main__":
    sys.exit(main())

The build_cached_geocoder reads recorded responses keyed by the SHA-256 of the normalized address and raises on a miss, guaranteeing the pull-request gate never touches the network. The build_live_geocoder calls the provider and writes each response into the same cache, so a nightly run populates the fixtures the next pull request will consume.

Vectorized pandas variant

To chart the accuracy trend across providers after the matrix finishes, download the report artifacts and reduce them into one frame:

import json
from pathlib import Path

import pandas as pd


def collect_reports(report_dir: Path) -> pd.DataFrame:
    """Load every report.*.json under a directory into a tidy DataFrame."""
    rows = [json.loads(p.read_text()) for p in report_dir.glob("report.*.json")]
    df = pd.DataFrame(rows)
    df["passed"] = df["exit_code"] == 0
    return df.sort_values("provider").reset_index(drop=True)

Failing the Job on the Right Thing

A gate that fails on any change is noise; a gate that fails on nothing is decoration. The comparison should be against a stored baseline with an explicit tolerance, and the failure message should say which stratum moved and by how much — otherwise the first response to a red build is to re-run it.

Three outcomes of a corpus comparison Three outcomes. A movement inside tolerance passes silently. A drop beyond tolerance fails the job with the stratum name and the delta in the message. An improvement beyond tolerance passes but is flagged, since an unexplained jump often means the corpus or the comparison changed rather than the pipeline. within tolerance → pass record the numbers in the run summary and move on — no notification drop beyond tolerance → fail message names the stratum, the metric, the baseline and the new value — never just "gate failed" improvement beyond tolerance → pass, but flag a large unexplained jump usually means the corpus or the comparison changed, not the pipeline

The third band catches a genuinely common mistake: a change that accidentally narrows the corpus — filtering out the hard rows — reports a large accuracy improvement and passes every gate. Treating an unexplained jump as something to investigate rather than celebrate is what keeps the baseline meaningful.

Edge cases

Cache miss on a fixture change

When fixtures/reference_sample.jsonl changes, the hashFiles cache key changes and the restore falls back to the restore-keys prefix, which returns a stale cache missing the new addresses. In cached mode the geocoder raises on those misses and the job fails clearly. Regenerate fixtures by running the nightly live job (or workflow_dispatch) once after adding sample addresses, then commit the refreshed cache or let the next scheduled run populate it.

A provider outage during the nightly run

The scheduled live leg can fail because a provider is down, not because accuracy regressed. Because fail-fast: false isolates each matrix leg, the other providers still report, and the failed leg is an alert rather than a merge blocker. Treat scheduled failures as tickets and inspect the uploaded report before assuming a real regression — the same reasoning behind routing flaky providers through fallback chains for failed lookups.

Integration note

This workflow is the executable form of the gate described in automating CI/CD sync for spatial enrichment: the YAML decides when the check runs and with what secrets, while the Python entrypoint reuses the baseline-comparison logic from that guide. The baseline and reference sample it consumes come from building regression corpora for geocoding accuracy. Once merges are gated, pair this with a scheduled data refresh so the enriched table stays current, not just correct.