David Zhang

Same place or not? What breaks address-matching models

November 2026 · 7 min read

Is 12 ELM ST the same place as 12 ELM STREET? Obviously. What about 431 HALLETT ST and 431 HARRIET ST? Probably not, but a model that only looks at how similar the letters are will say yes. And what about 74 BEACH RD and BEACH RD?

That last one turned out to be the most interesting question in this whole project.

This post walks through what I built and what I found. The full write-up, with every table, is the paper (PDF), and the code, trained models and dataset are all public (links at the end).

The problem

Geocoding means turning an address into a point on a map. Somewhere in that process there’s a step where you have an address someone typed and a list of candidate addresses from a reference database, usually the ones in the same ZIP code. You have to decide which candidate, if any, is the same place. That final yes/no comparison is address matching, and it’s what this project is about.

There’s a fair amount of work on it, from hand-tuned string-similarity rules to deep learning. Lin et al. (2020), for example, report an F1 of 0.97 with a neural model on Chinese addresses. But I kept running into the same issue when reading these papers: every paper evaluates on its own data, with one headline number. You can’t compare them, and a single number hides which mistakes a model makes. A model can score 98% accuracy while failing badly at the one case you actually care about, because the easy cases swamp the average.

So I set out to put very different kinds of models on the same pairs and score them by the type of pair.

The data

I used two open datasets that describe the same real-world places: Overture Maps and Foursquare Open Source Places. Overture publishes “bridge files” that say which Foursquare record each Overture place was matched to. So for every matched place in Connecticut I get two versions of its address, written by two different sources. That’s a positive pair, labelled “same place”.

A bit of cleaning was needed first. I matched at the building level, so suite and floor numbers get stripped (7 BACKUS AVE STE 101 becomes 7 BACKUS AVE). That creates lots of duplicates: one mall address appeared 77 times. I kept one copy of each, leaving 28,834 true matches.

Then the negatives, the pairs that are not the same place. Random pairs are too easy, so I built several kinds:

Pair type Example Same place?
Identical match 7 BACKUS AVE / 7 BACKUS AVE yes
Non-identical match 12 ELM ST / 12 ELM STREET yes
Case 1: same street, other number 12 ELM ST / 14 ELM ST no
Case 2s: look-alike street 431 HALLETT ST / 431 HARRIET ST no
Case 3: random two random addresses no

The look-alike streets (case 2s) were the hard part to get. Real examples are rare: only 15 in my whole test set. So I made them: take a real address, keep its house number, and swap in a different real street from the same ZIP code whose name looks similar. The non-identical matches and the look-alike streets are the two hard pair types. Every model gets the easy ones right, so they’re where the models differ.

Two more design choices mattered. There are nine non-matches for every match, which mimics a model having to reject many wrong candidates. And the train/validation/test split is by ZIP code: whole neighbourhoods go to one split, so the test set contains only areas no model has seen.

The models

I compared two very different approaches.

Classical models get the address pre-chopped into pieces: house number, street and ZIP code. For each piece I compute string-similarity scores (Jaro-Winkler, Levenshtein, character-overlap measures and so on), 23 features in total, and train logistic regression, a random forest and XGBoost on them.

Neural models get only the two raw strings and have to figure out the rest:

  • a siamese BiLSTM, which reads each address character by character, squashes it into one vector, and compares the two vectors at the very end;
  • ESIM, which reads word by word and lets each word of one address “attend” to the words of the other partway through;
  • DistilBERT as a bi-encoder, a pretrained transformer used like the siamese model: encode each address separately, compare at the end;
  • DistilBERT as a cross-encoder, where both addresses go in as one input, so every token can look at every token of the other address from the first layer.
Diagram of the three neural set-ups: bi-encoder, ESIM, cross-encoder
The neural set-ups differ in where the two addresses first meet (orange).

How I scored them

Two kinds of numbers:

  1. Per pair type at a 0.5 threshold: what share of true matches did the model find, and what share of each kind of non-match did it reject?
  2. AUC, which ignores the threshold. Take a random true (non-identical) match and a random look-alike-street fake. What’s the chance the model gives the true match the higher score? 0.5 is a coin flip; 1.0 is perfect.

The second one turned out to be essential. Two models can look very different at 0.5 just because their scores are shifted, while ranking the pairs equally well.

One number per modelChance that a random true (non-identical) match scores above a random look-alike-street fake. Test set; neural ranges span three seeds.
0.8 0.82 0.84 0.86 0.88 0.9 0.92 Random forest / XGBoost Random forest / XGBoost: AUC 0.902 DistilBERT cross-encoder DistilBERT cross-encoder: AUC 0.877–0.888 Siamese BiLSTM Siamese BiLSTM: AUC 0.854 DistilBERT bi-encoder DistilBERT bi-encoder: AUC 0.836–0.845 ESIM ESIM: AUC 0.815–0.842 AUC: non-identical matches vs look-alike streets (axis starts at 0.80)
Show as a table
ModelAUC
Random forest / XGBoost0.902
DistilBERT cross-encoder0.877–0.888
Siamese BiLSTM0.854
DistilBERT bi-encoder0.836–0.845
ESIM0.815–0.842

What I found

1. Everyone handles the easy stuff. Identical matches, a different house number on the same street, random pairs: every model gets about 98–100% of these right. If I’d reported overall accuracy, the models would look nearly identical.

2. The 0.5 threshold is misleading on its own. The DistilBERT bi-encoder finds 82–84% of the non-identical matches but rejects only 58–63% of the look-alike streets. The cross-encoder is the opposite: 70–73% found, 82–93% rejected. They look like completely different models. But their ranking ability (AUC) is much closer, and the gap between them has a very specific source.

3. That source is the house number. Here’s the key chart. It splits the true matches by what happened to the house number, and asks how well each model ranks them above the look-alike fakes:

Where the models differ: the house numberHow well each model ranks true non-identical matches above look-alike-street fakes, split by what happened to the house number. Test set; bars span three seeds.
Random forestCross-encoderBi-encoderESIM
Same house number on both sides (638 matches) Random forest Random forest, same house number on both sides: AUC 0.923 (1 run) Cross-encoder Cross-encoder, same house number on both sides: AUC 0.922–0.927 (3 seeds, min–max) Bi-encoder Bi-encoder, same house number on both sides: AUC 0.925–0.934 (3 seeds, min–max) ESIM ESIM, same house number on both sides: AUC 0.880–0.906 (3 seeds, min–max) House number missing on one side (164 matches) Random forest Random forest, house number missing on one side: AUC 0.845 (1 run) Cross-encoder Cross-encoder, house number missing on one side: AUC 0.689–0.729 (3 seeds, min–max) Bi-encoder Bi-encoder, house number missing on one side: AUC 0.448–0.497 (3 seeds, min–max) ESIM ESIM, house number missing on one side: AUC 0.547–0.667 (3 seeds, min–max) Two different house numbers (41 matches) Random forest Random forest, two different house numbers: AUC 0.798 (1 run) Cross-encoder Cross-encoder, two different house numbers: AUC 0.928–0.940 (3 seeds, min–max) Bi-encoder Bi-encoder, two different house numbers: AUC 0.949–0.963 (3 seeds, min–max) ESIM ESIM, two different house numbers: AUC 0.808–0.853 (3 seeds, min–max) 0.4 0.5 (chance) 0.6 0.7 0.8 0.9 1 AUC: true matches vs look-alike streets (0.5 = coin flip)
Show as a table
GroupnRandom forestCross-encoderBi-encoderESIM
Same house number on both sides6380.9230.922–0.9270.925–0.9340.880–0.906
House number missing on one side1640.8450.689–0.7290.448–0.4970.547–0.667
Two different house numbers410.7980.928–0.9400.949–0.9630.808–0.853
  • When both sides have the same house number, every model except ESIM is around 0.92–0.93. The transformers are just as good as the random forest at telling a typo’d street (ELM STRET) from a different street that looks alike.
  • When the house number is missing from one side (641 SCOFIELDTOWN RD / SCOFIELDTOWN RD), everything falls apart for the neural models. The random forest still manages 0.85. The cross-encoder drops to about 0.7. The bi-encoder is at or below chance: it scores those real matches lower than the fake streets.

So the transformers aren’t worse at streets. The tree models’ overall lead comes from this one group of 164 pairs. That makes sense: the classical models get the house number as its own field, plus a flag that says “one side has no number”. The neural models just see a string with a number missing, and treat that as a big difference.

4. Part of that group can’t be solved at all. Here’s where 74 BEACH RD / BEACH RD comes back. While building the case 1 negatives (same street, different number), I made a mistake: I checked that both places had a house number using their Overture addresses, but the pair actually uses the Foursquare address for one side, which sometimes leaves the number off. So 344 of the test non-matches look exactly like 74 BEACH RD / BEACH RD: two genuinely different businesses on the same road, one without its number.

Those pairs are labelled correctly (they really are different places), but they have the same text pattern as some true matches. No model can tell them apart from the text alone. Every model I scored is close to chance on them (AUC 0.50–0.64). Changing a model’s design only moves errors from one side to the other:

Same text, opposite labelsRandom forest on test pairs where exactly one address has a house number. Changing how the missing number is encoded moves errors from one row to the other.
Sentinel: missing number marked “not comparable” (−1)Zero-fill: missing number read as 0
0 25 50 75 100 True matches found n = 164 Sentinel (−1): 27.4% (true matches found)27.4% Zero-fill (0): 11.6% (true matches found)11.6% Same-street non-matches rejected n = 344 Sentinel (−1): 82.0% (same-street non-matches rejected)82.0% Zero-fill (0): 95.9% (same-street non-matches rejected)95.9% % correct
Show as a table
Pairs with one number missingnSentinelZero-fill
True matches found16427.4%11.6%
Same-street non-matches rejected34482.0%95.9%

I kept the “sentinel” version: when one side simply lacks a number and everything else agrees, nothing in the text says these are different places, so leaning towards “match” seemed more defensible.

A detour: ESIM and the invisible house numbers

One fun bug-hunt. ESIM, following the original paper, uses word2vec vectors and drops words seen only once. House numbers are mostly seen once, so they all become the same “unknown” token. That makes 2147 SUMMER ST / 1867 SUMMER ST literally identical to the model, with the opposite label to 2147 SUMMER ST / 2147 SUMMER ST. Each training run just picked one answer for the whole group. Giving every rare word its own fixed random vector fixed it, because now the model can at least tell “same token” from “different token”. Characters (siamese) and sub-word pieces (DistilBERT) never had this problem.

What I’d do next

  • Give the neural models the house number separately, or at least a “number missing” signal, like the classical models have. The data says that’s exactly where they lose.
  • Fix the case 1 construction so the ambiguous pairs aren’t in the data at all.
  • Test it in a retrieval setting: real geocoding ranks many candidates, not one fixed pair.
  • Go beyond Connecticut.

The main lesson for me: score by the type of mistake. A single number would have said “trees beat transformers” and stopped there. Splitting by pair type showed the transformers handle streets just as well, and that the whole gap is one specific, fixable thing.

Everything is public