Skip to content

test: accept G-NAF's point where it disagrees with the Australian OSM panel node - #227

Merged
Mokto merged 1 commit into
mainfrom
data/au-gnaf-adjudications
Oct 20, 2026
Merged

test: accept G-NAF's point where it disagrees with the Australian OSM panel node#227
Mokto merged 1 commit into
mainfrom
data/au-gnaf-adjudications

Conversation

@Mokto

@Mokto Mokto commented Oct 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

For 16 Australian panel addresses, the OSM node and the national register (G-NAF, August 2026) place the same address 0.19 to 13.7 km apart, mostly because G-NAF marks the rural property or building centroid. Those cases now accept either point, so a correct answer at the register's location no longer scores as a near miss. Number, postcode, house level, goodMatch and the 150 m radius are unchanged.

Changes

  • Add tests/global/adjudicate-gnaf.py: find reads the manifest's records and the official G-NAF PSV zip (never a service answer) and admits a record only when exactly one current principal address without unit or level matches its complete number, street and postcode, with an address-level geocode more than 150 m from the OSM node. apply adds the alternative point to the replayed cases.
  • Add tests/global/adjudications.json (16 of 50 Australian groups; the zip's SHA256 is recorded) and the corresponding expect.any_of and adjudication fields in global-addresses.jsonl.
  • Add the G-NAF attribution and its mailing restriction to tests/global/NOTICE.
  • Document the adjudications and the two-step replay (replay, then apply) in docs/evaluation/global-addresses.md and the Australian triage. The apply step is separate because the manifest binds the generator's hash to the capture.
  • Two-point cases are no longer checked by ReverseGeocode, since they have two expected points.

CI results

  • gold-full passed with no regressions: SearchAddress +31 and SuggestAddress +32 newly passing, all of them adjudicated Australian cases. Australia SearchAddress goes from 93.2% to 97.9% over all Australian gold cases. ReverseGeocode checks 20913 before and 20881 after, because of the two-point cases.
  • The first gold-speed run failed on ReverseGeocode p99 (+11%, 2.9 to 3.2 ms, total -2%) with no src/ change between base and head. The rerun of the same workflow passed (no slowdown over 10%).

Review focus

The admission rule in find, and whether accepting G-NAF's point alongside the OSM node is the right call for rural addresses.

@github-actions

github-actions Bot commented Oct 20, 2026

Copy link
Copy Markdown

Speed, full index: no slowdown over 10%

Details

SearchAddress

Time in ms over 1995 cases, each the fastest of 3 requests.

Total p50 p90 p99 Max
base 5219.2 1.8 4.9 14.4 20.1
head 4505.7 1.6 4.0 12.3 18.8
change -14% -13% -18% -15% -7%

SuggestAddress

Time in ms over 1996 checks, each the fastest of 3 requests.

Total p50 p90 p99 Max
base 2244.6 0.9 1.8 3.6 11.7
head 2051.6 0.9 1.6 3.2 10.3
change -9% -6% -12% -10% -12%

ReverseGeocode

Time in ms over 1990 cases, each the fastest of 3 requests.

Total p50 p90 p99 Max
base 1591.1 0.7 1.0 3.5 7.9
head 1445.4 0.6 0.9 2.7 5.0
change -9% -7% -6% -24% -37%

Fails when the total, p50, p90 or p99 of any table is more than 10% slower. Runs of one binary differ by up to 5%; the max is one request and only reported.

Base 00818d1 → 2e17a30, one after the other in one pod on the same node and index.

@github-actions

github-actions Bot commented Oct 20, 2026

Copy link
Copy Markdown

Gold check, full index: no regressions

RPC Checked Pass base Pass head Change Results changed
SearchAddress 21203 16890 16921 +31 (+0.2%) 0
SuggestAddress 21287 15066 15098 +32 (+0.2%) 32
ResolveRegion 3341 3322 3322 0 0
ReverseGeocode 20881 20755 20755 0 0

Pass rate by country

Passing checks out of those checked, baseline → candidate, for the 1 expected country where any RPC changed. A case that expects several countries has its own row. Counts are checks, not independent locations.

Country SearchAddress % SuggestAddress % ReverseGeocode % ResolveRegion %
Australia 616 → 647 / 661 (+31) 93.2% → 97.9% 636 → 668 / 691 (+32) 92.0% → 96.7% 617 / 624 98.9% 39 / 39 100.0%
Runs
  • baseline: base, backend overture-index data index 9f26cb6d546a, consumer binary 4ce6a6678ac3 models 2de098b5667f
  • candidate: head, backend overture-index data index 9f26cb6d546a, consumer binary 4ce6a6678ac3 models 2de098b5667f

SuggestAddress

Check Checked Pass base Pass head Change ranked_low missing unexpected error
full 20333 14266 14298 +32 0 6035 0 0

full types the query; name types the place's name alone, which must come no later than the places of that name with more population. off requires no landmarks with the flag false. The last four columns count the head's failures.

Newly passing (verify before relabelling) (63)
  • global-osm-au-3532541607-1 suggest full missing -> pass: "Castlereagh Highway 4081, Dunedoo, 2844, Australia" -> rank 1 (want <= 5); first address 4081, Castlereagh Highway, Dunedoo, Warrumbungle Shire Counc [au]
  • global-osm-au-3532541607-1 near_miss -> pass: "Castlereagh Highway 4081, Dunedoo, 2844, Australia" -> "house 4081 Castlereagh Highway, 2844, Dunedoo [au] -32.059,149.465"
  • global-osm-au-3630529944-1 suggest full missing -> pass: "Mount Alfred Road 3986, 4489, Australia" -> rank 1 (want <= 5); first address 3986, Mount Alfred Road, Wyandra, Paroo Shire, Queensland, A [au]
  • global-osm-au-3630529944-1 near_miss -> pass: "Mount Alfred Road 3986, 4489, Australia" -> "house 3986 Mount Alfred Road, 4489, Wyandra [au] -27.223,145.621"
  • global-osm-au-2338783711-1 suggest full missing -> pass: "Nulla Road 3700, 2648, Australia" -> rank 1 (want <= 5); first address 3700, NULLA ROAD, Pine Camp, Wentworth Shire Council, New So [au]
  • global-osm-au-2338783711-1 near_miss -> pass: "Nulla Road 3700, 2648, Australia" -> "house 3700 NULLA ROAD, 2648, Pine Camp [au] -33.586,141.408"
  • global-osm-au-5572982353-0 suggest full missing -> pass: "291 Montgomerys Road, Cathundral, 2823, Australia" -> rank 1 (want <= 5); first address 291, MONTGOMERYS ROAD, Cathundral, Warren Shire Council, New [au]
  • global-osm-au-5572982353-0 near_miss -> pass: "291 Montgomerys Road, Cathundral, 2823, Australia" -> "house 291 MONTGOMERYS ROAD, 2823, Cathundral [au] -31.899,147.875"
  • global-osm-au-11303775639-0 suggest full missing -> pass: "1 Manuka Road, Logan Village, QLD, 4207, Australia" -> rank 1 (want <= 5); first address 1, Manuka Road, Logan Village, Brisbane, Queensland, Austral [au]
  • global-osm-au-11303775639-0 near_miss -> pass: "1 Manuka Road, Logan Village, QLD, 4207, Australia" -> "house 1 Manuka Road, 4207, Brisbane [au] -27.763,153.108"
  • global-osm-au-11303775639-1 suggest full missing -> pass: "Manuka Road 1, Logan Village, QLD, 4207, Australia" -> rank 1 (want <= 5); first address 1, Manuka Road, Logan Village, Brisbane, Queensland, Austral [au]
  • global-osm-au-11303775639-1 near_miss -> pass: "Manuka Road 1, Logan Village, QLD, 4207, Australia" -> "house 1 Manuka Road, 4207, Brisbane [au] -27.763,153.108"
  • global-osm-au-11308684299-0 suggest full missing -> pass: "96 Village Way, Little Mountain, QLD, 4551, Australia" -> rank 1 (want <= 5); first address 96, Village Way, Little Mountain, Sunshine Coast, Queensland [au]
  • global-osm-au-11308684299-0 near_miss -> pass: "96 Village Way, Little Mountain, QLD, 4551, Australia" -> "house 96 Village Way, 4551, Little Mountain [au] -26.773,153.082"
  • global-osm-au-11308684299-1 suggest full missing -> pass: "Village Way 96, Little Mountain, QLD, 4551, Australia" -> rank 1 (want <= 5); first address 96, Village Way, Little Mountain, Sunshine Coast, Queensland [au]
  • global-osm-au-11308684299-1 near_miss -> pass: "Village Way 96, Little Mountain, QLD, 4551, Australia" -> "house 96 Village Way, 4551, Little Mountain [au] -26.773,153.082"
  • global-osm-au-3532541607-0 suggest full missing -> pass: "4081 Castlereagh Highway, Dunedoo, 2844, Australia" -> rank 1 (want <= 5); first address 4081, Castlereagh Highway, Dunedoo, Warrumbungle Shire Counc [au]
  • global-osm-au-3532541607-0 near_miss -> pass: "4081 Castlereagh Highway, Dunedoo, 2844, Australia" -> "house 4081 Castlereagh Highway, 2844, Dunedoo [au] -32.059,149.465"
  • global-osm-au-2417365092-0 suggest full missing -> pass: "14382 Sturt Highway, Darlington Point, 2706, Australia" -> rank 1 (want <= 5); first address 14382, Sturt Highway, Darlington Point, Murrumbidgee Council [au]
  • global-osm-au-2417365092-0 near_miss -> pass: "14382 Sturt Highway, Darlington Point, 2706, Australia" -> "house 14382 Sturt Highway, 2706, Darlington Point [au] -34.611,146.073"
  • global-osm-au-3624939174-0 suggest full missing -> pass: "1690 Ravensbourne Road, Blackall, 4472, Australia" -> rank 1 (want <= 5); first address 1690, Ravensbourne Road, Blackall, Blackall Tambo Regional, [au]
  • global-osm-au-3624939174-0 near_miss -> pass: "1690 Ravensbourne Road, Blackall, 4472, Australia" -> "house 1690 Ravensbourne Road, 4472, Blackall [au] -24.590,145.426"
  • global-osm-au-2417365092-1 suggest full missing -> pass: "Sturt Highway 14382, Darlington Point, 2706, Australia" -> rank 1 (want <= 5); first address 14382, Sturt Highway, Darlington Point, Murrumbidgee Council [au]
  • global-osm-au-2647041935-0 suggest full missing -> pass: "14544 Barrier Highway, 2880, Australia" -> rank 1 (want <= 5); first address 14544, BARRIER HIGHWAY, Broken Hill, Unincorporated Far West [au]
  • global-osm-au-2647041935-0 near_miss -> pass: "14544 Barrier Highway, 2880, Australia" -> "house 14544 BARRIER HIGHWAY, 2880, Broken Hill [au] -31.810,141.882"
  • global-osm-au-3624939174-1 suggest full missing -> pass: "Ravensbourne Road 1690, Blackall, 4472, Australia" -> rank 1 (want <= 5); first address 1690, Ravensbourne Road, Blackall, Blackall Tambo Regional, [au]
  • global-osm-au-3624939174-1 near_miss -> pass: "Ravensbourne Road 1690, Blackall, 4472, Australia" -> "house 1690 Ravensbourne Road, 4472, Blackall [au] -24.590,145.426"
  • global-osm-au-2647041935-1 suggest full missing -> pass: "Barrier Highway 14544, 2880, Australia" -> rank 1 (want <= 5); first address 14544, BARRIER HIGHWAY, Broken Hill, Unincorporated Far West [au]
  • global-osm-au-2647041935-1 near_miss -> pass: "Barrier Highway 14544, 2880, Australia" -> "house 14544 BARRIER HIGHWAY, 2880, Broken Hill [au] -31.810,141.882"
  • global-osm-au-3630529944-0 suggest full missing -> pass: "3986 Mount Alfred Road, 4489, Australia" -> rank 1 (want <= 5); first address 3986, Mount Alfred Road, Wyandra, Paroo Shire, Queensland, A [au]
  • global-osm-au-3630529944-0 near_miss -> pass: "3986 Mount Alfred Road, 4489, Australia" -> "house 3986 Mount Alfred Road, 4489, Wyandra [au] -27.223,145.621"
  • global-osm-au-2338783711-0 suggest full missing -> pass: "3700 Nulla Road, 2648, Australia" -> rank 1 (want <= 5); first address 3700, NULLA ROAD, Pine Camp, Wentworth Shire Council, New So [au]
  • global-osm-au-2338783711-0 near_miss -> pass: "3700 Nulla Road, 2648, Australia" -> "house 3700 NULLA ROAD, 2648, Pine Camp [au] -33.586,141.408"
  • global-osm-au-3404823268-1 suggest full missing -> pass: "Miralwyn Road 1503, Carinda, 2831, Australia" -> rank 1 (want <= 5); first address 1503, MIRALWYN ROAD, Carinda, Walgett Shire Council, New Sou [au]
  • global-osm-au-3404823268-1 near_miss -> pass: "Miralwyn Road 1503, Carinda, 2831, Australia" -> "house 1503 MIRALWYN ROAD, 2831, Carinda [au] -30.134,147.346"
  • global-osm-au-3626150293-1 suggest full missing -> pass: "Alva Road 1624, Blackall, 4472, Australia" -> rank 1 (want <= 5); first address 1624, ALVA ROAD, Blackall, Blackall Tambo Regional, Queensla [au]
  • global-osm-au-3626150293-1 near_miss -> pass: "Alva Road 1624, Blackall, 4472, Australia" -> "house 1624 ALVA ROAD, 4472, Blackall [au] -25.043,145.436"
  • global-osm-au-3626150293-0 suggest full missing -> pass: "1624 Alva Road, Blackall, 4472, Australia" -> rank 1 (want <= 5); first address 1624, ALVA ROAD, Blackall, Blackall Tambo Regional, Queensla [au]
  • global-osm-au-3626150293-0 near_miss -> pass: "1624 Alva Road, Blackall, 4472, Australia" -> "house 1624 ALVA ROAD, 4472, Blackall [au] -25.043,145.436"
  • global-osm-au-3404823268-0 suggest full missing -> pass: "1503 Miralwyn Road, Carinda, 2831, Australia" -> rank 1 (want <= 5); first address 1503, MIRALWYN ROAD, Carinda, Walgett Shire Council, New Sou [au]
  • global-osm-au-3404823268-0 near_miss -> pass: "1503 Miralwyn Road, Carinda, 2831, Australia" -> "house 1503 MIRALWYN ROAD, 2831, Carinda [au] -30.134,147.346"
  • global-osm-au-14030245366-1 suggest full missing -> pass: "Forestvale Road 3830, Forestvale, QLD, 4465, Australia" -> rank 1 (want <= 5); first address 3830, Forestvale Road, Forestvale, Maranoa Regional, Queensl [au]
  • global-osm-au-14030245366-1 near_miss -> pass: "Forestvale Road 3830, Forestvale, QLD, 4465, Australia" -> "house 3830 Forestvale Road, 4465, Forestvale [au] -26.160,147.949"
  • global-osm-au-5572982353-1 suggest full missing -> pass: "Montgomerys Road 291, Cathundral, 2823, Australia" -> rank 1 (want <= 5); first address 291, MONTGOMERYS ROAD, Cathundral, Warren Shire Council, New [au]
  • global-osm-au-5572982353-1 near_miss -> pass: "Montgomerys Road 291, Cathundral, 2823, Australia" -> "house 291 MONTGOMERYS ROAD, 2823, Cathundral [au] -31.899,147.875"
  • global-osm-au-14030245366-0 suggest full missing -> pass: "3830 Forestvale Road, Forestvale, QLD, 4465, Australia" -> rank 1 (want <= 5); first address 3830, Forestvale Road, Forestvale, Maranoa Regional, Queensl [au]
  • global-osm-au-14030245366-0 near_miss -> pass: "3830 Forestvale Road, Forestvale, QLD, 4465, Australia" -> "house 3830 Forestvale Road, 4465, Forestvale [au] -26.160,147.949"
  • global-osm-au-10982040319-0 suggest full missing -> pass: "49 Balala Road, Balala, NSW, 2358, Australia" -> rank 1 (want <= 5); first address 49, Balala Road, Balala, Uralla Shire Council, New South Wal [au]
  • global-osm-au-10982040319-0 near_miss -> pass: "49 Balala Road, Balala, NSW, 2358, Australia" -> "house 49 Balala Road, 2358, Balala [au] -30.621,151.341"
  • global-osm-au-2634985670-0 suggest full missing -> pass: "3424 Emaroo Road, 2840, Australia" -> rank 1 (want <= 5); first address 3424, EMAROO ROAD, Wanaaring, Unincorporated Far West Area, [au]
  • global-osm-au-2634985670-0 near_miss -> pass: "3424 Emaroo Road, 2840, Australia" -> "house 3424 EMAROO ROAD, 2840, Wanaaring [au] -30.118,144.331"
  • global-osm-au-2634985670-1 suggest full missing -> pass: "Emaroo Road 3424, 2840, Australia" -> rank 1 (want <= 5); first address 3424, EMAROO ROAD, Wanaaring, Unincorporated Far West Area, [au]
  • global-osm-au-2634985670-1 near_miss -> pass: "Emaroo Road 3424, 2840, Australia" -> "house 3424 EMAROO ROAD, 2840, Wanaaring [au] -30.118,144.331"
  • global-osm-au-10982040319-1 suggest full missing -> pass: "Balala Road 49, Balala, NSW, 2358, Australia" -> rank 1 (want <= 5); first address 49, Balala Road, Balala, Uralla Shire Council, New South Wal [au]
  • global-osm-au-10982040319-1 near_miss -> pass: "Balala Road 49, Balala, NSW, 2358, Australia" -> "house 49 Balala Road, 2358, Balala [au] -30.621,151.341"
  • global-osm-au-3984123319-0 suggest full missing -> pass: "45 Tinks Road, Quinninup, 6258, Australia" -> rank 1 (want <= 5); first address 45, Tinks Road, Quinninup, Shire of Manjimup, Western Austra [au]
  • global-osm-au-3984123319-0 near_miss -> pass: "45 Tinks Road, Quinninup, 6258, Australia" -> "house 45 Tinks Road, 6258, Quinninup [au] -34.467,116.206"
  • global-osm-au-4537152831-0 suggest full missing -> pass: "8469 Corona Road, Broughams Gate, 2880, Australia" -> rank 1 (want <= 5); first address 8469, CORONA ROAD, Broughams Gate, Unincorporated Far West A [au]
  • global-osm-au-4537152831-0 near_miss -> pass: "8469 Corona Road, Broughams Gate, 2880, Australia" -> "house 8469 CORONA ROAD, 2880, Broughams Gate [au] -31.009,141.150"
  • global-osm-au-3984123319-1 suggest full missing -> pass: "Tinks Road 45, Quinninup, 6258, Australia" -> rank 1 (want <= 5); first address 45, Tinks Road, Quinninup, Shire of Manjimup, Western Austra [au]
  • global-osm-au-3984123319-1 near_miss -> pass: "Tinks Road 45, Quinninup, 6258, Australia" -> "house 45 Tinks Road, 6258, Quinninup [au] -34.467,116.206"
  • global-osm-au-4537152831-1 suggest full missing -> pass: "Corona Road 8469, Broughams Gate, 2880, Australia" -> rank 1 (want <= 5); first address 8469, CORONA ROAD, Broughams Gate, Unincorporated Far West A [au]
  • global-osm-au-4537152831-1 near_miss -> pass: "Corona Road 8469, Broughams Gate, 2880, Australia" -> "house 8469 CORONA ROAD, 2880, Broughams Gate [au] -31.009,141.150"

Landmark checks (head)

Check Cohort pass ranked_low missing unexpected error
full suggest_landmark 53 0 18 0 0
off suggest_landmark 71 0 0 0 0
Check Cohort pass empty wrong_place wrong_region wrong_country
--- --- ---: ---: ---: ---: ---:

@Mokto Mokto changed the title test: accept G-NAF's point for Australian panel addresses the OSM node misplaces test: accept G-NAF's point where it disagrees with the Australian OSM panel node Oct 20, 2026
@Mokto
Mokto merged commit 9215db6 into main Oct 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Mokto