Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/evaluation/triage/portugal.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,24 @@ The full 21,282-case gold diff has no newly failing or worse cases. All 48 newly

- **Number absent from the index (18):** no street of the name within 150 m holds the number: Rua da República 15 (Mirandela) and 18 (5200-233), Rua do Corredouro 39, Praceta Frei Rodrigo Egídio 8, Av. Gonçalo Rodrigues Caldeira 19, Rua São Nicolau 18, Rua das Amendoeiras 107, Avenida dos Combatente do Ultramar 1, Avenida Nacional 100, Rua Teófilo Carvalho dos Santos 28, Rua Senhora da Abadia 473, Rua da Ceifeira 3, Largo Doutor Augusto de Oliveira Coimbra 8, Rua Marquês de Pombal 51, Rua da Valinha 90, Rua Comendador Joaquim Mendes Almeida 15, Rua do Queiró 9, and Rua Principal 79 (Santalha), whose nearby street has no houses, so a same-named street 2.4 km away answers. The register street is present in most of these with other numbers; this is source coverage, not a name mismatch.
- **Names this fix does not bridge (2):** `Avenida Virgílio Ferreira 2` is indexed 3 m away as `Avenida Vergílio Ferreira` (a spelling difference between sources; the misspelling repair runs only when no street matches), and `Rua da GNR 23` as `R DA GUARDA NACIONAL REPUBLICANA` 84 m away (an acronym). Neither is generalized here.

## Native address forms

The panel writes every query in two synthetic orders ending in `, Portugal`. Portuguese addresses are usually written otherwise: `Rua da República, 15, 5370-347 Mirandela`, `Rua José Relvas nº 13, …`, `R. dos Farias 12, …`, often without the country. Seven such forms of each passing panel group (133 tuning queries from 19 groups; the 28 queries of the 4 passing held-out groups were only scored), answered by the same index:

| | SearchAddress before | after | SuggestAddress before | after |
|---|---:|---:|---:|---:|
| tuning | 75/133 | 131/133 | 12/133 | 131/133 |
| held-out | 16/28 | 28/28 | 4/28 | 28/28 |

- Without a stated country, a postcode written `NNNN-NNN` (no other country writes them so) now lets search and SuggestAddress read the road as the register does. A town alone, or a postcode another country could share (`50667`), implies nothing: the resolver retries a query without its country, and a country-scoped spelling found that way would override a stated country.
- SuggestAddress reads a house number written as its own part (`Rua X, 12, …`, at most four digits and a letter, so not a postcode), skips `nº`, `n.º` and `n°`, and splits a part with no commas before a known postcode that follows the number (`Rua dos Farias 12 2955-243 Pinhal Novo`). A postcode with no town after it now scopes the house too.
- Not handled: `B.º` for Bairro (2 queries).

On the full gold suite, SearchAddress is unchanged (its queries state the country) and SuggestAddress gains 34 checks with none lost: 31 Dutch queries written without commas (`Chopinstraat 51 5481LP Schijndel Netherlands`), two Spanish and one Italian with the number as its own part (`Via Bernardo Clesio, 5, 38122 Trento, Italy`).

Still open without a country: a query whose number the index lacks can get no answer at all when the postcode's place and the stated town differ in name (`Estrada de Mem Martins 9999, 2725-377 Mem Martins`; 3 of 39 such probes), because accepting the postcode for that town requires a stated country. That is a resolver rule for every country, left for a separate change.

## Source coverage

The 18 absent numbers are coverage, not handling. Portugal's addresses come only from the register; Portugal is among the `osmLandmarks` extracts, read for landmarks only, so OpenStreetMap's Portuguese addresses never reach the index. Adding `portugal` to the `osm` sources would fill some of these gaps, but this panel is drawn from the same OpenStreetMap address nodes, so its Portuguese score would then largely measure the source agreeing with itself: that would be a data change to judge on other evidence, separately from this code.
3 changes: 3 additions & 0 deletions src/index/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2583,6 +2583,9 @@ impl Index {
// Farias".
if !housed && number.is_some_and(|n| !n.trim().is_empty()) {
let mut alternatives = cc
.or(postcode
.filter(|p| crate::matching::portuguese_postcode(p))
.map(|_| "pt"))
.map(|cc| crate::matching::road_expansions(road, cc))
.unwrap_or_default();
alternatives.extend(crate::matching::bare_road(road, cc));
Expand Down
89 changes: 85 additions & 4 deletions src/index/suggest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,17 @@ impl Typed {
}
}

/// A house number written as a part of its own, "12", "12A" or "nº 12", as the number:
/// short enough not to be a postcode.
fn comma_number(part: &str) -> Option<&str> {
let number = super::house_number(part);
let digits = number.bytes().take_while(u8::is_ascii_digit).count();
(matches!(digits, 1..=4)
&& number.len() <= digits + 1
&& number.bytes().skip(digits).all(|b| b.is_ascii_alphabetic()))
.then_some(number)
}

/// A Danish unit part: a floor with or without a door ("st.", "1. th", "st. 11", "1. c"), or a
/// side door alone.
fn danish_floor(text: &str) -> bool {
Expand Down Expand Up @@ -1606,7 +1617,11 @@ impl Index {
.map(|(_, rest)| Typed::new(rest).parts)
.unwrap_or_else(|| typed.parts[1..].to_vec());
let postcode = extracted.as_ref().map(|(pc, _)| pc.as_str());
let raw: Vec<&str> = typed.parts[0].split(' ').collect();
// "Rua José Relvas nº 13": the mark says only that a number follows
let raw: Vec<&str> = typed.parts[0]
.split(' ')
.filter(|w| !matches!(w.to_lowercase().as_str(), "nº" | "n.º" | "n°" | "n.°"))
.collect();
// the number: the first or last word, when a road remains; both when both could be
// ("11 Listopada 26" is house 26 on a road named for a date)
let number_at = (raw.len() > 1)
Expand Down Expand Up @@ -1641,7 +1656,7 @@ impl Index {
}
if postcode.is_some()
&& number_word.is_some()
&& (parts.len() == 1 || ((2..=3).contains(&parts.len()) && country.is_some()))
&& (parts.len() <= 1 || ((2..=3).contains(&parts.len()) && country.is_some()))
{
// "9 pittevej, aidt, 8881 thorsø, denmark": a place between the road and the
// postcode's town is another name of the house's place, or its postal town
Expand Down Expand Up @@ -1813,13 +1828,64 @@ impl Index {
return found;
}
}
// "Rua dos Farias 12 2955-243 Pinhal Novo": a known postcode after the number starts
// the town's part
if !out.iter().any(|s| s.kind == "address")
&& let [whole] = typed.parts.as_slice()
{
let words: Vec<&str> = whole.split(' ').collect();
let at = (2..words.len()).find(|&i| {
has_digit(words[i - 1])
&& has_digit(words[i])
&& (words[i].len() >= 5 || words[i].contains(|c: char| !c.is_ascii_digit()))
&& self.postcode_exists(words[i], cc)
});
if let Some(at) = at {
let found = self.suggest_streets(
&Typed {
parts: vec![words[..at].join(" "), words[at..].join(" ")],
open: typed.open,
},
cc,
focus,
want,
);
if found.iter().any(|s| s.kind == "address") {
return found;
}
}
}
// "Rua Pereira Charula, 12, 5340-278 Macedo de Cavaleiros": the number as its own part
if !out.iter().any(|s| s.kind == "address")
&& let [road, number, rest @ ..] = typed.parts.as_slice()
&& !road.split(' ').any(has_digit)
&& let Some(number) = comma_number(number)
{
let mut parts = vec![format!("{road} {number}")];
parts.extend_from_slice(rest);
let found = self.suggest_streets(
&Typed {
parts,
open: typed.open,
},
cc,
focus,
want,
);
if found.iter().any(|s| s.kind == "address") {
return found;
}
}
// "12 Rua dos Farias": Portugal's register names it "R FARIAS". Only Portugal's form,
// which expands no further: other countries' turn back ("straat" <-> "street").
if !out.iter().any(|s| s.kind == "address")
&& typed.parts[0].split(' ').any(has_digit)
&& cc.is_some_and(|c| c.eq_ignore_ascii_case("pt"))
&& cc.map_or_else(
|| postcode.is_some_and(crate::matching::portuguese_postcode),
|c| c.eq_ignore_ascii_case("pt"),
)
{
for road in crate::matching::road_expansions(&typed.parts[0], "pt") {
for road in crate::matching::road_expansions(&raw.join(" "), "pt") {
let mut parts = typed.parts.clone();
parts[0] = road;
let found = self.suggest_streets(
Expand Down Expand Up @@ -2134,6 +2200,21 @@ mod tests {
assert!(!danish_floor(other), "{other}");
}
}

#[test]
fn a_house_number_part_is_short_and_may_carry_its_mark() {
for (part, number) in [
("12", "12"),
("12A", "12A"),
("nº 12", "12"),
("1234", "1234"),
] {
assert_eq!(comma_number(part), Some(number), "{part}");
}
for other in ["12345", "2955-243", "12AB", "Lisboa", "", "1. th"] {
assert_eq!(comma_number(other), None, "{other}");
}
}
use super::*;

#[test]
Expand Down
8 changes: 8 additions & 0 deletions src/matching.rs
Original file line number Diff line number Diff line change
Expand Up @@ -771,6 +771,14 @@ fn ctt_road(road: &str) -> Vec<String> {
}
}

/// "2955-243": no other country writes its postcodes so, which tells a query that names no country
/// to read its road as Portugal's register does. Only the written shape counts: "50667" could be
/// German or French, and a country-scoped spelling needs the country to be certain.
pub(crate) fn portuguese_postcode(postcode: &str) -> bool {
let b = postcode.trim().as_bytes();
b.len() == 8 && b[4] == b'-' && b[..4].iter().chain(&b[5..]).all(u8::is_ascii_digit)
}

/// A Polish road as the index may name it: without "ul." (ulica), which most sources leave
/// out, and without its given names' initials ("ul. M. Kopernika" is Mikołaja Kopernika's
/// street). Without a stated country, only a road written with "ul." is taken to be Polish.
Expand Down
10 changes: 10 additions & 0 deletions tests/synthetic/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,12 @@ fn portuguese_houses_are_suggested_on_the_registers_abbreviated_street() {
for typed in [
"12 Rua dos Farias, Pinhal, Portugal",
"Rua dos Farias 12, 2955-243 Pinhal, Portugal",
// the forms Portuguese addresses are written in, often without the country
"Rua dos Farias 12, 2955-243 Pinhal",
"Rua dos Farias, 12, 2955-243 Pinhal",
"Rua dos Farias, 12, 2955-243",
"Rua dos Farias nº 12, 2955-243 Pinhal",
"Rua dos Farias 12 2955-243 Pinhal",
] {
let found = index().suggest(typed, 5, None, None, false);
assert!(!found.is_empty(), "{typed}");
Expand Down Expand Up @@ -1154,6 +1160,10 @@ fn cases() -> Vec<Case> {
"Rua dos Farias 12, Pinhal, Portugal",
at_house("R FARIAS", "12"),
),
case(
"Rua dos Farias, 12, 2955-243 Pinhal",
at_house("R FARIAS", "12"),
),
case(
"6 Avenida da Noruega, Pinhal, Portugal",
at_house("AV NORUEGA", "6"),
Expand Down