Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,29 @@ for candidate in result.candidates:

---

## Recognition Span

Every `Candidate` and the top-level `ExecutionResult` expose the half-open
`[start, end)` character range the recognition occupied in the input, so callers
can locate exactly where a canonicalized entity sat:

```python
from paxman.capabilities import Email

contract = Email.create_contract()
result = paxman.canonicalize("user@example.com", contract)

print(result.span) # (0, 16)
for candidate in result.candidates:
print(candidate.span) # (0, 16)
print(candidate.recognition_rule) # "standard_recognition"
```

`result.span` is the source span of the single resolved value on `SUCCESS`; it
is `None` for `MISSING`, `INVALID`, or `AMBIGUOUS`, since those cases have no
single resolved entity. For `AMBIGUOUS` results, locate each mention via the
per-candidate `Candidate.span`.

## Error Handling

Paxman raises typed exceptions for different failure modes:
Expand Down
162 changes: 162 additions & 0 deletions docs/adr/0004-single-value-invariant.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# ADR-0004: Single-Value Invariant — One Entity Per Call

## Status

Accepted

## Context

Paxman is a canonicalization *authority resolver*: it takes one ambiguous
human input and returns what the authoritative specification says that input
means, with provenance. From the outset the intended contract was that a single
`canonicalize()` call processes **one input that is expected to contain exactly
one presumed canonical value (or none at all)**.

Segmentation of free text into individual entities — "split this sentence into
the phone numbers / emails / dates it contains" — is the **caller's**
responsibility, not paxman's. Paxman operates at the *mention* level (it
normalizes a single recognized mention); it is not a document-level extractor.
When a caller passes input containing more than one entity, the honest outcome
is to **fail fast** rather than to spend recognition and validation work and
return a misleading aggregate status.

The pre-invariant behavior conflated two distinct situations under the single
`AMBIGUOUS` status:

1. **Genuine single-mention ambiguity** — one recognized span, but two or more
authoritative specifications disagree on its canonical value (e.g.
`01/02/2026`, which is `2026-01-02` under US order and `2026-02-01` under
European order). This is a *correct* and useful result: paxman surfaced the
conflict and declined to pick a winner.
2. **Segmentation violation** — the input actually contained *more than one*
entity (e.g. `"+60164041945 and +60164041946"`), and the multiple distinct
canonical values that surfaced are an artifact of un-split input, not a
genuine spec conflict.

Returning `AMBIGUOUS` for case 2 wastes the caller's processing and obscures a
usage error behind a status that looks like a real domain decision. The
invariant makes the distinction explicit and enforces it.

## Decision

**The Single-Value Invariant:** A single `canonicalize()` call must resolve at
most one canonical value. The caller is responsible for ensuring the input holds
one presumed entity (or none); paxman neither segments multi-entity input nor
aggregates multiple entities into one result.

**Enforcement — fail fast, opt-in.** The invariant is enforced only for
grammars that opt in by declaring `single_value = True` on their `Grammar`
subclass (the default is `False`). Opting in asserts that the grammar resolves
one mention per `canonicalize()` call; a grammar that deliberately emits
multiple spans for a *single* logical mention — e.g. a test probe exercising the
span-bearing seam — leaves the flag `False` and is exempt from the check.

For opted-in grammars, the engine treats **overlapping or contained spans as one
mention**: candidate spans are clustered, and any two spans that share a
character position fall into the same cluster. The check then inspects the
resulting mention clusters:

- **One mention** (any number of overlapping spans), even with multiple distinct
values → not a violation. This is genuine single-mention ambiguity (e.g.
`01/02/2026` under US vs European order, or cross-grammar reads of one span
such as `"€ 18 Dollar"`) and stays `AMBIGUOUS`.
- **Multiple separate mentions that all resolve to the same value** → not a
violation: coincidentally identical mentions (e.g. two copies of the same phone
number at different positions) still resolve to that one value via `SUCCESS`.
- **Two or more separate mentions resolving to more than one distinct value** →
**raise `MultipleMentionsError`**, a `PaxmanError` subclass, instead of
returning `AMBIGUOUS`.

Clustering by span overlap (rather than by grammar or by bare span count) is what
lets the invariant spare the cases it must not touch: a single mention read
several ways lands in one cluster and stays `AMBIGUOUS`, while two genuinely
separate entities — whether matched by one grammar (two phone numbers) or by
several (one via E.164, another via national) — form two non-overlapping clusters
and fail fast.

`MultipleMentionsError` is a *usage/contract* signal: it tells the caller "your
input contained more than one entity; split it and call `canonicalize()` once
per entity." It is distinct from `ContractError` (malformed contract
configuration) and from the `AMBIGUOUS` *status* (a legitimate domain result).

The error is raised at pipeline time, after recognition and before status
determination, so no partial `ExecutionResult` is produced for violating input.

**No known limitation on the dominant pattern.** Because clustering is performed
over the spans of *all* opted-in grammars, multi-entity input is caught whether its
separate instances are matched by one grammar (two phone numbers) or by several
(one via E.164, another via national). The only inputs that escape the check are
those whose grammars have not opted in (`single_value = False`), which is the
intended exemption for seam probes and any capability that elects not to assert
the invariant.

## Consequences

### Positive

- **Honest contract.** Multi-entity input fails fast instead of masquerading as
a domain ambiguity, saving caller processing and preventing silent
mis-aggregation.
- **`AMBIGUOUS` regains a single, precise meaning** — genuine single-mention
spec conflict only. This removes the previous overload and makes
status-based branching in callers reliable.
- **Caller-owned segmentation is now a hard, documented invariant** rather than
a soft convention, so future capabilities cannot accidentally reintroduce
multi-entity aggregation.
- **New public surface.** One new exception, ``MultipleMentionsError``, plus two
new optional span fields: ``Candidate.span`` and ``ExecutionResult.span`` carry
the half-open ``[start, end)`` recognition range. The ``Resolution`` enum and
capability contracts are unchanged.

### Negative

- **Breaking change for callers** that passed multi-entity inputs and inspected
the resulting `AMBIGUOUS` status. Those callers must now pre-split input (the
originally intended usage) and will observe `MultipleMentionsError` instead.
This is the desired correction, documented here so it is not treated as a
regression.
- **Tests that asserted multi-entity → `AMBIGUOUS`** must be rewritten to either
expect `MultipleMentionsError` (specifying the invariant) or to use
single-entity inputs.

### Risks

- **Over-strict on coincidentally identical multi-mention input.** Two copies of
the same value at different spans currently still resolve to `SUCCESS`. If the
project later wants strict single-*span* enforcement (fail fast even when
values coincide), the grouping predicate in `_enforce_single_value_invariant`
narrows from "spans disagree on value" to "more than one span exists." This ADR
deliberately chooses the value-based predicate to avoid surprising/pedantic
failures on duplicated input, and records the stricter option as a known
future lever.
- **Cross-grammar same-entity recognition must share a span.** If a future
grammar returns a *different* span for the same logical entity, it would be
misclassified as a second mention. The span contract (half-open,
`raw_text == text[start:end]`, bounds-checked in `_recognize`) keeps
same-entity matches aligned today; this invariant depends on that holding.

## Alternatives Considered

1. **Keep returning `AMBIGUOUS` for multi-entity input (status quo).** Rejected —
it overloads `AMBIGUOUS`, hides a caller usage error, and wastes processing,
directly contradicting the stated one-entity-per-call intent.
2. **Soft convention, no enforcement.** Rejected — without a hard signal the
invariant is unenforceable across nine capabilities and any future extension;
drift is inevitable.
3. **Strict single-span enforcement (fail fast on any >1 span, even identical
values).** Rejected for now as too pedantic: duplicated identical input still
has exactly one canonical value, which is what the invariant literally
guarantees. Recorded as a future lever if the project prefers span-strictness.
4. **Return a multi-value aggregate result type.** Rejected — contradicts the
single-value contract and the provenance-per-value model; segmentation
belongs to the caller.

## References

- ADR-0001 (Clean Architecture Pipeline) — pipeline layers and `Resolution`
status definitions this invariant refines.
- `paxman/engine/orchestrator.py` — `_collect_candidates`,
`_enforce_single_value_invariant`, `_dedup_candidates`, `_determine_status`.
- `paxman/core/errors.py` — `MultipleMentionsError`.
- `paxman/core/domain.py` — `RecognizedRep` (carries `start`/`end` spans used for
grouping).
1 change: 1 addition & 0 deletions paxman/capabilities/Country/grammar/alpha2_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ class Alpha2Grammar(Grammar[CountryNotation]):

name = "alpha2_recognition"
semantics = "alpha2_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CountryNotation]]:
"""Extract alpha-2 patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Country/grammar/alpha3_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ class Alpha3Grammar(Grammar[CountryNotation]):

name = "alpha3_recognition"
semantics = "alpha3_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CountryNotation]]:
"""Extract alpha-3 patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Country/grammar/name_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ class NameGrammar(Grammar[CountryNotation]):

name = "name_recognition"
semantics = "name_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CountryNotation]]:
"""Extract a country name representation from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Country/grammar/numeric_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ class NumericGrammar(Grammar[CountryNotation]):

name = "numeric_recognition"
semantics = "numeric_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CountryNotation]]:
"""Extract numeric patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Currency/grammar/code_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ class CodeRecognition(Grammar[CurrencyNotation]):

name = "code_recognition"
semantics = "code_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CurrencyNotation]]:
"""Extract standalone 3-letter code tokens from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Currency/grammar/symbol_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ class SymbolRecognition(Grammar[CurrencyNotation]):

name = "symbol_recognition"
semantics = "symbol_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CurrencyNotation]]:
"""Extract standalone symbol tokens from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Currency/grammar/word_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ class WordRecognition(Grammar[CurrencyNotation]):

name = "word_recognition"
semantics = "word_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[CurrencyNotation]]:
"""Extract standalone display-name word tokens from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Date/grammar/european_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ class EuropeanDateGrammar(Grammar[DateNotation]):

name = "european_recognition"
semantics = "european_calendar_date"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[DateNotation]]:
"""Extract European date patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Date/grammar/iso8601_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class ISO8601DateGrammar(Grammar[DateNotation]):

name = "iso8601_recognition"
semantics = "iso8601_calendar_date"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[DateNotation]]:
"""Extract ISO 8601 date patterns from text."""
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Date/grammar/slash_iso_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ class SlashISODateGrammar(Grammar[DateNotation]):

name = "slash_iso_recognition"
semantics = "iso8601_calendar_date"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[DateNotation]]:
"""Extract YYYY/MM/DD date patterns from text."""
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Date/grammar/us_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ class USDateGrammar(Grammar[DateNotation]):

name = "us_recognition"
semantics = "us_calendar_date"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[DateNotation]]:
"""Extract US date patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Email/grammar/localhost_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ class LocalhostEmailGrammar(Grammar[EmailNotation]):

name = "localhost_recognition"
semantics = "localhost_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[EmailNotation]]:
matches: list[RecognitionMatch[EmailNotation]] = []
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ class ObfuscatedEmailGrammar(Grammar[EmailNotation]):

name = "obfuscated_recognition"
semantics = "rfc5322_addr_spec"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[EmailNotation]]:
"""Extract obfuscated email patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Email/grammar/standard_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ class StandardEmailGrammar(Grammar[EmailNotation]):

name = "standard_recognition"
semantics = "rfc5322_addr_spec"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[EmailNotation]]:
matches: list[RecognitionMatch[EmailNotation]] = []
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/IP/grammar/ipv4_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ class IPv4Grammar(Grammar[IPNotation]):

name = "ipv4_recognition"
semantics = "ipv4_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[IPNotation]]:
"""Extract IPv4 dotted-decimal patterns from text."""
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/IP/grammar/ipv6_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ class IPv6Grammar(Grammar[IPNotation]):

name = "ipv6_recognition"
semantics = "ipv6_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[IPNotation]]:
"""Extract IPv6 address patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/ISBN/grammar/isbn10_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ class ISBN10RecognitionGrammar(Grammar[ISBNNotation]):

name = "isbn10_recognition"
semantics = "isbn10_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[ISBNNotation]]:
matches: list[RecognitionMatch[ISBNNotation]] = []
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/ISBN/grammar/isbn13_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ class ISBN13RecognitionGrammar(Grammar[ISBNNotation]):

name = "isbn13_recognition"
semantics = "isbn13_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[ISBNNotation]]:
matches: list[RecognitionMatch[ISBNNotation]] = []
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Money/grammar/code_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ class CodeRecognition(Grammar[MoneyNotation]):

name = "code_recognition"
semantics = "code_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[MoneyNotation]]:
"""Extract code+amount tokens from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Money/grammar/symbol_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ class SymbolRecognition(Grammar[MoneyNotation]):

name = "symbol_recognition"
semantics = "symbol_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[MoneyNotation]]:
"""Extract symbol+amount tokens from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Money/grammar/word_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ class WordRecognition(Grammar[MoneyNotation]):

name = "word_recognition"
semantics = "word_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[MoneyNotation]]:
"""Extract word+amount tokens from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Phone/grammar/e164_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ class E164Grammar(Grammar[PhoneNotation]):

name = "e164_recognition"
semantics = "e164_international"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[PhoneNotation]]:
"""Extract e164 patterns from text.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ class International00Grammar(Grammar[PhoneNotation]):

name = "international_00_recognition"
semantics = "e164_international"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[PhoneNotation]]:
"""Extract 00-prefixed international patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Phone/grammar/national_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ class NationalGrammar(Grammar[PhoneNotation]):

name = "national_recognition"
semantics = "national_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[PhoneNotation]]:
"""Extract national patterns from text.
Expand Down
1 change: 1 addition & 0 deletions paxman/capabilities/Phone/grammar/tel_uri_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ class TelUriGrammar(Grammar[PhoneNotation]):

name = "tel_uri_recognition"
semantics = "tel_uri_recognition"
single_value = True

def recognize(self, text: str) -> list[RecognitionMatch[PhoneNotation]]:
"""Extract tel: URI patterns from text.
Expand Down
Loading
Loading