Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
241 changes: 241 additions & 0 deletions docs/user/api-reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
# API Reference

This page is the concise reference for the public Python surface you import. For the mental model behind these types, see [Concepts](concepts/index.md).

---

## Installation and imports

```python
import paxman
from paxman.capabilities import Country, Currency, Date, Email, IP, ISBN, Money, Phone, SIUnit, URL
from paxman.core.domain import Resolution
from paxman.core.errors import (
PaxmanError, CapabilityError, ContractError,
MultipleMentionsError, RecognitionError, ValidationError,
)
```

Paxman has no runtime dependencies. `import paxman` is side-effect free — nothing runs until you register and call `canonicalize()`.

---

## Registration

Registration tells the engine which capabilities exist. It must complete from **one thread before the first `canonicalize()` call**; afterwards the registry freezes and reads are safe from any thread.

```mermaid
flowchart LR
A[import paxman] --> B[register]
B --> C[first canonicalize<br>freezes registry]
C --> D[further canonicalize<br>thread-safe reads]
B -.->|after freeze| E[CapabilityError]

style C fill:#fff8e1,stroke:#d4a017
style E fill:#fff5f5,stroke:#cc3333
```

| Function | Signature | What it does |
|----------|-----------|--------------|
| `paxman.register_all_shipped()` | `() -> None` | Registers every capability shipped in this release. Idempotent by name. |
| `paxman.register_capability(cap)` | `(cap: Capability) -> None` | Registers one capability instance, e.g. `Email()`. Fails if the name already exists or the registry is frozen. |
| `paxman.register_grammar(name, cls)` | `(capability_name: str, grammar_cls: type[Grammar]) -> None` | Registers a community grammar (see [Extending](extending.md)). Must happen before the first call. |
| `paxman.register_rule(name, cls)` | `(capability_name: str, rule_cls: type[Rule]) -> None` | Registers a community rule. Must happen before the first call. |

```python
# Quick exploration — everything
paxman.register_all_shipped()

# Explicit — only what you depend on
paxman.register_capability(Email())
paxman.register_capability(Date())
```

---

## `paxman.canonicalize()`

The sole entry point.

```python
def canonicalize(text: str, contract: CapabilityContract) -> ExecutionResult
```

| Parameter | Type | Description |
|-----------|------|-------------|
| `text` | `str` | Raw input. One presumed mention per call (see [Segmentation](../recipes/segmentation.md)). |
| `contract` | `CapabilityContract` | Created via `SomeCapability.create_contract(...)`. Carries `capability_name` so no separate name argument is needed. |

**Returns** `ExecutionResult` — always, for every domain outcome including missing or invalid input. Domain outcomes are statuses, not exceptions.

**Raises**

| Exception | Cause |
|-----------|-------|
| `CapabilityError` | No capability matches `contract.capability_name`, duplicate name, or registry frozen |
| `ContractError` | Malformed contract (unknown `pinned_rules`, unknown `output_format`, unknown semantics, missing required feature) |
| `MultipleMentionsError` | Two or more non-overlapping mentions resolved to different values — caller must split first |
| `RecognitionError` | Grammar raised or returned a malformed match (`rule`, `original_error`) |
| `ValidationError` | Rule raised inside `matches()`/`normalize()` (`rule`, `original_error`) |

`MISSING` / `INVALID` / `AMBIGUOUS` do **not** raise — they are `ExecutionResult.status` values (see [Execution Result](concepts/execution-result.md) and [Candidates & Ambiguity](concepts/candidates-and-ambiguity.md)).

---

## Contracts — `SomeCapability.create_contract()`

Every capability exposes a **keyword-only** factory. Common parameters come first, capability-specific ones after.

```python
contract = Email.create_contract(
# common — every capability
excluded_rules=(), # Sequence[str]
pinned_rules=None, # Sequence[str] | None — when not None, only these run
year=None, # int | None — only rules with publication_year <= year
output_format=None, # str | None — None / "default" / default / offered, else ContractError
extra_grammars=(), # tuple[str, ...] — opt-in community grammars
# capability-specific — varies by capability
include_obfuscated=False, # example: Email only
)
```

The contract is a **frozen dataclass** — construct it, pass it, inspect it, but do not mutate it. See [Contracts](concepts/contracts.md) for the full common-plus-specific table.

### `output_format` policy (identical for every capability)

- `None`, `"default"`, and the capability's `DEFAULT_OUTPUT_FORMAT` all resolve to the default.
- Any value in `OFFERED_OUTPUT_FORMATS` resolves to itself.
- Anything else raises `ContractError` immediately. Validation never consults `output_format`.

Current defaults and offered alternatives:

| Capability | Default (`DEFAULT_OUTPUT_FORMAT`) | Offered (`OFFERED_OUTPUT_FORMATS`) |
|------------|-----------------------------------|-------------------------------------|
| Country | `alpha2` | `alpha3`, `numeric`, `name` |
| Currency | `code` | *(none — single format)* |
| Date | `ISO` | `US` |
| Email | `email` | *(none)* |
| IP | `ip` | *(none)* |
| ISBN | `isbn13` | `hyphenated` |
| Money | `code_amount` | `compact` |
| Phone | `e164` | `rfc3966`, `national` |
| SI Unit | `symbol` | *(none)* |
| URL | `url` | *(none)* |

> The set of capabilities — and their offered formats — grows over time. Treat this table as the current release, not a closed list.

### Capability-specific flag summary (current release)

| Capability | Flag | Type | Default | Purpose |
|------------|------|------|---------|---------|
| Email | `include_obfuscated` | `bool` | `False` | `user at domain dot com` |
| Email | `include_localhost` | `bool` | `True` | `admin@localhost` |
| Country | `include_localized` | `bool` | `False` | CLDR multilingual names |
| Country | `include_historical` | `bool` | `False` | Deprecated names |
| Currency | `default_currency` | `str \| None` | `None` | Resolve shared bare symbol (`$`) to this alpha-3 code |
| IP | `include_ipv6` | `bool` | `True` | IPv6 |
| ISBN | `include_isbn10` | `bool` | `True` | Legacy ISBN-10 |
| ISBN | `include_range_validation` | `bool` | `False` | Registrant-range provenance |
| Money | `dollar_sign_currency` | `str \| None` | `None` | Resolve bare `$` amount to this alpha-3 code |
| Money | `precision` | `str` | `"strict"` | `strict` / `truncate` / `round` for over-precision amounts |
| Phone | `default_country` | `str \| None` | `None` | Interpret national numbers as if in this alpha-2 country |
| SI Unit | `allow_split_word_prefixes` | `bool` | `False` | `kilo gram` → `kg` |
| SI Unit | `allow_multi_solidus` | `bool` | `False` | `kg/m/s` preserved |
| Date | `two_digit_base_year` | `int \| None` | `None` | Base for 2-digit year expansion |

Validation: `default_currency` / `dollar_sign_currency` must be uppercase alpha-3; `default_country` must be uppercase alpha-2; `precision` must be one of the three values — otherwise `ContractError` at construction time.

---

## Execution result

```python
@dataclass(frozen=True)
class ExecutionResult:
status: Resolution # MISSING | INVALID | SUCCESS | AMBIGUOUS
canonicalized_value: str | None # str on SUCCESS, None otherwise
candidates: tuple[Candidate, ...] # all validated evidence, deduplicated
contract: CapabilityContract # the contract you passed in
version_stamp: VersionStamp # .paxman_version
span: tuple[int, int] | None # [start, end) of resolved value on SUCCESS, else None
```

```python
@dataclass(frozen=True)
class Candidate:
value: str
recognition_rule: str # grammar name, e.g. "standard_recognition"
validation_rule: str # rule name, e.g. "Section 3.4.1-addr-spec"
span: tuple[int, int] | None # half-open [start, end) in the input
provenance: tuple[Provenance, ...] # one or more authority citations (read-only property)

@dataclass(frozen=True)
class Provenance:
authority: str # "IETF", "ISO", "BIPM", "WHATWG", "CLDR", ...
specification_name: str # "RFC 5322", "ISO 3166-1", ...
kind: str # "specification" / "standard" / ...
reference_url: str
version: str | None
lifecycle: str # "active" / "deprecated" / ...
publication_year: int

@dataclass(frozen=True)
class VersionStamp:
paxman_version: str

class Resolution(Enum):
MISSING = "missing"
INVALID = "invalid"
SUCCESS = "success"
AMBIGUOUS = "ambiguous"
```

**Reading the result** (see also [Execution Result](concepts/execution-result.md)):

```python
from paxman.core.domain import Resolution

if result.status == Resolution.SUCCESS:
value = result.canonicalized_value # str, span is set
else:
# MISSING / INVALID / AMBIGUOUS — no single value, span is None
# inspect result.candidates and their provenance/spans
...
```

`span` semantics: on `SUCCESS` it is the span of the single resolved value; on `AMBIGUOUS` use each `candidate.span`; on `MISSING`/`INVALID` it is `None` and `candidates` is empty.

---

## Errors

All exceptions inherit from `PaxmanError`. See [Errors](concepts/errors.md) for handling patterns.

| Exception | Signal type | Typical cause |
|-----------|-------------|---------------|
| `CapabilityError` | setup | Unknown capability, duplicate name, registry already frozen |
| `ContractError` | setup | Bad `pinned_rules`, bad `output_format`, unknown semantics, missing required feature, bad `default_currency`/`default_country`/`precision` |
| `MultipleMentionsError` | usage | Two separate mentions with different values in one call — split first |
| `RecognitionError` | internal bug | Grammar raised or returned a bad span/raw_text. Fields: `rule`, `original_error` (`None` on structural failure) |
| `ValidationError` | internal bug | Rule raised inside `matches()`/`normalize()`. Fields: `rule`, `original_error` |

Statuses vs exceptions: statuses are domain answers returned inside `ExecutionResult`; exceptions mean the call was not valid to attempt. Catch exceptions at setup and at the outer edge of a batch loop; branch on statuses in normal flow.

---

## Quick lookup — choose the right capability

| Kind of text you have | Capability to use | Factory |
|-----------------------|-------------------|---------|
| Email addresses | Email | `Email.create_contract(...)` |
| Dates | Date | `Date.create_contract(...)` |
| Country codes or names | Country | `Country.create_contract(...)` |
| Currency codes / symbols / names (no amount) | Currency | `Currency.create_contract(...)` |
| IP addresses | IP | `IP.create_contract(...)` |
| ISBNs | ISBN | `ISBN.create_contract(...)` |
| Money amounts with currency | Money | `Money.create_contract(...)` |
| Phone numbers | Phone | `Phone.create_contract(...)` |
| SI unit expressions | SI Unit | `SIUnit.create_contract(...)` |
| Absolute URLs / IRIs | URL | `URL.create_contract(...)` |

New capabilities appear in minor releases — check `paxman.capabilities` for the current set. Each per-capability guide under [Capabilities](capabilities/index.md) details its recognized forms, output formats, contract flags, and provenance.
132 changes: 132 additions & 0 deletions docs/user/capabilities/country.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Country

Canonicalizes **one country mention** per call — a code or a name — to a canonical code.

> **In plain language:** give it `"US"`, `"United States"`, `"DE"`, or `"Alemania"` and it hands back the ISO alpha-2 code if a real spec says that name means that country. Multilingual and historical names are off by default so results are predictable; turn them on when you need them.

---

## What it recognizes — and what it does not

| Recognizes (current release — growing) | Does not recognize |
|----------------------------------------|--------------------|
| ISO 3166-1 alpha-2 (`US`, `DE`), alpha-3 (`USA`), numeric (`840`) | Ad-hoc abbreviations without a spec (`'Murica`) |
| Country names in English (`United States`, `Germany`) | Regions or subdivisions (`California`, `Bavaria`) — countries only |
| CLDR localized names (`Alemania` → `DE`) — only when `include_localized=True` | Localized names when the flag is off — recognized but `INVALID` (no authority claims them) |
| Historical/deprecated names (`Burma`) — only when `include_historical=True`, via ISO 3166-3 | Historical names when the flag is off — `INVALID` |

---

## Canonical output

Default `output_format` is `"alpha2"`.

| `output_format` | Renders | Example for United States |
|-----------------|---------|---------------------------|
| *(default)* `alpha2` / `None` / `"default"` | ISO 3166-1 alpha-2 | `US` |
| `alpha3` | ISO 3166-1 alpha-3 | `USA` |
| `numeric` | ISO 3166-1 numeric | `840` |
| `name` | English short name | `United States` |

Any other value raises `ContractError`.

```python
from paxman.capabilities import Country
import paxman

paxman.register_all_shipped()
paxman.canonicalize("United States", Country.create_contract()).canonicalized_value # "US"
paxman.canonicalize("United States", Country.create_contract(output_format="alpha3")).canonicalized_value # "USA"
paxman.canonicalize("United States", Country.create_contract(output_format="numeric")).canonicalized_value # "840"
paxman.canonicalize("US", Country.create_contract(output_format="name")).canonicalized_value # "United States"
```

---

## Contract

```python
contract = Country.create_contract(
include_localized=False, # bool, default False — CLDR multilingual names
include_historical=False, # bool, default False — ISO 3166-3 deprecated names
output_format=None, # "alpha2" (default), "alpha3", "numeric", "name"
# plus every common field: excluded_rules / pinned_rules / year / extra_grammars
)
```

- `include_localized` gates the CLDR localized-name rule; when `False`, `Alemania` is recognized but **no rule validates it** → `INVALID`.
- `include_historical` gates ISO 3166-3; a historical name that validates returns the **historical entity's own former code** (e.g. `Burma` → `BU`), not a successor state's code.

Historical names can also be viewed as excluding the current ISO 3166-1 name rules and pinning to the 3166-3 rule — the contract controls which specs are consulted (see [Contracts](../concepts/contracts.md)).

---

## Statuses

| Input | Contract | Status | Value / why |
|-------|----------|--------|-------------|
| `US` | defaults | `SUCCESS` | `"US"` |
| `United States` | defaults | `SUCCESS` | name → `"US"` |
| `Alemania` | defaults | `INVALID` | recognized (name grammar) but no rule accepts it without `include_localized` |
| `Alemania` | `include_localized=True` | `SUCCESS` | → `"DE"` with CLDR provenance |
| `Burma` | defaults | `INVALID` | needs `include_historical` |
| `Burma` | `include_historical=True` | `SUCCESS` | → `"BU"` (historical entity's own code) |
| `ZZ` | any | `INVALID` or `MISSING` | no spec claims it |
| `hello world` | any | `MISSING` | no country pattern |
| `US and DE` (two different mentions) | any | raises `MultipleMentionsError` | split first |

```mermaid
flowchart TB
A[Text e.g. Alemania] --> G[Grammars:<br>alpha2 / alpha3 / numeric / name]
G --> R1{ISO 3166-1 name rule}
G --> R2{CLDR localized rule<br>only if include_localized}
G --> R3{ISO 3166-3 historical<br>only if include_historical}
R1 & R2 & R3 --> D{Dedup values}
D -->|one value| OK[SUCCESS]
D -->|none| INV[INVALID]
G -->|no pattern| MISS[MISSING]

style OK fill:#e6ffed,stroke:#2d8a4e
style INV fill:#fff5f5,stroke:#cc3333
style MISS fill:#fff5f5,stroke:#cc3333
```

---

## Notebook snippet — normalize a mixed column

```python
import paxman
from paxman.capabilities import Country
from paxman.core.domain import Resolution
from paxman.core.errors import CapabilityError, ContractError, MultipleMentionsError

paxman.register_all_shipped()
contract = Country.create_contract(include_localized=True)
contract_hist = Country.create_contract(include_localized=True, include_historical=True)

rows = ["US", "United States", "Alemania", "Burma", "not a country", "US and DE"]

for text in rows:
for label, c in [("default+localized", contract), ("+historical", contract_hist)]:
try:
r = paxman.canonicalize(text, c)
except (MultipleMentionsError, CapabilityError, ContractError) as e:
print(f"{text!r:20} [{label}] → exception {type(e).__name__}: {e}")
continue
Comment thread
coderabbitai[bot] marked this conversation as resolved.
val = r.canonicalized_value if r.status == Resolution.SUCCESS else "—"
prov = r.candidates[0].provenance[0].specification_name if r.candidates else "—"
print(f"{text!r:20} [{label:18}] → {r.status.value:10} {val!r:8} ({prov})")
```

---

## Provenance

- **ISO 3166-1** (alpha-2 / alpha-3 / numeric / English names)
- **ISO 3166-3** (historical, when enabled)
- **CLDR** (Unicode localized names, when enabled)

Each candidate's `validation_rule` carries the section, and `candidate.provenance[0].publication_year` the year.

See also: [Execution Result](../concepts/execution-result.md), [Provenance](../concepts/provenance.md), [Segmentation](../../recipes/segmentation.md).
Loading
Loading