Release span-panel-api 3.8.0b1 and span-panel-api-schema-1 1.4.0b1

AGENTS.md

@@ -0,0 +1,61 @@
+# Working on span-panel-api
+
+Guidance for anyone changing this repository, person or coding agent. It lists what has to be kept up when the code changes; the procedures themselves live in [DEVELOPMENT.md](DEVELOPMENT.md) (setup, tests, captures, conformance) and
+[RELEASE.md](RELEASE.md) (versions, floors, tags, pre-releases). Read those before changing a capture, a dependency floor or a version.
+
+## The rule every field follows
+
+A value the panel did not publish is `None` (or unknown), never a constant. A `0.0`, `False`, `""` or empty collection in a snapshot field is a reading the panel published, or an answer the library can derive from what the panel declared, and the docstring
+of the field says which. Where a field is not yet optional, the adapter fidelity harness records its absence value as an explicit, reasoned exception rather than letting it pass unnoticed.
+
+## When you add, rename or change a snapshot field or an adapter read
+
+Every read the schema-1 adapter makes is accounted for in more than one place, and each place fails the suite if it is left behind:
+
+- **Field metadata and discovery** (`packages/schema-1/src/span_panel_api_schema_1/field_metadata.py`). A reading with a unit gets a `_PROPERTY_FIELD_MAP` row, or a row in a lugs table; a property read into the snapshot without one goes in
+  `_CONSUMED_WITHOUT_A_ROW`. Otherwise it is reported as discovered and becomes an extension reading. `tests/test_schema_one_discovery.py` proves each entry by perturbing it against the reference tree; an entry the tree does not declare goes in
+  `_ABSENT_FROM_THE_CAPTURE` with a synthetic experiment of its own.
+- **Conformance** (`tests/test_schema_one_conformance.py`). A property no vendored catalog defines goes in `_SPAN_EXTENSIONS`, and one the pinned emitter's reference tree never declares goes in `_NOT_EXERCISED_BY_THE_EMITTER`, each with its reason.
+- **Public API** (`tests/test_public_api_unchanged.py`). A new export is added to `__all__` and to the expected set, with a dated note.
+- **The adapter fidelity harness** (`tests/adapter_fidelity/`), below.
+
+## The adapter fidelity harness
+
+`harness.py` replays a captured device tree through the real adapter, perturbs every declared property, and classifies each one: mapped, an extension reading, intentionally dropped by code that says why, or silently dropped. It then holds every value to a
+documented rule:
+
+- **Transforms** (`rules.TRANSFORMS`): for each row (a device role and a `node/property`) and each snapshot field it moves, how the field follows from the wire value: identity, negation, integer, literal, boolean, or a derivation with the rule stated where
+  it is simple. Every cell shares them, and `test_every_mapped_field_has_a_documented_transform` requires one for every field-metadata row. Change the transform when you change a read, and give a device whose frame differs its own role rather than bending
+  a shared row.
+- **Derivations**: a field that holds a value no published property moves on its own, such as one decided by a declaration, or a documented elimination. Each cell carries only the derivations its captures need. `test_every_derivation_is_needed` refuses one
+  that no cell uses, because an unused justification is how a list of reasons becomes an allowlist. `rules.py` holds the derivations for the two captures in `tests/fixtures/`, and `rules_captured.py` holds those for the reference captures.
+- **Drops** (`DropReason`): the three kinds of intentional drop, each citing the code. A cell pins every row it drops _with a value_, each with a reviewed reason (`dropped_with_a_value` in `test_main32_fidelity.py`, `DROPPED_WITH_A_VALUE` in
+  `test_capture_fidelity.py`), so a drop rule that newly covers a published property waits for review. Rows the library drops today but should not are listed in `deferred` with the reason, and a deferral that no longer holds fails like a new silent drop.
+
+`test_capture_fidelity.py` measures every capture in `tests/fixtures/captures/` and pins by row, never by capture, so a capture added to that directory is covered without editing the tests. `test_harness.py` plants each kind of defect and checks the
+harness names it.
+
+## The expected-failure registry
+
+`tests/expected_failures.py` lists, by `<file>::<function>`, the acceptance tests this line does not pass yet. `tests/conftest.py` marks each one `xfail(strict=True)`, so:
+
+- a `PENDING` row is behaviour a later change brings, and that change deletes the row: a listed test that starts passing fails the run until it is;
+- a `DIVERGENT` row is behaviour this library deliberately does not have, and stays until the test or the decision changes;
+- a row naming a file that does not exist, or a test function that file does not define (at module level or on a `Test` class), stops the run. The check reads the files rather than the tests a run collected, so a partial run (`-k`, one file) judges every
+  row exactly as a full run does, and a renamed or deleted test file cannot orphan its rows.
+
+Ported acceptance tests are never edited. A strict xfail stops at its first failing line, so a companion file (`*_companions.py`) asserts what each divergent test checks after its divergent line.
+
+## Vendored captures
+
+Captures are byte copies and must stay that way:
+
+- `tests/fixtures/captures/` holds the reference captures with their upstream `LICENSE` and a `README.md` naming the upstream repository, path and commit; `SHA256SUMS` pins each capture and the `LICENSE` by SHA-256. `tests/test_captures_unchanged.py`
+  checks every pin, requires a pin for every capture and the directory to hold nothing but the pinned files, `README.md` and `SHA256SUMS`; those two are this repository's records and are not pinned. The two captures in `tests/fixtures/` are pinned the same
+  way by `tests/adapter_fidelity/test_main32_fidelity.py`.
+- `.pre-commit-config.yaml` excludes them from every hook that rewrites files. Keep any new capture path in those excludes.
+- A capture is replaced only by copying it again from its upstream commit and updating the pins, never by editing it.
+
+The schema-1 reference tree (`packages/schema-1/src/span_panel_api_schema_1/reference/parent_child_tree.json`) is package data, produced by `scripts/capture_parent_child_reference.py` from `scripts/reference_panel.yaml` with the pinned emitter;
+DEVELOPMENT.md, "Regenerating a vendored capture", has the procedure. Change the manifest, never the output, and keep every circuit inside the position range `PANEL_POSITIONS_BY_MODEL` gives the panel's model: the adapter reports the table's range and
+warns once for a panel whose circuits fall outside it. A changed capture is a release of the adapter that ships it (RELEASE.md).

CHANGELOG.md

@@ -13,6 +13,7 @@ Snapshot fields that held a fabricated constant can now be unknown, and with `sp
 
 ### Added
 
+- **spanos3/r202639/03 batch 2: reads more of the data this firmware publishes.**
 - **`V2StatusInfo.hardware_version` reports the hardware version string from `GET /api/v2/status`.**
 
 ### Changed
@@ -22,7 +23,10 @@ Snapshot fields that held a fabricated constant can now be unknown, and with `sp
 - **BREAKING FOR CONSUMERS: `SpanPanelSnapshot.uptime_s` becomes `int | None`**, and with those parsers it is `None` on a parent/child panel and before a flat panel first reports ready, where it was a constant `0`.
 - **BREAKING FOR CONSUMERS: `SpanCircuitSnapshot.instant_power_update_time_s` and `energy_accum_update_time_s` become `int | None` and default to `None`**, and with those parsers they are `None` until the reading has arrived and always on a parent/child
   panel, where they were `0`.
-- **The `schema-0` and `schema-1` extras require `span-panel-api-schema-0` and `span-panel-api-schema-1` 1.3.0 or newer**, the adapters that report these values as unknown.
+- **BREAKING FOR CONSUMERS: `SpanCircuitSnapshot.is_240v` becomes `bool | None`**, and with `span-panel-api-schema-1` 1.4.0 or newer it is `None` where the pole count is unpublished.
+- **BREAKING FOR CONSUMERS: `SpanPanelSnapshot.lugs_at_service_entrance` becomes `bool | None`**, and with `span-panel-api-schema-1` 1.4.0 or newer it is `None` while the upstream lugs declare their feed but leave it unpublished.
+- **The `schema-0` extra requires `span-panel-api-schema-0` 1.3.0 or newer and the `schema-1` extra requires `span-panel-api-schema-1` 1.4.0 or newer**, the adapters that report these values as unknown; schema-1 1.4.0 also fills the snapshot fields this
+  release adds.
 
 ### Removed
 

CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

packages/schema-1/CHANGELOG.md

@@ -11,10 +11,15 @@ Pre-releases are not listed separately. A beta is a step towards the next public
 
 ## [Unreleased]
 
-Requires `span-panel-api` **3.7.0 or newer**, whose snapshot fields accept the unknown values this parser now reports.
+Requires `span-panel-api` **3.8.0 or newer**, whose snapshot fields accept the unknown values this parser now reports and carry the fields it now fills.
+
+### Added
+
+- **spanos3/r202639/03 batch 2: reads more of the data this firmware publishes.**
 
 ### Changed
 
+- The shipped reference tree is recaptured with every circuit inside the panel's breaker positions.
 - **`proximity_proven` is `None`**, where it was a hard-coded `True`, because the tree publishes no proximity property.
 - **`uptime_s` is `None`**, where it was always `0`, because no device publishes an uptime and this parser keeps no readiness clock to measure one.
 

packages/schema-1/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "span-panel-api-schema-1"
-version = "1.3.0b1"
+version = "1.4.0b1"
 description = "Parent/child schema (data-model-version 1.x) parser for span-panel-api"
 authors = [
     {name = "SpanPanel"}
@@ -31,9 +31,9 @@ dependencies = [
     # Raised to 3.7.0b1, whose snapshot fields accept the unknown values this
     # parser now reports: against an earlier one, the update times it leaves
     # unset read as a constant 0.
-    # To rise to 3.8.0b1 when that release is cut: this parser fills snapshot
-    # fields new in it and imports `device_id_order` from its models.
-    "span-panel-api>=3.7.0b1,<4.0",
+    # Raised to 3.8.0b1: this parser fills snapshot fields new in it and imports
+    # `device_id_order` from its models.
+    "span-panel-api>=3.8.0b1,<4.0",
     # Only this distribution depends on the eBus SDK. The bootstrap and
     # schema-0 stay clean, so a flat-panel install never pulls it in — which is
     # what bounds the release coupling this dependency introduces to panels on

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "span-panel-api"
-version = "3.7.0b1"
+version = "3.8.0b1"
 description = "A client library for SPAN Panel API"
 authors = [
     {name = "SpanPanel"}
@@ -69,7 +69,9 @@ schema-0 = ["span-panel-api-schema-0>=1.3.0b1"]
 # Raised to 1.2.2 for 3.6.2: the readiness gate every snapshot passes asks that
 # adapter what is still missing, and 1.2.2 restarts its feed grace correctly.
 # Raised to 1.3.0b1 for 3.7.0b1, for the same reason as schema-0's.
-schema-1 = ["span-panel-api-schema-1>=1.3.0b1"]
+# Raised to 1.4.0b1 for 3.8.0b1: that adapter fills the snapshot fields this
+# bootstrap adds.
+schema-1 = ["span-panel-api-schema-1>=1.4.0b1"]
 
 [project.urls]
 Homepage = "https://github.com/SpanPanel/span-panel-api"

src/span_panel_api/models.py

@@ -1291,11 +1291,12 @@ class SpanPanelSnapshot:
     first_position: int | None = None
     """The panel's first breaker position. v1.0 only.
 
-    From the reported model where its range is known, otherwise the lowest
-    occupied space, and widened to any occupied space outside the model's
-    range. `None` on the flat schema and where neither is known. Together with
-    `last_position` it says which positions exist, occupied or not; no circuit
-    is made up for an empty one.
+    The reported model's range as the adapter's model table gives it, where
+    the table knows the model, otherwise the lowest occupied space. An occupied
+    space outside a known model's range does not move it: the adapter warns
+    once for that panel instead. `None` on the flat schema and where neither is
+    known. Together with `last_position` it says which positions exist,
+    occupied or not; no circuit is made up for an empty one.
     """
     last_position: int | None = None
     """The panel's last breaker position. v1.0 only; see `first_position`."""

tests/conftest.py

@@ -2,12 +2,13 @@
 
 from __future__ import annotations
 
+import ast
 import asyncio
 import itertools
 import json
 import os
 from pathlib import Path
-from collections.abc import AsyncGenerator
+from collections.abc import AsyncGenerator, Iterable
 from unittest.mock import MagicMock, patch
 
 import paho.mqtt.client as paho
@@ -50,44 +51,55 @@ def _load_dotenv() -> None:
 _load_dotenv()
 
 
-def pytest_collection_modifyitems(items: list[pytest.Item]) -> None:
+def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
     """Mark every test `expected_failures.py` lists as a strict expected failure.
 
-    A row naming anything but a test function of its collected module is a typo,
-    a rename or a helper that would silently mark nothing, so it stops the run.
+    A row naming a file that does not exist, or a test that file does not
+    define, is a typo, a rename or a deletion that would silently mark nothing,
+    so it stops the run. The check reads the files themselves rather than what
+    this run collected, so a partial run (`-k`, one file) judges every row the
+    same way a full one does.
     """
-    modules: dict[str, object] = {}
+    stale = stale_rows(EXPECTED_FAILURES, config.rootpath)
+    if stale:
+        raise pytest.UsageError(f"expected_failures.py names tests that do not exist: {', '.join(stale)}")
     for item in items:
         if not isinstance(item, pytest.Function):
             continue
         path = item.nodeid.partition("::")[0]
-        modules[path] = item.module
         reason = EXPECTED_FAILURES.get(f"{path}::{item.originalname}")
         if reason is not None:
             item.add_marker(pytest.mark.xfail(strict=True, reason=reason))
 
-    stale = [
-        key
-        for key in EXPECTED_FAILURES
-        if (path := key.partition("::")[0]) in modules and not _is_test_function(modules[path], key.partition("::")[2])
-    ]
-    if stale:
-        raise pytest.UsageError(f"expected_failures.py names tests that do not exist: {', '.join(sorted(stale))}")
 
+def stale_rows(rows: Iterable[str], root: Path) -> list[str]:
+    """The rows, sorted, whose file under `root` is missing or defines no such test."""
+    defined: dict[str, frozenset[str]] = {}
+    stale: list[str] = []
+    for key in rows:
+        path, _, name = key.partition("::")
+        if path not in defined:
+            defined[path] = _test_functions(root / path)
+        if name not in defined[path]:
+            stale.append(key)
+    return sorted(stale)
 
-def _is_test_function(module: object, name: str) -> bool:
-    """Whether `name` is a function pytest collects from `module` under the default `test` prefix.
 
-    At module level or as a method of one of the module's `Test` classes.
+def _test_functions(path: Path) -> frozenset[str]:
+    """The test functions `path` defines under the default `test` prefix, or none if it is not a file.
+
+    At module level or as a method of one of its `Test` classes, read from the
+    source, so nothing is imported to answer.
     """
-    if not name.startswith("test"):
-        return False
-    if callable(getattr(module, name, None)):
-        return True
-    return any(
-        isinstance(owner, type) and owner_name.startswith("Test") and callable(getattr(owner, name, None))
-        for owner_name, owner in vars(module).items()
-    )
+    if not path.is_file():
+        return frozenset()
+    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
+    functions = (ast.FunctionDef, ast.AsyncFunctionDef)
+    names = {node.name for node in tree.body if isinstance(node, functions)}
+    for node in tree.body:
+        if isinstance(node, ast.ClassDef) and node.name.startswith("Test"):
+            names.update(member.name for member in node.body if isinstance(member, functions))
+    return frozenset(name for name in names if name.startswith("test"))
 
 
 @pytest.fixture(autouse=True)

tests/expected_failures.py

@@ -3,8 +3,8 @@
 Keyed by pytest node id without parameters or class (`<file>::<function>`), so
 every parametrized case of a listed function is expected to fail. `conftest.py` marks
 each one `xfail(strict=True)`: a listed test that starts passing fails the run
-until its row is deleted, and a row naming anything but a test function of its
-module stops the run.
+until its row is deleted, and a row naming a file that does not exist, or a test
+function that file does not define, stops the run, partial runs included.
 
 `PENDING` rows assert behaviour a later change brings; that change deletes
 them. `DIVERGENT` rows assert behaviour this library deliberately does not

tests/test_expected_failures_registry.py

@@ -0,0 +1,89 @@
+"""The expected-failure registry stops the run on a row that names nothing.
+
+A row whose file was renamed or deleted, or whose test no longer exists, would
+otherwise mark nothing and stay in the registry unnoticed. The check reads the
+files, not what a run collected, so a partial run judges the rows the same way
+a full one does and raises no false alarm.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+
+import conftest
+from conftest import stale_rows
+from expected_failures import EXPECTED_FAILURES
+
+_ROOT = Path(__file__).resolve().parent.parent
+
+_SOURCE = '''
+def test_module_level():
+    pass
+
+
+async def test_async():
+    pass
+
+
+def helper():
+    pass
+
+
+class TestGroup:
+    def test_method(self):
+        pass
+
+
+class Helpers:
+    def test_not_collected(self):
+        pass
+'''
+
+
+def _tree(tmp_path: Path) -> Path:
+    (tmp_path / "tests").mkdir()
+    (tmp_path / "tests" / "test_sample.py").write_text(_SOURCE, encoding="utf-8")
+    return tmp_path
+
+
+def test_the_registry_names_only_tests_that_exist() -> None:
+    assert stale_rows(EXPECTED_FAILURES, _ROOT) == []
+
+
+def test_tests_at_module_level_and_on_test_classes_are_found(tmp_path: Path) -> None:
+    root = _tree(tmp_path)
+    rows = [f"tests/test_sample.py::{name}" for name in ("test_module_level", "test_async", "test_method")]
+
+    assert stale_rows(rows, root) == []
+
+
+@pytest.mark.parametrize(
+    "row",
+    [
+        "tests/test_renamed.py::test_module_level",
+        "tests/test_sample.py::test_gone",
+        "tests/test_sample.py::helper",
+        "tests/test_sample.py::test_not_collected",
+    ],
+    ids=["missing-file", "missing-test", "not-a-test", "not-on-a-test-class"],
+)
+def test_a_row_naming_nothing_is_stale(tmp_path: Path, row: str) -> None:
+    assert stale_rows([row], _tree(tmp_path)) == [row]
+
+
+def test_a_stale_row_stops_even_a_run_that_collected_nothing_of_it(
+    monkeypatch: pytest.MonkeyPatch, pytestconfig: pytest.Config
+) -> None:
+    """A partial run collects none of the registry's files, and still sees a stale row."""
+    stale = "tests/test_does_not_exist.py::test_anything"
+    monkeypatch.setattr(conftest, "EXPECTED_FAILURES", {**EXPECTED_FAILURES, stale: "planted"})
+
+    with pytest.raises(pytest.UsageError, match="test_does_not_exist"):
+        conftest.pytest_collection_modifyitems(pytestconfig, [])
+
+
+def test_a_partial_run_raises_no_false_alarm(pytestconfig: pytest.Config) -> None:
+    """With every row naming a test that exists, a run that collected none of them proceeds."""
+    conftest.pytest_collection_modifyitems(pytestconfig, [])