From 7c111d5953316ef5ff0343429ae5d3f60f93adfa Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 19 Oct 2026 12:37:12 -0400 Subject: [PATCH] docs(changelog): link the 1.5.0 entries to the API (#1941) - docs/source/changelog/1.5.0.rst: literals naming a documented pcapkit object become Python-domain roles; names resolvable by suffix use the `.Name` search form, ambiguous ones an explicit pcapkit path. Removed names, concepts, pip extras, CI cell labels and stdlib names stay literal. - util/changelog_md.py: rule 2 renders :class:/:meth:/:func:/:attr:/ :data:/:mod:/:exc: as code spans of the name Sphinx displays (tilde, leading dot, explicit title, `!`, and `()` on callables). - tests: new PythonRoleTests; the guard tests use roles rule 2 still does not know. - CHANGELOG.md regenerated; only the callables gain `()`. Closes #1941 --- CHANGELOG.md | 134 +- docs/source/changelog/1.5.0.rst | 2531 +++++++++++++++------------- tests/project/test_changelog_md.py | 115 +- util/changelog_md.py | 72 +- 4 files changed, 1567 insertions(+), 1285 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 53955a869..e5fced5b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,7 +15,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - **a breaking change to** `AppType.proto`: it is the single transport of the registry the member lives in, not every transport IANA assigned the service, and `register_apptype` takes its transports as varargs ([#806](https://github.com/JarryShaw/PyPCAPKit/issues/806), [#815](https://github.com/JarryShaw/PyPCAPKit/pull/815)). Aliases show in `repr()` and `str()`, and can be registered per transport ([#807](https://github.com/JarryShaw/PyPCAPKit/issues/807), [#837](https://github.com/JarryShaw/PyPCAPKit/pull/837)). - **a breaking change to** `TransportProtocol`: its base moves from `aenum.IntFlag` to `aenum.IntEnum`, so `|` between two members gives a plain `int` rather than a composite member ([#808](https://github.com/JarryShaw/PyPCAPKit/issues/808), [#836](https://github.com/JarryShaw/PyPCAPKit/pull/836)). - **a breaking change to** `TLSKeyLabel`: the hand-written class in `pcapkit.protocols.misc.pcapng` becomes the generated `pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` (`PCAPNG_TLSKeyLabel` in `pcapkit.const.pcapng`), crawled from the IANA registry [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) creates ([#886](https://github.com/JarryShaw/PyPCAPKit/issues/886), [#890](https://github.com/JarryShaw/PyPCAPKit/pull/890)), and the old path is removed ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). -- **a breaking change to** `Method.get`: the method token is case-sensitive, per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so `Method.get('get')` no longer resolves to `GET` ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), [#907](https://github.com/JarryShaw/PyPCAPKit/pull/907)). It also looks a key up by value, so `BASELINE-CONTROL` and `VERSION-CONTROL` resolve ([#908](https://github.com/JarryShaw/PyPCAPKit/issues/908), [#915](https://github.com/JarryShaw/PyPCAPKit/pull/915)). +- **a breaking change to** `Method.get()`: the method token is case-sensitive, per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so `Method.get('get')` no longer resolves to `GET` ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), [#907](https://github.com/JarryShaw/PyPCAPKit/pull/907)). It also looks a key up by value, so `BASELINE-CONTROL` and `VERSION-CONTROL` resolve ([#908](https://github.com/JarryShaw/PyPCAPKit/issues/908), [#915](https://github.com/JarryShaw/PyPCAPKit/pull/915)). - **a breaking change to** `ExtensionHeader`: it is generated from IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers registry's flag, and the two disagree on 147 (`BIT-EMU`) ([#925](https://github.com/JarryShaw/PyPCAPKit/issues/925), [#926](https://github.com/JarryShaw/PyPCAPKit/pull/926)). - the TCP and UDP `AppType` registries load on first use rather than at import ([#1538](https://github.com/JarryShaw/PyPCAPKit/issues/1538), [#1560](https://github.com/JarryShaw/PyPCAPKit/pull/1560)). - **a breaking change to** `LLACode`: `Wilcard` is spelled `Wildcard` and `MH` is `MN`, the names [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) gives codes 0 and 2, with no alias for either old name ([#1848](https://github.com/JarryShaw/PyPCAPKit/issues/1848), [#1872](https://github.com/JarryShaw/PyPCAPKit/pull/1872)). @@ -23,28 +23,28 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Fixed - constant lookups that rejected a value the registry defines. `RouterAlert(0)` is the only value [RFC 2113](https://datatracker.ietf.org/doc/html/rfc2113) defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPX `Socket(0)` is that protocol's own default, so `bytes(IPX(...))` crashed on its own defaults; and the FTP `_missing_` overrides were plain methods rather than classmethods, so every unregistered value raised `TypeError` instead of extending the enumeration ([#492](https://github.com/JarryShaw/PyPCAPKit/issues/492), [#503](https://github.com/JarryShaw/PyPCAPKit/pull/503)). -- the string-keyed `get()` in `pcapkit.const.ftp.command` and `pcapkit.const.http.method` tested membership with the raw key but registered `key.upper()`, so the *first* lowercase or mixed-case token raised `TypeError: 'RETR' already in use`. Reachable from wire data: `pcapkit.protocols.application.ftp` matches with `re.I` and passes the match verbatim, and [RFC 959 Section 5](https://datatracker.ietf.org/doc/html/rfc959#section-5) makes FTP commands case-insensitive, so `retr file.txt` was a valid request the library could not parse. `get()` and `_missing_` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate `GET` would have lacked the real one's `safe` and `idempotent` attributes ([#582](https://github.com/JarryShaw/PyPCAPKit/issues/582), [#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). `http.method` no longer folds case: [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1) makes the method token case-sensitive, so `Method.get` matches it exactly ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), above). +- the string-keyed `get()` in `pcapkit.const.ftp.command` and `pcapkit.const.http.method` tested membership with the raw key but registered `key.upper()`, so the *first* lowercase or mixed-case token raised `TypeError: 'RETR' already in use`. Reachable from wire data: `pcapkit.protocols.application.ftp` matches with `re.I` and passes the match verbatim, and [RFC 959 Section 5](https://datatracker.ietf.org/doc/html/rfc959#section-5) makes FTP commands case-insensitive, so `retr file.txt` was a valid request the library could not parse. `get()` and `_missing_` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate `GET` would have lacked the real one's `safe` and `idempotent` attributes ([#582](https://github.com/JarryShaw/PyPCAPKit/issues/582), [#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). `http.method` no longer folds case: [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1) makes the method token case-sensitive, so `Method.get()` matches it exactly ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), above). - `get()`'s documented `default` was ignored on the integer path across the generated `pcapkit.const` tree, because `_missing_` cannot see the caller's `default` -- `Hardware.get(99999, 0)` raised `ValueError` instead of returning `0`. The integer path now consults `default` before letting the error escape. `-1`, the placeholder the generated signature already carried, still separates "no default" from "a default"; [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859) replaced it with an identity sentinel once the shared base [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below) put `get()` behind one call site. The sweep covers the integer registries generated from the shared template, not only those [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584) named; `pcapng` `OptionType` and `reg` `AppType` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix ([#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)). - `_missing_` in the Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). Each now ends in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and most const modules ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s new ones included) already did, so `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. Minting through `extend_enum` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a member per bit pattern. The range guard is untouched ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). - constant registries rejected an invalid value differently from the built-in `enum`, or did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. These now raise a bare `ValueError`, as `enum.IntEnum` does and the other modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in every generated `get()` body (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register; the [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) and [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) entries below record how each came to return an unregistered member instead. The flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed; `Flags` accepts `0 <= value <= 0xFFFF`. A new `tests/const/test_const_enum_builtin_parity.py` asserts that every registry raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how the `IntFlag` ones escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). -- **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) (below) made it an unregistered member. `Transport._make_port` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk**; in 1.4.1, `TCP(srcport=N)` raised `AttributeError` for any bare `int`. Separately, the crawler rendered one `_missing_` branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the branches naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). -- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` was still an `aenum.IntFlag` and a member's `proto` carried the whole set IANA assigned the service, so `tcp | udp` was an ordinary value to pass back in; the `AppType.proto` and `TransportProtocol` entries above record how [#806](https://github.com/JarryShaw/PyPCAPKit/issues/806) and [#808](https://github.com/JarryShaw/PyPCAPKit/issues/808) later retired both. Over every multi-transport member this misrouted port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). +- **a breaking change to** `AppType.get()`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) (below) made it an unregistered member. `Transport._make_port()` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk**; in 1.4.1, `TCP(srcport=N)` raised `AttributeError` for any bare `int`. Separately, the crawler rendered one `_missing_` branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the branches naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). +- **a breaking change to** `AppType._dispatch()`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` was still an `aenum.IntFlag` and a member's `proto` carried the whole set IANA assigned the service, so `tcp | udp` was an ordinary value to pass back in; the `AppType.proto` and `TransportProtocol` entries above record how [#806](https://github.com/JarryShaw/PyPCAPKit/issues/806) and [#808](https://github.com/JarryShaw/PyPCAPKit/issues/808) later retired both. Over every multi-transport member this misrouted port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). - `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. Every generated registry is updated; the non-minting `get()` miss applies to all of them, but the `_missing_` fix changes behaviour only for those with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; the reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a `FILTER_NAME_TO_LINKTYPE` map (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. The other minting call sites in `_missing_`, `get` and the hand-written `mh.py` and `ngap.py` were deferred; the `register`/`register_alias` sites are not deferred but the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). - **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: archived range branches that were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 values that** `_missing_` **resolves (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). - `LinkType(209).name` returned `'IPMB_LINUX'`, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current `I2C_LINUX = 209` and `aenum` gives a value to the member defined first. `pcapkit/vendor/reg/linktype.py` sinks a row whose note mentions "legacy" after every current row, so the current name is defined first; in the registry crawled for this release that affects value 209 alone. `IPMB_LINUX` stays a reachable alias (`LinkType['IPMB_LINUX']` still resolves to 209). Not breaking: only the canonical name for the shared value moves ([#848](https://github.com/JarryShaw/PyPCAPKit/pull/848)). The rule sinks only a row whose value another row also holds, counting values inside ranges, so a current row that merely mentions a legacy name stays first ([#852](https://github.com/JarryShaw/PyPCAPKit/issues/852), [#854](https://github.com/JarryShaw/PyPCAPKit/pull/854)). - **a breaking change to** more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): a final, concrete assigned name mints, and a notation for readers does not) across the `_missing_` files still calling `extend_enum`. Most convert wholly, each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Some are mixed: `EtherType`'s `DEC Unassigned` and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." branches convert, while vendor-company names (Xyplex, Motorola, Walker Richer & Quinn and others) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. Left alone are `CGAType`, whose `Tag_` mint is not an IANA-style range, and the files holding the classes [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) moves onto `EnumRegistry` (the `AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method` and `OptionType` modules; [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) and [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. The worked examples in `docs/source/contributing/conventions/mint-criterion.rst` keep `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). - **a breaking change to** `EtherType`'s `_missing_` ordering: values `0x0101`-`0x01FF` now resolve to their Old Xerox Experimental name instead of an `IEEE802_3_Length_Field_0x0101`-style one. The generated method tested `0x0000 <= value <= 0x05DC` (IEEE802.3 Length Field) first, and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike [#841](https://github.com/JarryShaw/PyPCAPKit/issues/841)'s IPX fix, these bounds come from the live IANA CSV, so the fix is general: `pcapkit/vendor/reg/ethertype.py`'s `process()` collects range rows and a new `EtherType._insert_range` places each ahead of the first already-placed range that fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only containment among the CSV's ranges. Closes [#862](https://github.com/JarryShaw/PyPCAPKit/issues/862) ([#865](https://github.com/JarryShaw/PyPCAPKit/pull/865)). - **a breaking change to** `Method`: `__new__` called `str.__new__(cls)` with no argument, so every registered member's `str` payload was empty regardless of value -- `str(Method.GET) == ''` and `Method.GET == 'GET'` was `False`; `.value` was always correct. [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) (above) had fixed the *unregistered* path only, leaving `str(Method('frob')) == 'frob'` beside `str(Method.GET) == ''`. Fixed to `str.__new__(cls, value)`, mirroring `Command.__new__`, in the generator (`pcapkit/vendor/http/method.py`) and the generated module. No sibling hand-rolled `__new__` has the defect (`OptionType`/`AppType` deliberately store a display string). Consequences, all corrections: `Method.GET == 'GET'` is `True`; `bool(Method.GET)` flips `False` to `True`; `json.dumps(Method.GET)` emits `"GET"` not `""`; `sorted()` is lexicographic rather than input-order-stable; and `{m: ... for m in Method}` gains an entry per member where it had one, because every member previously hashed as `''` and compared equal. Nothing under `pcapkit/protocols` relied on the old behaviour. Closes [#870](https://github.com/JarryShaw/PyPCAPKit/issues/870) ([#871](https://github.com/JarryShaw/PyPCAPKit/pull/871)). -- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), the `AppType` half (the other bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s range branches and `get()`'s own mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that only IANA-registered values are legitimate and that `get()` lacks the information to create new ones. Every branch converts, including those naming a real service assigned to a whole port span (`x11`, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). +- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), the `AppType` half (the other bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s range branches and `get()`'s own mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that only IANA-registered values are legitimate and that `get()` lacks the information to create new ones. Every branch converts, including those naming a real service assigned to a whole port span (`x11`, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get()` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). - closes the remaining scope of [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): `EtherType` and `Socket`, the last registries whose `_missing_` minted a permanent member, now return a non-registering `cls._unregistered_member(value, name)`. The issue's AST census of `extend_enum` in `_missing_`/`get` is down to the single `CGAType` carve-out (its mint is not an IANA-style range). `EtherType(0x0888)` used to add a member to `__members__` and return the identical object on a repeat; it now returns `` with `__members__` unchanged, absent from `_value2member_map_`, equal but not identical across lookups -- likewise `Socket(0x0010)`. The regenerated `EtherType` was not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape `test_const_enum_no_mint.py`'s `is_manufactured()` sweep forbids; rather than weaken the predicate, a narrow exemption covers exactly these files, since the collision the check guards against cannot occur once `_unregistered_member` never registers ([#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)). - `OptionType.register()` and `Command.register()` built members that `get()` could not find again, and `OptionType.register_alias()` raised `not a registered OptionType` for every argument. Both registries now round-trip through `get()`, in the generated files and the vendor templates ([#1311](https://github.com/JarryShaw/PyPCAPKit/pull/1311), [#1330](https://github.com/JarryShaw/PyPCAPKit/issues/1330), [#1369](https://github.com/JarryShaw/PyPCAPKit/pull/1369)). - the Quick-Start function registries rejected 9 to 15, though [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) gives the Function field four bits; the whole 4-bit range now resolves ([#1350](https://github.com/JarryShaw/PyPCAPKit/issues/1350), [#1353](https://github.com/JarryShaw/PyPCAPKit/pull/1353)). - every three-digit FTP reply and HTTP status code resolves, including those no registry row names ([#1443](https://github.com/JarryShaw/PyPCAPKit/issues/1443), [#1444](https://github.com/JarryShaw/PyPCAPKit/pull/1444)). - the `AppType` registry's dict union is defined only on interpreters whose `dict` has one ([#1694](https://github.com/JarryShaw/PyPCAPKit/issues/1694), [#1703](https://github.com/JarryShaw/PyPCAPKit/pull/1703)). - NGAP's `ProcedureCode` and `ProtocolIE` are generated into `pcapkit.const.ngap` from `pycrate`'s compiled NGAP specification, with no network fetch. An unseen value gets a non-registering member, where every unknown key used to mint a member at the shared value `-1` and so aliased every other ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#899](https://github.com/JarryShaw/PyPCAPKit/pull/899)). -- `FEATCode.get` matches case-insensitively, as [RFC 5797 Section 2](https://datatracker.ietf.org/doc/html/rfc5797#section-2) defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold ([#903](https://github.com/JarryShaw/PyPCAPKit/issues/903), [#913](https://github.com/JarryShaw/PyPCAPKit/pull/913)). +- `FEATCode.get()` matches case-insensitively, as [RFC 5797 Section 2](https://datatracker.ietf.org/doc/html/rfc5797#section-2) defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold ([#903](https://github.com/JarryShaw/PyPCAPKit/issues/903), [#913](https://github.com/JarryShaw/PyPCAPKit/pull/913)). - `register` refuses what `get` refuses. `TCP.register(70000, 'x')` and `Hardware.register(70000, 'x')` minted a member that `get` then returned, because no registry overrode the `_validate_value` hook `register` calls. Each generated registry, `AppType` included, now overrides it with the check its `_missing_` makes, raising `EnumValueError`. `get(value, default)` on a value the hook refuses still returns `default` and logs nothing ([#1792](https://github.com/JarryShaw/PyPCAPKit/issues/1792), [#1818](https://github.com/JarryShaw/PyPCAPKit/pull/1818), [#1816](https://github.com/JarryShaw/PyPCAPKit/issues/1816), [#1823](https://github.com/JarryShaw/PyPCAPKit/pull/1823); the quiet miss is [#1845](https://github.com/JarryShaw/PyPCAPKit/issues/1845), [#1873](https://github.com/JarryShaw/PyPCAPKit/pull/1873), under `pcapkit.corekit`). -- `Command.get` refuses a key that is not a `str` with `EnumValueError`, as `Method.get` does. `get(b'\x01')` minted a member keyed by `bytes`, and `get(-1)` raised `AttributeError` whatever its `default` ([#1824](https://github.com/JarryShaw/PyPCAPKit/issues/1824), [#1826](https://github.com/JarryShaw/PyPCAPKit/pull/1826)). +- `Command.get()` refuses a key that is not a `str` with `EnumValueError`, as `Method.get()` does. `get(b'\x01')` minted a member keyed by `bytes`, and `get(-1)` raised `AttributeError` whatever its `default` ([#1824](https://github.com/JarryShaw/PyPCAPKit/issues/1824), [#1826](https://github.com/JarryShaw/PyPCAPKit/pull/1826)). ### pcapkit.corekit @@ -61,7 +61,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `Probe`, `CipherSuite` and `IntegritySuite` are `Info` subclasses rather than `typing.NamedTuple`. They are Mappings now, so `len()` and iteration yield field names rather than values. `TCPSegment` in `pcapkit.toolkit.pcap` stays a `typing.NamedTuple`, since it is built for every TCP frame and an `Info` costs more to construct. - `ModuleDescriptor.klass` reads an already-imported module out of `sys.modules` rather than re-entering `importlib.import_module`, since next layer dispatch resolves a descriptor there on a per-frame path. Neither a registry *hit* holding a `ModuleDescriptor` nor a *miss* writes the resolved class back: a memoised hit went stale on `importlib.reload` ([#1559](https://github.com/JarryShaw/PyPCAPKit/issues/1559)), and recording a miss in a class-level `collections.defaultdict` is the defect [#421](https://github.com/JarryShaw/PyPCAPKit/issues/421)/[#426](https://github.com/JarryShaw/PyPCAPKit/pull/426) fixed at this layer, [#425](https://github.com/JarryShaw/PyPCAPKit/issues/425)/[#428](https://github.com/JarryShaw/PyPCAPKit/pull/428) for the option, chunk and block registries, and [#555](https://github.com/JarryShaw/PyPCAPKit/issues/555)/[#560](https://github.com/JarryShaw/PyPCAPKit/pull/560) at the schema layer. A resolution now costs ~117 ns rather than ~436 ns, which is **not measurable in** `extract()` **wall clock**. The change is taken for its shape: `sys.modules` stays the only module cache, whereas a memo of the class would serve the pre-reload class after an `importlib.reload` forever and fail `isinstance` against the live one. Proposed by `@Ts-Boom` in [#563](https://github.com/JarryShaw/PyPCAPKit/pull/563); the implementation differs because that one added a second, never-invalidated cache of resolved classes ([#574](https://github.com/JarryShaw/PyPCAPKit/issues/574)). - `SeekableReader.truncate()` raises instead of returning a size, and the misspelled `writeable()` is now `writable()`. Both break external callers; nothing inside the package calls either. `io.IOBase` documents one gate over two methods -- "If False, write() and truncate() will raise OSError" -- and this reader's `writable()` answers False, yet `truncate` returned its new size, where `open(path, 'rb').truncate()` raises. The refusal is `UnsupportedOperation('truncate')`, the same one `write` raises; pcapkit's subclasses `io.UnsupportedOperation`. The old resizing is kept as the private `_truncate_buffer`: it resizes a private lookback window, not the stream, and it is the only route to the "position sits before the window" state that `seek` and the buffered read paths must refuse, for which [#643](https://github.com/JarryShaw/PyPCAPKit/issues/643) and [#644](https://github.com/JarryShaw/PyPCAPKit/issues/644) landed tests. `writeable()` was never an override of anything, so `io`, `shutil` and third parties never saw it, and the value reported is unchanged ([#645](https://github.com/JarryShaw/PyPCAPKit/issues/645)). -- **a breaking change to** `@final`, now enforced at runtime on `Info` and `Schema`. `info_final` and `schema_final` both end `return final(cls)`, so every finalised class carried `typing.final`'s `__final__` marker and nothing read it. `@final` **alone on an** `Info` **or** `Schema` **subclass -- without** `@info_final` **or** `@schema_final` **-- now raises** `InfoError` **or** `SchemaError` **at first construction**, the class being marked final but never finalised and so carrying no generated `__init__`. `__init_subclass__` cannot catch that, since `final` applies after class creation, so the check sits in the existing one-shot `FinalisedState.NONE` branch and a finalised class pays nothing. **Deriving from a finalised class raises too**, from new `Info.__init_subclass__` and `Schema.__init_subclass__` hooks (before, only `EnumSchema` had one). `SchemaError` **is a** `ValueError`, so code catching `TypeError` around a subclass declaration will not see it. `@info_final @final` and `@final @info_final` both finalise silently; the re-entry check keys on `__finalised__` rather than `__final__`, because only the former records *the decorator* having run. `@info_final` twice warns and returns the class unchanged. `pcapkit.utilities.compat` takes `final` from `typing_extensions` below 3.11 rather than 3.8, because `typing.final` only records `__final__` from 3.11. The guard cannot fire on the library's own classes ([#778](https://github.com/JarryShaw/PyPCAPKit/issues/778)). +- **a breaking change to** `@final`, now enforced at runtime on `Info` and `Schema`. `info_final` and `schema_final` both end `return final(cls)`, so every finalised class carried `typing.final`'s `__final__` marker and nothing read it. `@final` **alone on an** `Info` **or** `Schema` **subclass -- without** `@info_final` **or** `@schema_final` **-- now raises** `InfoError` **or** `SchemaError` **at first construction**, the class being marked final but never finalised and so carrying no generated `__init__`. `__init_subclass__` cannot catch that, since `final` applies after class creation, so the check sits in the existing one-shot `FinalisedState.NONE` branch and a finalised class pays nothing. **Deriving from a finalised class raises too**, from new `Info.__init_subclass__()` and `Schema.__init_subclass__()` hooks (before, only `EnumSchema` had one). `SchemaError` **is a** `ValueError`, so code catching `TypeError` around a subclass declaration will not see it. `@info_final @final` and `@final @info_final` both finalise silently; the re-entry check keys on `__finalised__` rather than `__final__`, because only the former records *the decorator* having run. `@info_final` twice warns and returns the class unchanged. `pcapkit.utilities.compat` takes `final` from `typing_extensions` below 3.11 rather than 3.8, because `typing.final` only records `__final__` from 3.11. The guard cannot fire on the library's own classes ([#778](https://github.com/JarryShaw/PyPCAPKit/issues/778)). - **a breaking change to** `Info.to_dict()`: it returns an `OrderedMultiDict` at every level rather than a `dict`, so a key that repeats, such as a second IPv6 extension header of one type, survives a `from_dict(to_dict())` rebuild. `type(d) is dict` no longer holds ([#1453](https://github.com/JarryShaw/PyPCAPKit/issues/1453), [#1466](https://github.com/JarryShaw/PyPCAPKit/pull/1466), [#1484](https://github.com/JarryShaw/PyPCAPKit/issues/1484), [#1523](https://github.com/JarryShaw/PyPCAPKit/pull/1523)). - **a breaking change to** `pcapkit.utilities.decorators`: it moves to `pcapkit.corekit.decorators`, with no re-export. `seekset`, `beholder` and `prepare` are no longer importable from `pcapkit.utilities`, and the module logs as `pcapkit.corekit.decorators` ([#1735](https://github.com/JarryShaw/PyPCAPKit/issues/1735), [#1737](https://github.com/JarryShaw/PyPCAPKit/pull/1737)). - **a breaking change to** the sentinels: all live in `pcapkit.corekit.sentinels`, which exports the objects `NULL`, `NO_VALUE` and `NO_DEFAULT` and their common base `Sentinel`, but not their types, and `NoValue` is spelled `NO_VALUE` ([#916](https://github.com/JarryShaw/PyPCAPKit/pull/916), [#911](https://github.com/JarryShaw/PyPCAPKit/issues/911), [#922](https://github.com/JarryShaw/PyPCAPKit/pull/922), [#937](https://github.com/JarryShaw/PyPCAPKit/issues/937), [#939](https://github.com/JarryShaw/PyPCAPKit/pull/939)). No moved name keeps a re-export, so `NoValueType` has only that home, not its 1.4.x one in `pcapkit.corekit.fields.field` ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). @@ -69,21 +69,21 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Fixed - stdlib exceptions leaking out where the library's own were promised: a malformed IP field value raised a bare `ValueError` instead of `FieldValueError` ([#465](https://github.com/JarryShaw/PyPCAPKit/issues/465)); a `bool` address was silently packed as `0.0.0.1` or `0.0.0.0`, `bool` being an `int` subclass ([#491](https://github.com/JarryShaw/PyPCAPKit/issues/491), [#500](https://github.com/JarryShaw/PyPCAPKit/pull/500)); and `@prepare` discarded extra arguments silently and treated a *declared* zero length as end of stream, which is now distinguished from a genuinely exhausted one and raises `StreamEOFError` ([#454](https://github.com/JarryShaw/PyPCAPKit/issues/454), [#458](https://github.com/JarryShaw/PyPCAPKit/issues/458)). -- `FieldBase.unpack` zero-padded up to a field's declared `length` with `rjust()` however little data `buffer` held; a ~40-octet PCAP-NG Decryption Secrets Block with a bogus inner length forced a multi-gigabyte allocation, since `length` is often wire-derived. A declared length past 262144 octets -- libpcap's `MAXIMUM_SNAPLEN` and this package's default `snaplen` -- that the buffer cannot back now raises `FieldValueError`; the option and list loops' tolerance for a short read past a truncated area ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) is far under that ceiling and untouched ([#554](https://github.com/JarryShaw/PyPCAPKit/issues/554)). -- that ceiling bounds one field, and a packet holds many, so the *sum* of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring `secrets_length` 262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing. `FieldBase.unpack` now keeps a per-context ledger of octets supplied against octets synthesised, and raises `FieldValueError` once the total of shortfalls *past 65,536 octets* passes `262144 + 16 * supplied`. The same 200 blocks retain 0.2 MiB (54.6x). A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded unconditionally and charged to nothing, so no snapshot-truncated capture is subject to the budget; without that band the same legitimate frame parsed differently depending on what had been parsed before. **What this does not close**: a length declared by a 16-bit field can still be repeated without limit, and at this layer a crafted file is indistinguishable from a legitimate snapshot-truncated capture of offload-sized segments; separating them needs the frame's own `incl_len`/`orig_len`. Nothing resets the ledger, so it bounds everything a context has parsed rather than one file ([#573](https://github.com/JarryShaw/PyPCAPKit/issues/573)). -- which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised `ProtocolError` as documented, but a process that had popped `pcapkit.corekit.fields.misc` from `sys.modules` (as the [#439](https://github.com/JarryShaw/PyPCAPKit/issues/439) ABC-cache regression tests do in every `setUp`/`tearDown`) raised `FieldValueError` from a different layer, before the documented check ([#525](https://github.com/JarryShaw/PyPCAPKit/issues/525)). `ListField.unpack` resolved `SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so `isinstance` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** `FieldValueError` **now gets** `ProtocolError`, deterministically, matching the docstring. +- `FieldBase.unpack()` zero-padded up to a field's declared `length` with `rjust()` however little data `buffer` held; a ~40-octet PCAP-NG Decryption Secrets Block with a bogus inner length forced a multi-gigabyte allocation, since `length` is often wire-derived. A declared length past 262144 octets -- libpcap's `MAXIMUM_SNAPLEN` and this package's default `snaplen` -- that the buffer cannot back now raises `FieldValueError`; the option and list loops' tolerance for a short read past a truncated area ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) is far under that ceiling and untouched ([#554](https://github.com/JarryShaw/PyPCAPKit/issues/554)). +- that ceiling bounds one field, and a packet holds many, so the *sum* of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring `secrets_length` 262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing. `FieldBase.unpack()` now keeps a per-context ledger of octets supplied against octets synthesised, and raises `FieldValueError` once the total of shortfalls *past 65,536 octets* passes `262144 + 16 * supplied`. The same 200 blocks retain 0.2 MiB (54.6x). A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded unconditionally and charged to nothing, so no snapshot-truncated capture is subject to the budget; without that band the same legitimate frame parsed differently depending on what had been parsed before. **What this does not close**: a length declared by a 16-bit field can still be repeated without limit, and at this layer a crafted file is indistinguishable from a legitimate snapshot-truncated capture of offload-sized segments; separating them needs the frame's own `incl_len`/`orig_len`. Nothing resets the ledger, so it bounds everything a context has parsed rather than one file ([#573](https://github.com/JarryShaw/PyPCAPKit/issues/573)). +- which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised `ProtocolError` as documented, but a process that had popped `pcapkit.corekit.fields.misc` from `sys.modules` (as the [#439](https://github.com/JarryShaw/PyPCAPKit/issues/439) ABC-cache regression tests do in every `setUp`/`tearDown`) raised `FieldValueError` from a different layer, before the documented check ([#525](https://github.com/JarryShaw/PyPCAPKit/issues/525)). `ListField.unpack()` resolved `SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so `isinstance` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** `FieldValueError` **now gets** `ProtocolError`, deterministically, matching the docstring. - a `NumberField` whose `length` was a callable could not pack or parse at any width `struct` has a native integer code for. `length` is `-1` until the callable resolves, `-1` has no native code, and the template builder raised `_need_process` for it and never cleared it -- a latch. Resolving the real width rebuilt the template but left the latch set, so `pre_process` handed `bytes` to a template that had become `>Q` (`struct.error: required argument is not an integer`), and parsing failed in mirror by calling `int.from_bytes` on an integer `struct.unpack` had produced. The flag is now recomputed from the width in force, which also keeps a callable resolving to a width with no native code (3 octets) byte-packed. **Every native width (1, 2, 4 and 8 octets) was affected, not only the 8 reported**, on `NumberField` and `EnumField` alike. This made every extended 8-octet MPTCP DSS form unbuildable, those widths being chosen at runtime from the DSS flags; [#585](https://github.com/JarryShaw/PyPCAPKit/pull/585) worked around it in the TCP schema alone ([#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)). -- `NumberField.pre_process` sized a value with floor division dressed as a ceiling. When a field is packed while `length` is still the `-1` placeholder, the width came from `math.ceil(value.bit_length() // 8)`; the `//` had already floored, so `math.ceil` did nothing and the width was one octet short: `256` at one octet, `65536` at two, `16777216` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so `1` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s suite, whose values all had byte-aligned bit lengths, missed it; [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack `128`. The same typo in the ILNP nonce option builders is a separate change ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). -- `FieldBase.unpack` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) -- the shortfall was zero-filled with `rjust()`, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now `ljust()`, correct for **both** byte orders: one octet of a four-octet little-endian `120` read as `2013265920`, and three octets of a four-octet big-endian `0x01020304` as `0x10203`. A full read is untouched ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but separate defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Further contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). -- the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A companion `bounded_area` clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). +- `NumberField.pre_process()` sized a value with floor division dressed as a ceiling. When a field is packed while `length` is still the `-1` placeholder, the width came from `math.ceil(value.bit_length() // 8)`; the `//` had already floored, so `math.ceil` did nothing and the width was one octet short: `256` at one octet, `65536` at two, `16777216` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so `1` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s suite, whose values all had byte-aligned bit lengths, missed it; [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack `128`. The same typo in the ILNP nonce option builders is a separate change ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). +- `FieldBase.unpack()` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) -- the shortfall was zero-filled with `rjust()`, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now `ljust()`, correct for **both** byte orders: one octet of a four-octet little-endian `120` read as `2013265920`, and three octets of a four-octet big-endian `0x01020304` as `0x10203`. A full read is untouched ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- `SeekableReader.truncate()` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but separate defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Further contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). +- the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A companion `bounded_area` clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack()` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). - **a breaking change to** `FieldBase.length`: it called `struct.calcsize` on a template built from a negative resolved length (e.g. `'-5s'`, from a `length=` callback whose running counter had been overdrawn), raising a bare `struct.error` that ordinary caller code does not catch. The chokepoint every affected schema shares now raises `ProtocolError`, covering `application/{ftp,httpv1,httpv2,ngap}.py`, `internet/{hip,ipv6_route,mh}.py`, `link/ethernet.py`, `misc/pcapng.py` and `transport/sctp.py` with no other file changed; a non-negative length is unaffected. This closes the follow-on the `httpv2` length-guard entry under `pcapkit.protocols` filed as out of scope ([#805](https://github.com/JarryShaw/PyPCAPKit/issues/805)). - **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the owner's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it first converts registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and the `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those: the other const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). These also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other registries ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). - **a breaking change to** `EnumRegistry.register()`, extended: the const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype` among them) are left alone, so **every other class** inherits `EnumRegistry` ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the owner's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). -- `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). +- `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_()` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). - **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but each `cls(default)` call site reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, each site now resolves `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). -- `ProtoChain.__add__` returned the left operand's cached `protocols` and `aliases`: once `a.protocols` had been read, `a + ProtoChain(IPv4)` reported `chain == 'Ethernet:IPv4'` but `protocols == (Ethernet,)`. `copy.copy(self)` carried the cache across; the result is now built fresh, as `from_list` builds one, and `+=` follows, since it falls back to `__add__` ([#1099](https://github.com/JarryShaw/PyPCAPKit/issues/1099)). -- a `SchemaField` with no declared length handed its `-1` placeholder to the nested schema as that schema's length, so every item of such a list warned `packet length < 0`: a HIP `LOCATOR_SET` and an MH CGA Parameters item among them. `SchemaField.unpack` now passes what is left in the buffer as a declared length -- not `None`, which would turn a truncated item's `FieldValueError` into `prepare()`'s quiet `StreamEOFError`. Fixed-length items are unchanged; the HTTP/2 `SETTINGS` pair, whose size is fixed, is given it in its own schema instead ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140), below) ([#1148](https://github.com/JarryShaw/PyPCAPKit/issues/1148)). +- `ProtoChain.__add__()` returned the left operand's cached `protocols` and `aliases`: once `a.protocols` had been read, `a + ProtoChain(IPv4)` reported `chain == 'Ethernet:IPv4'` but `protocols == (Ethernet,)`. `copy.copy(self)` carried the cache across; the result is now built fresh, as `from_list` builds one, and `+=` follows, since it falls back to `__add__` ([#1099](https://github.com/JarryShaw/PyPCAPKit/issues/1099)). +- a `SchemaField` with no declared length handed its `-1` placeholder to the nested schema as that schema's length, so every item of such a list warned `packet length < 0`: a HIP `LOCATOR_SET` and an MH CGA Parameters item among them. `SchemaField.unpack()` now passes what is left in the buffer as a declared length -- not `None`, which would turn a truncated item's `FieldValueError` into `prepare()`'s quiet `StreamEOFError`. Fixed-length items are unchanged; the HTTP/2 `SETTINGS` pair, whose size is fixed, is given it in its own schema instead ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140), below) ([#1148](https://github.com/JarryShaw/PyPCAPKit/issues/1148)). - a signed `NumberField` decoded as unsigned, a zero-length `PayloadField` misread, and an empty `Raw` did not rebuild ([#1331](https://github.com/JarryShaw/PyPCAPKit/pull/1331)). - `from_dict(to_dict())` lost nested values, and `len()` of an `Info` counted its hidden keys ([#1334](https://github.com/JarryShaw/PyPCAPKit/pull/1334)). - `StringField` packs back exactly the octets it unpacked, where a guessed charset, a `U+FFFD` substitution or a character-count length used to change them; the decoded text is unchanged ([#1293](https://github.com/JarryShaw/PyPCAPKit/issues/1293), [#1336](https://github.com/JarryShaw/PyPCAPKit/pull/1336)). A charset that rejects `'replace'` falls back to UTF-8 ([#1706](https://github.com/JarryShaw/PyPCAPKit/issues/1706), [#1728](https://github.com/JarryShaw/PyPCAPKit/pull/1728)), and Python 3.8 unquotes through `pcapkit.utilities.compat.unquote` ([#1693](https://github.com/JarryShaw/PyPCAPKit/issues/1693), [#1707](https://github.com/JarryShaw/PyPCAPKit/pull/1707)). @@ -97,8 +97,8 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - an `*IntField` given a `signed=` that contradicts its fixed sign, such as `UInt8Field(signed=True)`, raises `FieldValueError`; the argument used to be discarded silently ([#545](https://github.com/JarryShaw/PyPCAPKit/issues/545), [#549](https://github.com/JarryShaw/PyPCAPKit/pull/549)). - negative lengths raise `ProtocolError` rather than a bare stdlib error: `FieldBase.length` tells a malformed template from a negative resolved length ([#825](https://github.com/JarryShaw/PyPCAPKit/issues/825), [#827](https://github.com/JarryShaw/PyPCAPKit/pull/827)), and `NumberField` checks a negative resolved `length` and a negative `bit_length` ([#828](https://github.com/JarryShaw/PyPCAPKit/issues/828), [#829](https://github.com/JarryShaw/PyPCAPKit/pull/829), [#831](https://github.com/JarryShaw/PyPCAPKit/issues/831), [#834](https://github.com/JarryShaw/PyPCAPKit/pull/834)). - `ModuleDescriptor.klass` raises `ProtocolError` for a class name that resolves to nothing, or one never given, rather than a bare `AttributeError`; the registry modules share one `NULL` sentinel compared by identity, where each had its own string ([#832](https://github.com/JarryShaw/PyPCAPKit/issues/832), [#833](https://github.com/JarryShaw/PyPCAPKit/issues/833), [#835](https://github.com/JarryShaw/PyPCAPKit/pull/835)). -- `PortEnumField` given a `proto` with no `.name` -- a string such as `'tcp'`, or a composite such as `TransportProtocol.tcp | TransportProtocol.udp`, which is a bare `int` -- raised `AttributeError` on a port the registry does not hold. The miss takes its transport from the registry's `__transport__`, as `AppType.get` does ([#1774](https://github.com/JarryShaw/PyPCAPKit/issues/1774), [#1781](https://github.com/JarryShaw/PyPCAPKit/pull/1781)). -- `EnumLookup.get` answers a value miss quietly, whether `default` answers it or the miss is raised: a registry's `_validate_value` rejection logs nothing and installs no exception hook there, while `EnumRegistry.register` still reports the same rejection loudly ([#1845](https://github.com/JarryShaw/PyPCAPKit/issues/1845), [#1873](https://github.com/JarryShaw/PyPCAPKit/pull/1873)). +- `PortEnumField` given a `proto` with no `.name` -- a string such as `'tcp'`, or a composite such as `TransportProtocol.tcp | TransportProtocol.udp`, which is a bare `int` -- raised `AttributeError` on a port the registry does not hold. The miss takes its transport from the registry's `__transport__`, as `AppType.get()` does ([#1774](https://github.com/JarryShaw/PyPCAPKit/issues/1774), [#1781](https://github.com/JarryShaw/PyPCAPKit/pull/1781)). +- `EnumLookup.get()` answers a value miss quietly, whether `default` answers it or the miss is raised: a registry's `_validate_value` rejection logs nothing and installs no exception hook there, while `EnumRegistry.register()` still reports the same rejection loudly ([#1845](https://github.com/JarryShaw/PyPCAPKit/issues/1845), [#1873](https://github.com/JarryShaw/PyPCAPKit/pull/1873)). - each sentinel is a singleton, as `None` is: calling its type, `copy`, `deepcopy` and `pickle` at every protocol give back the one object, and `NO_VALUE` reprs as its own name rather than as an object address. Their new common base, `Sentinel`, is public; a sentinel's type cannot be subclassed, and neither it nor the sentinel takes an attribute write. A pickle written before this change at protocol 0 or 1 still loads, though not as the singleton ([#1841](https://github.com/JarryShaw/PyPCAPKit/issues/1841), [#1842](https://github.com/JarryShaw/PyPCAPKit/issues/1842), [#1876](https://github.com/JarryShaw/PyPCAPKit/pull/1876)). ### pcapkit.dumpkit @@ -119,7 +119,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Added - the `engine='pypcap'` and `engine='pcap_ct'` extraction engines, independent distributions of the same `libpcap` interface, and `engine='pypcapfile'` ([#386](https://github.com/JarryShaw/PyPCAPKit/pull/386), [#405](https://github.com/JarryShaw/PyPCAPKit/pull/405)). They buy speed by doing less -- neither `pypcap` nor `pcap_ct` dissects at all, so they offer no reassembly or flow tracing, and `pypcapfile` has no IPv6 decoder. Install only **one** of `pypcap` and `pcap-ct`: both own the top-level `pcap` module, and with both present `pcap-ct` wins the import and the other becomes unselectable. The missing interface constants `PyPCAP`, `PCAP_CT` and `PyPCAPFile` are now exported alongside `DPKT`, `Scapy`, `PyShark` and `PCAPKit` ([#412](https://github.com/JarryShaw/PyPCAPKit/pull/412)). Python 3.11 is the last interpreter on which every built-in engine can run, and even there `pypcap` and `pcap-ct` cannot coexist. -- `EngineBase.unsupported_reason`, a preflight every engine answers and `Extractor.run` consults before anything is imported. An engine that cannot run in the current environment now gives one warning naming the real cause (a Python version, a missing `tshark` or `libpcap`, the wrong `pcap` distribution) and a clean fall back to `pcapkit`'s own parser, rather than an error from inside the third-party package ([#396](https://github.com/JarryShaw/PyPCAPKit/pull/396), [#405](https://github.com/JarryShaw/PyPCAPKit/pull/405)). The same holds for an engine whose package is not installed at all ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045)). +- `EngineBase.unsupported_reason()`, a preflight every engine answers and `Extractor.run()` consults before anything is imported. An engine that cannot run in the current environment now gives one warning naming the real cause (a Python version, a missing `tshark` or `libpcap`, the wrong `pcap` distribution) and a clean fall back to `pcapkit`'s own parser, rather than an error from inside the third-party package ([#396](https://github.com/JarryShaw/PyPCAPKit/pull/396), [#405](https://github.com/JarryShaw/PyPCAPKit/pull/405)). The same holds for an engine whose package is not installed at all ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045)). - `conflict` on the reassembly data models: absolute, inclusive ranges where two fragments claimed the same span with different bytes, previously lost silently on both the IP ([#482](https://github.com/JarryShaw/PyPCAPKit/pull/482)) and TCP ([#443](https://github.com/JarryShaw/PyPCAPKit/issues/443), [#478](https://github.com/JarryShaw/PyPCAPKit/pull/478)) paths. #### Changed @@ -129,9 +129,9 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - conflicting TCP overlaps resolve first-write-wins, per [RFC 9293](https://datatracker.ietf.org/doc/html/rfc9293) section 3.10, where they had silently resolved last-write-wins ([#443](https://github.com/JarryShaw/PyPCAPKit/issues/443), [#478](https://github.com/JarryShaw/PyPCAPKit/pull/478)). A deliberate, narrow behaviour break: a conforming retransmission carries identical bytes. IP fragment reassembly keeps last-write-wins, because [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) specifies the opposite resolution, and records the disagreement instead ([#482](https://github.com/JarryShaw/PyPCAPKit/pull/482)). - renames with no compatibility alias left behind: `HoleDiscriptor` is spelled `HoleDescriptor` and its package alias `TCP_HoleDiscriptor` is `TCP_HoleDescriptor` ([#350](https://github.com/JarryShaw/PyPCAPKit/pull/350)). - subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` ([#514](https://github.com/JarryShaw/PyPCAPKit/issues/514)). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias, which every built-in does (hence no public class had a subclass, while every built-in derives from its `*Base` class). **This breaks out-of-tree code that subclasses one of these classes and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Silent failures are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` (passing `name=` to a `Reassembly` subclass silently ignored the key, since `protocol=` is the real one), and so does `Dumper`'s `ext=` without `fmt=`. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass gained a class-level `registry` property mirroring `EnumSchema.registry`, and a `Dumper` subclass no longer touches the filesystem while its `class` statement runs. `Engine`'s keyword is `engine=` rather than the `name=` first shipped, because `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s parameters on Python 3.10 and raise `TypeError` before the hook runs. There is no `name=` alias: a keyword that works on some interpreters and not others is the trap being removed. -- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (every built-in derives from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its sibling internal registrars check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched ([#1020](https://github.com/JarryShaw/PyPCAPKit/pull/1020)). +- **a breaking change to** `Extractor.register_engine()`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (every built-in derives from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its sibling internal registrars check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched ([#1020](https://github.com/JarryShaw/PyPCAPKit/pull/1020)). - extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output ([#420](https://github.com/JarryShaw/PyPCAPKit/pull/420)). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% (IP reassembly submits a datagram for every frame, fragmented or not) ([#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow dumper handed each record to a `Frame` constructor that re-dissected the whole stack to return the bytes it had just been given; options are no longer parsed twice either ([#427](https://github.com/JarryShaw/PyPCAPKit/pull/427)). -- **a breaking change to** `Extractor.engine` and `Extractor.record_header`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. The built-in engines subclass `EngineBase` directly, so none was ever an `Engine`, which adds only the `__init_subclass__` registration hook. A type checker sees the change: code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). +- **a breaking change to** `Extractor.engine` and `Extractor.record_header()`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. The built-in engines subclass `EngineBase` directly, so none was ever an `Engine`, which adds only the `__init_subclass__` registration hook. A type checker sees the change: code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). - the documentation of the engine base classes: the member stubs (`name`, `module`, `registry`, `extractor`, `unsupported_reason`, `run`, `read_frame`, `close` and the rest) moved from `Engine` to `EngineBase`, where they are defined, and `Engine` now documents only the `__init_subclass__` registration hook it adds, so the `EngineBase[_P]` that `Extractor.engine` returns leads to an API. Cross-references to `Engine.run`, `Engine.read_frame`, `Engine.close` and `Engine.unsupported_reason` now point at `EngineBase`, which stays out of `__all__`, as [#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016) settled ([#1024](https://github.com/JarryShaw/PyPCAPKit/issues/1024)). - **a breaking change to** `register_mh_extension`: `schema` is keyword-only, as in the other schema-taking registrars ([#516](https://github.com/JarryShaw/PyPCAPKit/issues/516), [#526](https://github.com/JarryShaw/PyPCAPKit/pull/526)). @@ -141,15 +141,15 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `format='text'` raised `AttributeError` before writing anything, naming a `dictdumper.Text` that has never existed. It now points at `Tree`, as the `'txt'` alias beside it already did. - `Extractor` had the ownership of its input stream inverted: a handle it opened itself, from `fin` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. `_cleanup` closed under `not self._flag_s` (the flag that gated the `open()`), and the `SeekableReader` wrapping a non-seekable input asked for `stream_closing=not self._flag_s` in the same wrong direction. The leak was one descriptor per extraction, the root cause of the `ResourceWarning` that reddened [#577](https://github.com/JarryShaw/PyPCAPKit/pull/577), [#596](https://github.com/JarryShaw/PyPCAPKit/pull/596) and [#600](https://github.com/JarryShaw/PyPCAPKit/pull/600) ([#606](https://github.com/JarryShaw/PyPCAPKit/issues/606) had only made the test immune). Both conditions now follow ownership via a single `Extractor._owns_input`, and a finaliser releases the handle of an extraction *abandoned* before end of file (`auto=False`, iterated part way and dropped); the files [#610](https://github.com/JarryShaw/PyPCAPKit/issues/610) names emit no such warning. `__exit__` deliberately still closes unconditionally, since a caller who scopes an `Extractor` with `with` asked for that ([#610](https://github.com/JarryShaw/PyPCAPKit/issues/610)). - `extract(..., no_eof=True)` never returned. End of stream was detected and `ExtractionWarning: EOF reached` fired, but every loop that handles it (`record_frames`, `__next__`, `__call__`) read `if self._flag_n: continue`, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- `__main__` sets `no_eof` for `fin='-'`, and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one: `prepare` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (`Extractor._note_eof_progress`). A pipe is unaffected, since a read there *blocks* while the writer is open but idle. This also fixes `pcapkit -`, which hung on **any** finished stdin. **One deliberate narrowing**: a *seekable* input does not block, so a file still being appended to ends at the data present when the extraction reached it; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own. Pre-existing and untouched: an append landing *mid-record* raises `ValueError: read length must be non-negative or -1`. The end-to-end tests are bounded by a **child process** rather than `tests._support.time_limit`, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than a red test. Guesses at the cause are ruled out: the parse path's `except Exception` handlers are never entered during the spin, and the memory is the retry loop's `EOF reached` records, which the runner retains ([#620](https://github.com/JarryShaw/PyPCAPKit/issues/620)). -- `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and several dispatchable classes are named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The guard deliberately reads "key present and the incumbent is a different class", not mere presence ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave its sibling registrars the same identity guard): this function is the funnel every wrapper registrar ends in, so registering one class under two codes (`register_tcp` then `register_udp`, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514). The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). +- `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and several dispatchable classes are named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__()` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The guard deliberately reads "key present and the incumbent is a different class", not mere presence ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave its sibling registrars the same identity guard): this function is the funnel every wrapper registrar ends in, so registering one class under two codes (`register_tcp` then `register_udp`, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514). The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). - the `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` that named a bare `Type`, which Sphinx resolves against every class named `Type` in the project rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. These sites now spell it `~typing.Type[Dumper]`, among the sites [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) lists. The other bare `Type` sites, in hand-written `:type:` fields under `docs/source/pcapkit/foundation/`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). - `register_protocol`'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (`incumbent is not protocol`, an identity check from [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)), but the message built both operands with a bare `repr()`. A factory defining a same-named closure-local class on every call (as `tests/protocols/test_construction_keyword_check_unit.py`'s `_protocol_class` does) produces two distinct classes sharing one `__module__` and `__qualname__`, so a real overwrite read "overwriting X with X." The message now compares the two reprs and, only when they coincide, appends each object's `id()`; `__module__`/`__qualname__` would not disambiguate, being what the repr already renders ([#710](https://github.com/JarryShaw/PyPCAPKit/issues/710)). - more registrars warned on mere key presence rather than an actual overwrite, outside the wording of [#718](https://github.com/JarryShaw/PyPCAPKit/issues/718)'s identity guard (`incumbent is not None and incumbent is not new`, landed by [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)), whose issue named only code-keyed registrars: `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, and `register_dumper` on both `Extractor` and `TraceFlow`. Each now compares the incumbent by identity before warning; both `register_dumper` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings ([#739](https://github.com/JarryShaw/PyPCAPKit/issues/739)). -- **the class registrars listed below** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the `register_dumper` methods, and from inside `abc` for the others. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in those that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocol `register` classmethods is fixed separately by [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). -- a missing extraction engine package now gives a single `EngineWarning` rather than a duplicate ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045), [#1046](https://github.com/JarryShaw/PyPCAPKit/pull/1046)). `Extractor.import_test` warned that the package was absent, then `Extractor.run` warned again when it fell back to the default engine; `run` now stays silent there, and `import_test` says what `run` used to: `engine () is not installed; using default engine instead`, the module name in backticks. Only code that matched `import_test`'s old wording, or that expects the duplicate warning, breaks. +- **the class registrars listed below** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the `register_dumper` methods, and from inside `abc` for the others. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper()` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol()`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in those that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocol `register` classmethods is fixed separately by [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). +- a missing extraction engine package now gives a single `EngineWarning` rather than a duplicate ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045), [#1046](https://github.com/JarryShaw/PyPCAPKit/pull/1046)). `Extractor.import_test()` warned that the package was absent, then `Extractor.run()` warned again when it fell back to the default engine; `run` now stays silent there, and `import_test` says what `run` used to: `engine () is not installed; using default engine instead`, the module name in backticks. Only code that matched `import_test`'s old wording, or that expects the duplicate warning, breaks. - `Extractor` defects: `Extractor.output`'s `UnsupportedCall` named `format` rather than `output`; `make_name` looked an unknown format up in the `__output__` `defaultdict` and so inserted it; and the Scapy engine never set `_offmt`, so `Extractor.format` raised `AttributeError` under `engine='scapy'` ([#1095](https://github.com/JarryShaw/PyPCAPKit/issues/1095)). - the DPKT, Scapy and PyShark engines call `Extractor.record_header()` at setup again, as they did until 2023, so the global-header step is shared with the default engine and sets `_offmt` for each of them. **Their dumped output changes**: JSON, plist and tree reports gain the leading `Global Header` record (`Section Header 1` for PCAP-NG) the default engine already writes, identical to its, and `files=True` writes that record's file too; frame records are unchanged. `Extractor.format` now also answers on a capture with no frames. `format='pcap'` failing on every engine is a separate defect, [#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127) (below) ([#1104](https://github.com/JarryShaw/PyPCAPKit/issues/1104)). -- `format='pcap'` failed on every engine with a `TypeError` from `PCAPIO.__init__()`, missing `protocol`: `Extractor.__init__` created the writer before any global header was read, so `PCAPIO` had no link type. The header step now creates it through `Extractor._open_output`, and `run()` opens it for engines with no such step; a PCAP writer takes the frame itself and skips the `Global Header` record, so the default engine writes `in.pcap` back byte for byte. The third-party engines hand the writer no frame octets, so they fall back to JSON with a `FormatWarning`, as their flow tracing already does, and a PCAP-NG input raises `FormatError` ([#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127)). The written global header also dropped the input's `thiszone`, `sigfigs` and `snaplen`, so most sample PCAP files did not come back byte for byte; the PCAP engine now passes those fields through `_open_output` to `PCAPIO`, and every one does ([#1165](https://github.com/JarryShaw/PyPCAPKit/issues/1165)). +- `format='pcap'` failed on every engine with a `TypeError` from `PCAPIO.__init__()`, missing `protocol`: `Extractor.__init__()` created the writer before any global header was read, so `PCAPIO` had no link type. The header step now creates it through `Extractor._open_output()`, and `run()` opens it for engines with no such step; a PCAP writer takes the frame itself and skips the `Global Header` record, so the default engine writes `in.pcap` back byte for byte. The third-party engines hand the writer no frame octets, so they fall back to JSON with a `FormatWarning`, as their flow tracing already does, and a PCAP-NG input raises `FormatError` ([#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127)). The written global header also dropped the input's `thiszone`, `sigfigs` and `snaplen`, so most sample PCAP files did not come back byte for byte; the PCAP engine now passes those fields through `_open_output` to `PCAPIO`, and every one does ([#1165](https://github.com/JarryShaw/PyPCAPKit/issues/1165)). - IP reassembly reports only the octets it holds: a truncated fragment, a strict run and a zero-length datagram no longer report octets from the declared Total Length ([#1368](https://github.com/JarryShaw/PyPCAPKit/pull/1368)). Only the final fragment's partial block counts, and a datagram too long for its length field never completes ([#1582](https://github.com/JarryShaw/PyPCAPKit/pull/1582), [#1585](https://github.com/JarryShaw/PyPCAPKit/issues/1585), [#1598](https://github.com/JarryShaw/PyPCAPKit/pull/1598)). - TCP reassembly judges completion from each buffer's own gaps, compares sequence numbers modulo `2**32`, and holds a FIN until reordered data arrives ([#1375](https://github.com/JarryShaw/PyPCAPKit/pull/1375)). A TSO fragment's length is its captured length, and `RCVBT` covers the last block ([#1564](https://github.com/JarryShaw/PyPCAPKit/pull/1564)). - a second TCP flow with the same label truncated the first flow's file; a reused label now gets the next free suffix ([#1360](https://github.com/JarryShaw/PyPCAPKit/issues/1360), [#1371](https://github.com/JarryShaw/PyPCAPKit/pull/1371)). @@ -175,10 +175,10 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) ([#251](https://github.com/JarryShaw/PyPCAPKit/discussions/251), [#417](https://github.com/JarryShaw/PyPCAPKit/pull/417)). Decoding is generic by ASN.1 shape rather than per-procedure, so every elementary procedure and protocol IE works and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` was at first left out of the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module -- until [#910](https://github.com/JarryShaw/PyPCAPKit/issues/910) counted it among `all`'s core addons, so `all` installs it (the `all` entry under *Project infrastructure*). - the Mobility Header registry, completed ([#383](https://github.com/JarryShaw/PyPCAPKit/pull/383), [#437](https://github.com/JarryShaw/PyPCAPKit/pull/437)): the [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options, then every registered message data type, every registered option (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families) and every CGA extension. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the CGA extensions round-trip end to end. - dispatch entries for dissectors that existed but were reachable from no registry ([#436](https://github.com/JarryShaw/PyPCAPKit/pull/436)): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, `L2TPv2` on UDP 1701 and OSPF at `TransType` 89. `VLAN` became an abstract base with `C_Tag` (802.1Q) and `S_Tag` (802.1ad) as concrete subclasses, so a Q-in-Q frame no longer collapses into one opaque `Raw`; `L2TP` likewise became a base, with `L2TPv2` carrying the [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) implementation. -- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other registrar families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) needs; [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) turned out to need a missing class instead (115 is [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3 over IP, not [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661); see the corresponding **Fixed** entry below), so its worked example names `L2TPv3` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). +- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other registrar families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code()`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) needs; [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) turned out to need a missing class instead (115 is [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3 over IP, not [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661); see the corresponding **Fixed** entry below), so its worked example names `L2TPv3` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). - `tests/protocols/test_dispatch_reachability_unit.py`, the coverage [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) asked for: every `ProtocolBase` descendant whose `__index__` returns an enum member must be reachable under that code in the registry its enum *type* designates, read from the same `_CODE_DESTINATIONS` table backing `code=` so the two cannot drift. It walks the classes rather than the registry entries, so it catches a class nothing registered, the shape in which `OSPF` once shipped ([#548](https://github.com/JarryShaw/PyPCAPKit/issues/548)). - `tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py`, covering all three MP_JOIN layouts through the *public* constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the statement order, and controls for the parse path and the flag-independent options. The old cases bypassed the public path, so full coverage coexisted with a broken one. The stale `tcp-mptcp/MP_JOIN` entry is deleted from `EXPECTED_FAILURES` ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). -- `tests/protocols/test_option_generator_tcp_base_unit.py`, asserting that every `TCP_BASE` key is a parameter `TCP.make` declares (derived from `inspect.signature`) and that the built segment carries the stated header in both data model and packed octets ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). +- `tests/protocols/test_option_generator_tcp_base_unit.py`, asserting that every `TCP_BASE` key is a parameter `TCP.make()` declares (derived from `inspect.signature`) and that the built segment carries the stated header in both data model and packed octets ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). - ILNP nonce sizing coverage in `tests/protocols/internet/test_ipv6_extension_unit.py`, a test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over a spread of nonces, most with a bit length that is *not* a multiple of eight, where floor division and the ceiling disagree ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). - `SOLUTION` parameter width coverage in `tests/protocols/internet/test_hip_unit.py`: a spread of widths, each asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) requires of HIPv1. Some widths are deliberately *not* multiples of eight, where the defective `ceil(bits / 4)` differs from the correct width ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). - the BSD loopback layer for `LINKTYPE_NULL` and `LINKTYPE_LOOP`, and dissection of `LINKTYPE_RAW` by the version nibble ([#1594](https://github.com/JarryShaw/PyPCAPKit/pull/1594)). @@ -202,7 +202,7 @@ Import paths, before to after: Layer-limited extraction: `layer='link'` no longer sets the termination flag on OSPF and `layer='application'` now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every `layer=` value, including `'internet'` (`IPv4:OSPFIGP` before and after, as IPv4 ends the chain itself). - renames with no compatibility alias left behind: PCAP-NG `Option` subclasses spell the namespace class keyword `ns=` instead of `namespace=` ([#439](https://github.com/JarryShaw/PyPCAPKit/issues/439)). - `tests/protocols/transport/test_tcp_udp_unit.py` now reaches the MP_JOIN dispatchers through `TCP()` itself, instead of assigning a Python `set` to `_flags` on a bare `TCP.__new__(TCP)`. A `set` answers the same membership tests, so every branch ran while the attribute had neither the type production assigns nor the ordering that governs when it exists -- how [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587) stayed invisible. Reverting [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)'s hoist, or restoring the `cast('Enum_Flags', 0)` seed, now fails the file. Test-only ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- building a protocol through its constructor with a keyword that names nothing now raises `UnsupportedCall` instead of discarding it. **This is a behaviour change to a public API**: every `make` in the tree ends its signature with `**kwargs` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602), [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) and [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556). The check sits in `ProtocolBase.__init__` rather than `make`, because `__post_init__` passes one `**kwargs` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by `read` legitimately travels through `make` (`HIP.read` declares `extension`). The accepted set is the union of every keyword-taking parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__` across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new `__keywords__` opts a class out: a set, for a keyword read out of `**kwargs` by name (`ESP.read` with `packet`); and `None`, for a dispatcher whose real signature belongs to a class chosen at call time (only `HTTP.make`). The message names the near neighbour (`seq` reports *did you mean 'seq_no'?*). `from_data` warns `UnknownFieldWarning` rather than raising for a key its own `_make_data` returned, which the caller cannot fix; a keyword passed to `from_data` itself raises, as for a direct construction ([#1147](https://github.com/JarryShaw/PyPCAPKit/pull/1147)). The leftover copies of [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)'s misspelled TCP base, in `examples/generators/dispatch.py` and `tests/protocols/transport/`, are fixed. Some `_make_data` keys that `make` does not declare are reported, not fixed: `Frame._make_data` returns `ts_src` where `make` declares `ts_sec`, `L2TPv2._make_data` returns `prio` where it declares `priority`, and `Header._make_data` returns a `magic_number` that `Header.make` does not take. A *direct* `SomeProtocol.make(...)` call is not checked and still discards silently ([#617](https://github.com/JarryShaw/PyPCAPKit/issues/617)). +- building a protocol through its constructor with a keyword that names nothing now raises `UnsupportedCall` instead of discarding it. **This is a behaviour change to a public API**: every `make` in the tree ends its signature with `**kwargs` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602), [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) and [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556). The check sits in `ProtocolBase.__init__()` rather than `make`, because `__post_init__` passes one `**kwargs` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by `read` legitimately travels through `make` (`HIP.read()` declares `extension`). The accepted set is the union of every keyword-taking parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__` across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new `__keywords__` opts a class out: a set, for a keyword read out of `**kwargs` by name (`ESP.read()` with `packet`); and `None`, for a dispatcher whose real signature belongs to a class chosen at call time (only `HTTP.make`). The message names the near neighbour (`seq` reports *did you mean 'seq_no'?*). `from_data` warns `UnknownFieldWarning` rather than raising for a key its own `_make_data` returned, which the caller cannot fix; a keyword passed to `from_data` itself raises, as for a direct construction ([#1147](https://github.com/JarryShaw/PyPCAPKit/pull/1147)). The leftover copies of [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)'s misspelled TCP base, in `examples/generators/dispatch.py` and `tests/protocols/transport/`, are fixed. Some `_make_data` keys that `make` does not declare are reported, not fixed: `Frame._make_data()` returns `ts_src` where `make` declares `ts_sec`, `L2TPv2._make_data()` returns `prio` where it declares `priority`, and `Header._make_data()` returns a `magic_number` that `Header.make()` does not take. A *direct* `SomeProtocol.make(...)` call is not checked and still discards silently ([#617](https://github.com/JarryShaw/PyPCAPKit/issues/617)). - **a breaking change to** `Application`: it no longer forbids payload dispatch outright. It encodes "no further protocol layer above this one", which undissected trailer bytes are not, so `_decode_next_layer` and `_import_next_layer` accept the `-1` sentinel -- the rest of the packet, resolving to `Raw`, or to `NoPayload` when nothing remains -- and `__post_init__` keeps a payload that `read` set instead of overwriting it with `NoPayload`. A real protocol number (any `proto` other than `-1`) is still refused with `UnsupportedCall`. A subclass that calls either method with `-1` changes from raising to succeeding; no subclass in the package does ([#719](https://github.com/JarryShaw/PyPCAPKit/issues/719)). - the `Protocol` and `ProtocolMeta` docstrings. `Protocol` claimed to be the base of every protocol family, though nothing in the package subclasses it, and `ProtocolMeta` claimed to generate attributes its empty body does not. `Protocol` is now documented as the base for third-party protocols, and `ProtocolBase` as the one the families derive from ([#1101](https://github.com/JarryShaw/PyPCAPKit/issues/1101)). - **a breaking change to** `OSPF_CrytographicAuthentication`: the misspelling is corrected to `OSPF_CryptographicAuthentication`, and `CrytographicAuthentication` in `pcapkit.protocols.schema.application.ospf` and `pcapkit.protocols.data.application.ospf` to `CryptographicAuthentication`, with no alias left for the old spelling. Both OSPF names also move to the application group of `schema.__all__` and `data.__all__`, after [#719](https://github.com/JarryShaw/PyPCAPKit/issues/719)'s move ([#1090](https://github.com/JarryShaw/PyPCAPKit/issues/1090)). @@ -218,69 +218,69 @@ Layer-limited extraction: `layer='link'` no longer sets the termination flag on - next-layer, option, chunk, block and parameter dispatch all read `defaultdict` registries, so a lookup miss inserted the key into class-level state shared by every later instance, after which a legitimate `register_*` call warned that the code was already registered. Every read now goes through a lookup that does not grow the table, and `IPv4.__option__` and `HIP.__parameter__` became inspectable class attributes rather than names assembled at call time ([#426](https://github.com/JarryShaw/PyPCAPKit/pull/426), [#428](https://github.com/JarryShaw/PyPCAPKit/pull/428), [#429](https://github.com/JarryShaw/PyPCAPKit/issues/429), [#434](https://github.com/JarryShaw/PyPCAPKit/pull/434)). What breaks: a tuple-registered handler pair written to the documented `OptionParser`/`OptionConstructor` signature now works where it could never be called, and a pair written with an explicit leading `self` -- the only shape that used to work -- now does not. - the identical defect one layer up, in the schema layer's `EnumSchema.registry`: `Option.registry[code]` for an unregistered `code` inserted the default schema under that code, so a single lookup made an unassigned TCP option number such as `156` read back as registered for the rest of the process. `EnumSchema.__enum__` is now built (or, when a subclass seeds it manually, as `PCAPNG.Option`'s namespaced mapping and `TCP.MPTCP`'s plain one do) as a retention-safe mapping that still returns the registered default on a miss but stops recording it; `.registry` returns the same object as before, so nothing holding a reference is affected ([#555](https://github.com/JarryShaw/PyPCAPKit/issues/555)). - on Python 3.10 and older, no `Schema` subclass got its own `_abc_impl`: all fell through to `collections.abc.Mapping`'s, so a single `isinstance` or `issubclass` answer poisoned every later question about that class for the rest of the process. A terminating PCAP-NG `EndRecord` tested `True` as an `IPv4Record` ([#439](https://github.com/JarryShaw/PyPCAPKit/issues/439)). -- construction, broken in several places at once: the generated typed `__init__` was never installed, so `__post_init__` did not run and a schema built from a subset of its fields could not be packed -- `UDP(srcport=53, dstport=5353)` now packs ([#430](https://github.com/JarryShaw/PyPCAPKit/pull/430)); IPv6 and Mobility Header option padding was wrong, leaving construction wholly broken ([#398](https://github.com/JarryShaw/PyPCAPKit/pull/398)); `HTTP.make` called the versioned `make` unbound, so every real call raised `TypeError` ([#452](https://github.com/JarryShaw/PyPCAPKit/issues/452), [#462](https://github.com/JarryShaw/PyPCAPKit/pull/462)); and `IPv4._make_data` returned the fragment offset in octets where the wire wants 8-octet units, and read `data.options` on a packet that has none ([#494](https://github.com/JarryShaw/PyPCAPKit/issues/494), [#499](https://github.com/JarryShaw/PyPCAPKit/pull/499)). +- construction, broken in several places at once: the generated typed `__init__` was never installed, so `__post_init__` did not run and a schema built from a subset of its fields could not be packed -- `UDP(srcport=53, dstport=5353)` now packs ([#430](https://github.com/JarryShaw/PyPCAPKit/pull/430)); IPv6 and Mobility Header option padding was wrong, leaving construction wholly broken ([#398](https://github.com/JarryShaw/PyPCAPKit/pull/398)); `HTTP.make` called the versioned `make` unbound, so every real call raised `TypeError` ([#452](https://github.com/JarryShaw/PyPCAPKit/issues/452), [#462](https://github.com/JarryShaw/PyPCAPKit/pull/462)); and `IPv4._make_data()` returned the fragment offset in octets where the wire wants 8-octet units, and read `data.options` on a packet that has none ([#494](https://github.com/JarryShaw/PyPCAPKit/issues/494), [#499](https://github.com/JarryShaw/PyPCAPKit/pull/499)). - a truncated or under-declared area no longer parses "successfully", and no longer wedges the process. The option and list loops could spin forever with no exception on a truncated area, reachable from untrusted input through HOPOPT, IPv6-Opts, MH, HIP and SCTP; each iteration must now advance the stream by at least one octet, and the error names the option, the offset and the octets remaining ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431), [#432](https://github.com/JarryShaw/PyPCAPKit/pull/432)). Separately, wire-derived lengths in `ipv6_opts`, CALIPSO, MPL, REG_INFO and HIP list callbacks underflowed below zero, which `ListField`'s own `while length > 0` turned into a silent empty list; they are floored and raise instead ([#449](https://github.com/JarryShaw/PyPCAPKit/pull/449), [#456](https://github.com/JarryShaw/PyPCAPKit/pull/456), [#460](https://github.com/JarryShaw/PyPCAPKit/pull/460), [#463](https://github.com/JarryShaw/PyPCAPKit/issues/463)). - field widths and units, each measured against the specification: HIP's `TRANSPORT_FORMAT_LIST`, `NAT_TRAVERSAL_MODE` and `ESP_TRANSFORM` list entries are two octets, not one [[RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401), [RFC 5770](https://datatracker.ietf.org/doc/html/rfc5770), [RFC 7402](https://datatracker.ietf.org/doc/html/rfc7402)] ([#463](https://github.com/JarryShaw/PyPCAPKit/issues/463), [#472](https://github.com/JarryShaw/PyPCAPKit/issues/472)); the MN-ID option sizes from its subtype, not from `identifier`'s Python type ([#448](https://github.com/JarryShaw/PyPCAPKit/issues/448), [#464](https://github.com/JarryShaw/PyPCAPKit/pull/464), [#467](https://github.com/JarryShaw/PyPCAPKit/issues/467)); IPv6-Route's `Hdr Ext Len` is computed in 8-octet units on both sides [[RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200)] ([#487](https://github.com/JarryShaw/PyPCAPKit/issues/487), [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489)); and the Fast Binding Update and Acknowledgment Lifetimes are plain seconds [[RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568)], not [RFC 6275](https://datatracker.ietf.org/doc/html/rfc6275)'s four-second units ([#502](https://github.com/JarryShaw/PyPCAPKit/pull/502)). -- the remaining call sites where a `_make_*` method converts an address itself, ahead of the schema, so the same `bool`-as-`int` mistake reached them, unguarded by the schema-layer fix above: `ARP._make_proto_resolve`, `IPv6_Route._make_data_type_rpl` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the `dst` parameter of `IPv6_Route.make`, and the latent `OSPF._make_id_numbers`. Each now goes through `parse_ip_address`, the helper [#539](https://github.com/JarryShaw/PyPCAPKit/pull/539) and [#552](https://github.com/JarryShaw/PyPCAPKit/issues/552) route their own sites through, completing the programme [#481](https://github.com/JarryShaw/PyPCAPKit/pull/481) and [#508](https://github.com/JarryShaw/PyPCAPKit/issues/508) started. Pinning `version=6` on the `IPv6_Route` sites is a further, smaller behaviour change: both converted a plain integer through the family-inferring `ipaddress.ip_address`, so `ip=[258]` packed a 4-octet IPv4 address (`0.0.1.2`) inside an IPv6-only header; it now packs the 16-octet `::102` ([#540](https://github.com/JarryShaw/PyPCAPKit/issues/540)). +- the remaining call sites where a `_make_*` method converts an address itself, ahead of the schema, so the same `bool`-as-`int` mistake reached them, unguarded by the schema-layer fix above: `ARP._make_proto_resolve`, `IPv6_Route._make_data_type_rpl()` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the `dst` parameter of `IPv6_Route.make()`, and the latent `OSPF._make_id_numbers`. Each now goes through `parse_ip_address`, the helper [#539](https://github.com/JarryShaw/PyPCAPKit/pull/539) and [#552](https://github.com/JarryShaw/PyPCAPKit/issues/552) route their own sites through, completing the programme [#481](https://github.com/JarryShaw/PyPCAPKit/pull/481) and [#508](https://github.com/JarryShaw/PyPCAPKit/issues/508) started. Pinning `version=6` on the `IPv6_Route` sites is a further, smaller behaviour change: both converted a plain integer through the family-inferring `ipaddress.ip_address`, so `ip=[258]` packed a 4-octet IPv4 address (`0.0.1.2`) inside an IPv6-only header; it now packs the 16-octet `::102` ([#540](https://github.com/JarryShaw/PyPCAPKit/issues/540)). - PCAP and PCAP-NG output and parsing: `bytes(frame)` returned the *next* frame's octets, `files=True` wrote names like `Frame 1..json`, and a PCAP-NG `timestamp_epoch` was shifted by the reading host's timezone ([#403](https://github.com/JarryShaw/PyPCAPKit/pull/403)); further parser defects ([#341](https://github.com/JarryShaw/PyPCAPKit/issues/341)--[#347](https://github.com/JarryShaw/PyPCAPKit/issues/347), [#371](https://github.com/JarryShaw/PyPCAPKit/pull/371)) and, on the write path, more block-parsing ones ([#388](https://github.com/JarryShaw/PyPCAPKit/pull/388)); `BitField` packed every named bit as set ([#359](https://github.com/JarryShaw/PyPCAPKit/issues/359), [#374](https://github.com/JarryShaw/PyPCAPKit/pull/374)); the extension-header walk failed to advance past the last IPv6 extension header ([#348](https://github.com/JarryShaw/PyPCAPKit/issues/348), [#373](https://github.com/JarryShaw/PyPCAPKit/pull/373)); and IPv6 fragment offsets went unscaled, with reassembly keyed on the flow label -- optional, and routinely zero, so distinct datagrams collapsed together -- rather than on the fragment identification ([#389](https://github.com/JarryShaw/PyPCAPKit/pull/389)). -- `TCP._make_mptcp_addaddr` could not build an `ADD_ADDR` option end to end: its `kind=`/`length=` arguments were rejected with `UnknownFieldWarning` and dropped, and `.pack()` then raised `KeyError: 'length'` from `port`'s own condition, `pkt['length'] in (10, 22)`. The cause was one layer up: `MPTCP`, the base of every Multipath TCP subtype schema, declared `kind` and `length` only under `typing.TYPE_CHECKING` rather than as real fields, unlike `Option`. That dropped `kind=`/`length=` for every `_make_mptcp_*` constructor, so `MPTCP` now declares both for real ([#541](https://github.com/JarryShaw/PyPCAPKit/issues/541)). The missing fields broke parsing too: a subtype schema's leading field read the `kind` octet itself, an off-by-two in field alignment rather than a wire-format change (a correct sender's octets were always right). Spec-correct `ADD_ADDR` and `MP_PRIO` options failed with `FieldError: TCP: [OptNo 30] 3 invalid IP version` and `KeyError: 'length'`; both parse now. -- dropped-keyword/wrong-cast defects flagged in review during this release and filed only as [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556): HIP's `_make_param_encrypted` passed `cipher=` to a schema with no such field, so an AES-cipher `ENCRYPTED` parameter built through `make` packed without its IV; and IPv6-Route's `RPL.post_process`, which runs on every `Schema.pack`, assumed `self.addresses` was still the concatenated `bytes` a parse leaves, and raised slicing the `list[bytes]` a `make`-built multi-address header holds ([#556](https://github.com/JarryShaw/PyPCAPKit/issues/556)). -- further defects [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) exposed by letting construction reach code that had never run. `MPTCP.subtype` was still `typing.TYPE_CHECKING`-only, so `TCP(options=[(Enum_Option.Multipath_TCP, ...)])` raised `AttributeError: ... has no attribute 'subtype'` for every subtype but `MP_JOIN`: the convenience constructor reads the schema straight back through `_read_mptcp_*` with no byte round trip, so `_MPTCP.post_process`, the only code that set `subtype`, never ran. Fixed on the construction path (`TCP._make_mode_mp`) rather than by making `subtype` a real field too, the way `kind`/`length` became in [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541): `subtype` is already packed as 4 bits of each subtype's own `test` bitfield, and a second field for the same bits would double-encode them or need a "derive, don't pack" field kind this library lacks ([#566](https://github.com/JarryShaw/PyPCAPKit/issues/566)). Separately, `_make_mptcp_capable` wrote `length=20 if rkey is None else 32` where [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 gives 12 and 20, and `MPTCPCapable.rkey`'s condition (`pkt['length'] != 32`) dropped the receiver's key for exactly the length the maker used for "key present": a key-absent MP_CAPABLE could not be built, and a key-present one lost its key on the wire. Both, and the guard in `_read_mptcp_capable`, now agree on 12/20. **This changes MP_CAPABLE's packed output**: a 20-octet key-present option built or parsed under the old code becomes 12 octets with no key, or 20 octets with the key present, depending on which the caller meant ([#567](https://github.com/JarryShaw/PyPCAPKit/issues/567)). +- `TCP._make_mptcp_addaddr()` could not build an `ADD_ADDR` option end to end: its `kind=`/`length=` arguments were rejected with `UnknownFieldWarning` and dropped, and `.pack()` then raised `KeyError: 'length'` from `port`'s own condition, `pkt['length'] in (10, 22)`. The cause was one layer up: `MPTCP`, the base of every Multipath TCP subtype schema, declared `kind` and `length` only under `typing.TYPE_CHECKING` rather than as real fields, unlike `Option`. That dropped `kind=`/`length=` for every `_make_mptcp_*` constructor, so `MPTCP` now declares both for real ([#541](https://github.com/JarryShaw/PyPCAPKit/issues/541)). The missing fields broke parsing too: a subtype schema's leading field read the `kind` octet itself, an off-by-two in field alignment rather than a wire-format change (a correct sender's octets were always right). Spec-correct `ADD_ADDR` and `MP_PRIO` options failed with `FieldError: TCP: [OptNo 30] 3 invalid IP version` and `KeyError: 'length'`; both parse now. +- dropped-keyword/wrong-cast defects flagged in review during this release and filed only as [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556): HIP's `_make_param_encrypted` passed `cipher=` to a schema with no such field, so an AES-cipher `ENCRYPTED` parameter built through `make` packed without its IV; and IPv6-Route's `RPL.post_process()`, which runs on every `Schema.pack()`, assumed `self.addresses` was still the concatenated `bytes` a parse leaves, and raised slicing the `list[bytes]` a `make`-built multi-address header holds ([#556](https://github.com/JarryShaw/PyPCAPKit/issues/556)). +- further defects [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) exposed by letting construction reach code that had never run. `MPTCP.subtype` was still `typing.TYPE_CHECKING`-only, so `TCP(options=[(Enum_Option.Multipath_TCP, ...)])` raised `AttributeError: ... has no attribute 'subtype'` for every subtype but `MP_JOIN`: the convenience constructor reads the schema straight back through `_read_mptcp_*` with no byte round trip, so `_MPTCP.post_process()`, the only code that set `subtype`, never ran. Fixed on the construction path (`TCP._make_mode_mp()`) rather than by making `subtype` a real field too, the way `kind`/`length` became in [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541): `subtype` is already packed as 4 bits of each subtype's own `test` bitfield, and a second field for the same bits would double-encode them or need a "derive, don't pack" field kind this library lacks ([#566](https://github.com/JarryShaw/PyPCAPKit/issues/566)). Separately, `_make_mptcp_capable` wrote `length=20 if rkey is None else 32` where [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 gives 12 and 20, and `MPTCPCapable.rkey`'s condition (`pkt['length'] != 32`) dropped the receiver's key for exactly the length the maker used for "key present": a key-absent MP_CAPABLE could not be built, and a key-present one lost its key on the wire. Both, and the guard in `_read_mptcp_capable`, now agree on 12/20. **This changes MP_CAPABLE's packed output**: a 20-octet key-present option built or parsed under the old code becomes 12 octets with no key, or 20 octets with the key present, depending on which the caller meant ([#567](https://github.com/JarryShaw/PyPCAPKit/issues/567)). - TCP's MP_CAPABLE now parses and builds all five lengths of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where `_read_mptcp_capable` raised `ProtocolError: TCP: [OptNo 30] invalid format` for anything but 12 and 20. In the schema `skey` is now present only for `length > 4` and `rkey` for `length > 12`, and the fields `dl_len` (16 bits, `length > 20`) and `checksum` (2 octets, `length > 22`) are new. `_make_mptcp_capable` takes `dl_len` and `checksum` too, where it used to drop them silently through `**kwargs`, and picks the length from how many of `skey`, `rkey`, `dl_len` and `checksum` are not `None`. A field given without the one before it raises `ProtocolError`. The data model `MPTCPCapable` gains `dl_len` and `checksum`, and `skey` becomes `Optional[int]`: code that constructs it directly must now pass both new fields, and `to_dict()` carries the new keys ([#1042](https://github.com/JarryShaw/PyPCAPKit/issues/1042)). -- IPv6-Route's RPL routing data was five octets wide at the front where [RFC 6554](https://datatracker.ietf.org/doc/html/rfc6554) section 3 gives four, with further defects behind it. `CmprI`, `CmprE` and `Pad` are each 4 bits sharing a 32-bit word with a 20-bit `Reserved`, but the schema declared `cmpr_i` and `cmpr_e` as whole octets, so a constructed two-address header packed to 41 octets while its `Hdr Ext Len` of 5 declared 48. Correcting the width is a **wire-format change** and reshapes the schema: `RPL(cmpr_i=..., cmpr_e=...)` is now `RPL(cmpr={'cmpr_i': ..., 'cmpr_e': ...})`, beside the existing `pad={'pad_len': ...}`. Behind it, the reader's `header.length % 16` guard read `Hdr Ext Len` as an octet count and assumed 16-octet addresses, which an SRH carries only when `CmprI` and `CmprE` are both 0 -- the unit confusion [#487](https://github.com/JarryShaw/PyPCAPKit/issues/487) fixed for Source Route and Type 2, left by [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489) for want of a working RPL round trip to validate against. It is replaced by section 4.2's address-count arithmetic, `n = (((Hdr Ext Len * 8) - Pad - (16 - CmprE)) / (16 - CmprI)) + 1`, which the reader now requires to be non-negative and whole. `RPL.post_process` subtracted `pad_len` a second time from a buffer whose length callback had already taken it off, losing one address per `16 - CmprI` octets of padding, and set `ip` only when it had parsed octets, so `IPv6_Route(type=..., data={'ip': [...]})` raised a bare `AttributeError: 'RPL' object has no attribute 'ip'`. And `_make_data_type_rpl` computed `Pad` as `8 - length % 8` without the outer `% 8`, so an aligned address vector got a full 8 octets of padding where section 3 requires `Pad` to be 0 when `CmprI` and `CmprE` are both 0. The narrower `cmpr_i` also removes a latent divide-by-zero (a whole octet could hold 16, making `16 - CmprI` zero). `ipv6-route-type/RPL_Source_Route_Header` round-trips and its `EXPECTED_FAILURES` entry is deleted; none of this has been checked against a real RPL capture ([#564](https://github.com/JarryShaw/PyPCAPKit/issues/564)). +- IPv6-Route's RPL routing data was five octets wide at the front where [RFC 6554](https://datatracker.ietf.org/doc/html/rfc6554) section 3 gives four, with further defects behind it. `CmprI`, `CmprE` and `Pad` are each 4 bits sharing a 32-bit word with a 20-bit `Reserved`, but the schema declared `cmpr_i` and `cmpr_e` as whole octets, so a constructed two-address header packed to 41 octets while its `Hdr Ext Len` of 5 declared 48. Correcting the width is a **wire-format change** and reshapes the schema: `RPL(cmpr_i=..., cmpr_e=...)` is now `RPL(cmpr={'cmpr_i': ..., 'cmpr_e': ...})`, beside the existing `pad={'pad_len': ...}`. Behind it, the reader's `header.length % 16` guard read `Hdr Ext Len` as an octet count and assumed 16-octet addresses, which an SRH carries only when `CmprI` and `CmprE` are both 0 -- the unit confusion [#487](https://github.com/JarryShaw/PyPCAPKit/issues/487) fixed for Source Route and Type 2, left by [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489) for want of a working RPL round trip to validate against. It is replaced by section 4.2's address-count arithmetic, `n = (((Hdr Ext Len * 8) - Pad - (16 - CmprE)) / (16 - CmprI)) + 1`, which the reader now requires to be non-negative and whole. `RPL.post_process()` subtracted `pad_len` a second time from a buffer whose length callback had already taken it off, losing one address per `16 - CmprI` octets of padding, and set `ip` only when it had parsed octets, so `IPv6_Route(type=..., data={'ip': [...]})` raised a bare `AttributeError: 'RPL' object has no attribute 'ip'`. And `_make_data_type_rpl` computed `Pad` as `8 - length % 8` without the outer `% 8`, so an aligned address vector got a full 8 octets of padding where section 3 requires `Pad` to be 0 when `CmprI` and `CmprE` are both 0. The narrower `cmpr_i` also removes a latent divide-by-zero (a whole octet could hold 16, making `16 - CmprI` zero). `ipv6-route-type/RPL_Source_Route_Header` round-trips and its `EXPECTED_FAILURES` entry is deleted; none of this has been checked against a real RPL capture ([#564](https://github.com/JarryShaw/PyPCAPKit/issues/564)). - `L2TPv2` parsed any version nibble, so an L2TPv3 datagram was reported as v2 with a tunnel and session ID read out of v3's Control Connection ID. [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) §3.1 fixes `Ver` at 2 and reserves 1 for L2F, and `L2TPv2.version` documented that any other value "is a different protocol reached through a different class", but nothing enforced it, so the hard-coded `Literal[2]` property and `info.version` answered 2 and 3 on the same octets. `read` now raises `ProtocolError`, degrading the payload to `Raw` through the existing `beholder` path with the reason recorded. **This affects real captures**: [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) §4.1.2 puts L2TPv3 on port 1701 too, the port `UDP.__proto__` binds, so `Ethernet:IPv4:UDP:L2TPv2:Raw` with invented field values becomes `Ethernet:IPv4:UDP:Raw` with the octets preserved. This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which reported `TransType.L2TP` (115) as registered nowhere and proposed binding `L2TPv2` there. That binding is wrong rather than awkward: [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) §4.1.1 gives 115 to *L2TPv3 over IP*, whose session header carries **no version nibble at all**, so a v2 parser cannot detect that the datagram is not its own. 115 is a missing *class*, not a missing registration, and stays unbound until an `L2TPv3` class exists; no dissector was invented to fill it. The reasoning is recorded in `pcapkit.protocols.link.l2tp`, and `register_protocol_code`'s worked example, which named `L2TPv2` at 115, names `L2TPv3` instead ([#548](https://github.com/JarryShaw/PyPCAPKit/issues/548)). -- building any `MP_JOIN` option raised `AttributeError: 'TCP' object has no attribute '_flags'`. `TCP.make` constructed the options *before* assigning the `self._flags` that the option makers read, and `_make_mptcp_join` branches on it to choose among the three layouts of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2 (figure 5, SYN, 12 octets; figure 6, SYN/ACK, 16; figure 7, ACK, 24). The parse path was never affected, since `read` assigns the flags first. Fixed by hoisting the flag resolution above the `_make_tcp_options` call, leaving only the data-offset computation, which needs the options' total length, after it. **Not** fixed by a zero default for `_flags`, which would have been worse: `Protocol.pack` is public and calls `make`, so an instance that had parsed a segment had the attribute set and silently built the option for the segment it had *read*. A parsed MP_JOIN-SYN instance asked to pack an MP_JOIN-ACK segment emitted an ACK header carrying figure 5's 12-octet SYN option, with the caller's 20-octet HMAC replaced by an all-zero token and nonce, and nothing raised. Hoisting also made a branch reachable for the first time, an MP_JOIN on a segment with neither SYN nor ACK: the accumulator was seeded with `cast('Enum_Flags', 0)`, and `typing.cast` being a runtime no-op, `self._flags` stayed a plain `int` and the first membership test raised `TypeError` rather than the documented `ProtocolError`. It is now seeded with `Enum_Flags(0)`, a real flagless member that equals `0` and ORs identically. The read path's seed is unchanged: a flagless MP_JOIN is rejected by `mptcp_data_selector` before `_read_mptcp_join` runs ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). +- building any `MP_JOIN` option raised `AttributeError: 'TCP' object has no attribute '_flags'`. `TCP.make()` constructed the options *before* assigning the `self._flags` that the option makers read, and `_make_mptcp_join` branches on it to choose among the three layouts of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2 (figure 5, SYN, 12 octets; figure 6, SYN/ACK, 16; figure 7, ACK, 24). The parse path was never affected, since `read` assigns the flags first. Fixed by hoisting the flag resolution above the `_make_tcp_options` call, leaving only the data-offset computation, which needs the options' total length, after it. **Not** fixed by a zero default for `_flags`, which would have been worse: `Protocol.pack()` is public and calls `make`, so an instance that had parsed a segment had the attribute set and silently built the option for the segment it had *read*. A parsed MP_JOIN-SYN instance asked to pack an MP_JOIN-ACK segment emitted an ACK header carrying figure 5's 12-octet SYN option, with the caller's 20-octet HMAC replaced by an all-zero token and nonce, and nothing raised. Hoisting also made a branch reachable for the first time, an MP_JOIN on a segment with neither SYN nor ACK: the accumulator was seeded with `cast('Enum_Flags', 0)`, and `typing.cast` being a runtime no-op, `self._flags` stayed a plain `int` and the first membership test raised `TypeError` rather than the documented `ProtocolError`. It is now seeded with `Enum_Flags(0)`, a real flagless member that equals `0` and ORs identically. The read path's seed is unchanged: a flagless MP_JOIN is rejected by `mptcp_data_selector` before `_read_mptcp_join` runs ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). - the ILNP Nonce option builders in `HOPOPT` and `IPv6_Opts` sized the option with `math.ceil(nonce.bit_length() // 8)`, floor division dressed as a ceiling (`//` floors and `math.ceil` of an `int` is a no-op). The nonce is packed by a `NumberField` whose width *is* the declared `len`, so an under-declared length silently truncated the nonce on the wire. Every nonce whose bit length was not a multiple of eight was affected, and **small values were the worst case rather than boundary values**: any nonce below 256 was declared as *zero* octets and dropped (`nonce=9` packed to `b'\x8b\x00'` and parsed back as `0`). Fixed to `max(1, math.ceil(nonce.bit_length() / 8))`, the form used at the other sizing sites in `hip.py` and `mh.py`. The one-octet floor is the `mh.py` convention and is load-bearing because `nonce` defaults to `0`, whose bit length is `0`: without it the default builds an option with no Nonce Value field at all, collapsing "the nonce is 0" into "there is no nonce" when [RFC 6744](https://datatracker.ietf.org/doc/html/rfc6744) gives the option that field. The read path takes its width from the `len` octet on the wire and was never affected ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). -- `httpv1`'s `_RE_METHOD` was unanchored and `re.match` anchors only at the start, so it prefix-matched, and the request-line reader passed the whole `para1` to `Method.get` rather than the captured `method` group. So `b'Get'` matched on the single character `G`, satisfied the guard that decides a start-line is a request, and handed the mixed-case token to a lookup that raised on it. Fixing either half alone still gives a wrong answer (normalising the lookup would parse `b'Get'` as `GET` off a one-character match; passing the group would parse it as a method named `G`). The pattern is anchored at both ends and the captured group is looked up, so a non-method token is a malformed request line rather than a mis-parsed one. Method tokens are case-sensitive per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so no `re.I` was added: `GET` parses, `Get` and `get` are rejected ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). +- `httpv1`'s `_RE_METHOD` was unanchored and `re.match` anchors only at the start, so it prefix-matched, and the request-line reader passed the whole `para1` to `Method.get()` rather than the captured `method` group. So `b'Get'` matched on the single character `G`, satisfied the guard that decides a start-line is a request, and handed the mixed-case token to a lookup that raised on it. Fixing either half alone still gives a wrong answer (normalising the lookup would parse `b'Get'` as `GET` off a one-character match; passing the group would parse it as a method named `G`). The pattern is anchored at both ends and the captured group is looked up, so a non-method token is a malformed request line rather than a mis-parsed one. Method tokens are case-sensitive per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so no `re.I` was added: `GET` parses, `Get` and `get` are rejected ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). - `_RE_STATUS` in the same reader had the same unanchored prefix defect, found by auditing `_RE_METHOD`'s siblings, and escaped as the wrong exception type. The pattern is only a guard (the value comes from `int(para2)` on the raw token), so a prefix match let a malformed status past the guard and out of `int()` uncaught, where `_read_http_header` documents `ProtocolError`: `200x` raised `ValueError: invalid literal for int()` and `2000` raised `ValueError: 2000 is not a valid StatusCode`; both are now `ProtocolError`. [RFC 9112 Section 4](https://datatracker.ietf.org/doc/html/rfc9112#section-4) gives `status-code = 3DIGIT`, exactly three, so the anchor is what the grammar said (the production lives in HTTP/1.1 because `status-code` is part of its `status-line`; [RFC 9110 Section 15](https://datatracker.ietf.org/doc/html/rfc9110#section-15) covers code semantics and the registry). `_RE_VERSION` was audited and is safe, since every call site reads the captured group ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). - as a consequence of the above, the unhandled `MemoryError` in `pcapkit/protocols/protocol.py`'s `self._file.read()` call on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). +- reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack()` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack()` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). - documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, whereas a `SwitchField` always resolves to a concrete field ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). - the assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, which [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) turned red on `main`. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - the HIP `SOLUTION` builder sized the parameter with `4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)`, an invalid shorthand for two fields. [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5) gives `Length` as `4 + RHASH_len / 4` over a `Random #I` and a `Puzzle solution #J` of `RHASH_len / 8` octets **each**, and `/ 4` equals twice `/ 8` only because `RHASH_len` is a whole number of octets. For an arbitrary `bit_length()` the builder emitted an *odd* contents width, which its own reader refused: `_read_param_solution`'s `(schema.len - 4) % 2` guard exists because `SolutionParameter` splits that width into two equal halves. `random=0x1` with `solution=0xfff` declared `len=7` and raised `ProtocolError: HIPv2: [ParamNo 321] invalid format` on input the library had produced. The undersized length was worse: both fields take their width from that `len`, so `solution=0xfff` was packed into one octet and came back as `0xff`, silently. Fixed to `4 + 2 * math.ceil(max(...) / 8)`, the form the sibling `PUZZLE` builder uses, which is even by construction. Loosening the reader was rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). -- `TCP.read` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. A dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). -- **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading** `frame.len` **or** `frame.cap_len` **from a** `.pcap` **gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). +- `TCP.read()` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. A dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). +- **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading** `frame.len` **or** `frame.cap_len` **from a** `.pcap` **gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read()` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). - the HIP `PUZZLE` and `SOLUTION` builders derived wire-format values from the payload value instead of the data model, and each was wrong. **This changes the** `PuzzleParameter` **and** `SolutionParameter` **data models and the octets both parameters emit.** The width of `Random #I` and `Puzzle solution #J` came from `int.bit_length()` alone, so every leading zero octet was dropped on re-serialisation: a `SOLUTION` read with `Length = 20` rebuilt as `Length = 6`, a `PUZZLE` with `Length = 12` as `Length = 5`, silently. That width is `RHASH_len / 8` octets [[RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4), [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5)], a property of the Responder's HIT Suite, so both data models now carry it as `rhash_len` and both builders prefer it; a new `rhash_len=` keyword states it for a build from scratch, which under HIPv2 is the only place it can come from, since `RSA,DSA/SHA-256` is the REQUIRED HIT Suite [[RFC 7401 Section 5.2.10](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.10)] and makes the field 32 octets rather than 8. On the mandatory path about one parameter in 256 has a zero top octet and lost it ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653)). `SOLUTION`'s second contents octet is `Reserved`, "zero when sent, ignored when received" [[RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5), and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) identically], not the `Lifetime` that only [RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4) defines, which pcapkit encoded there as `2^(value - 32)` seconds. It wrote `0x20`, `0x21`, `0x25` or `0x2b` into a field the RFC requires to be zero, and could not write that zero: a conformant `SOLUTION` with `Reserved` `0x00` parsed to `timedelta(0)` and could not be re-serialised, escaping a bare `ValueError` from `math.log2(0.0)` that was not a `BaseError`. The field is renamed `reserved`, defaults to zero, and is carried verbatim across a round trip ([#654](https://github.com/JarryShaw/PyPCAPKit/issues/654)). And neither builder read its own `version` keyword, so `version=1` and `version=2` computed identical lengths, where [RFC 5201 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.4) and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) state both fields as literally 8 bytes and `Length` as literally 12 and 20. Under HIPv1 each builder accepted only a `bit_length()` of 57..64 and built, for anything narrower, a parameter this library's reader rejects -- the shape of [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608), in the `PUZZLE` builder that [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629) never opened ([#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). The remaining `math.log2` sites, in `PUZZLE` where a lifetime is real, raise `ProtocolError` (`(BaseError, ValueError)`, so a caller written around the bare exception still catches it). Migration: `SolutionParameter.lifetime` is now `reserved` and an `int` rather than a `timedelta`; `PuzzleParameter` and `SolutionParameter` gained a required `rhash_len`; and `_make_param_solution` no longer takes `lifetime=`. Each was reachable end to end only once [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608) was fixed in [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629), which removed the parity guard that had failed these rebuilds loudly; the round trip then stopped raising and started quietly emitting a different parameter, which is why they were fixed together ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653), [#654](https://github.com/JarryShaw/PyPCAPKit/issues/654), [#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). -- every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **every HIP parameter but** `LOCATOR_SET`, excluded below. **This changes the octets those parameters write, and the** `length` **their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; every padding site computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. Every site but `LOCATOR_SET`'s (the `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and the record lengths in `pcapkit/protocols/internet/hip.py`) now goes through one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other parameters move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). -- HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only `_make_http_*` method that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at each of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since each passes `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668), below) ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). +- every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **every HIP parameter but** `LOCATOR_SET`, excluded below. **This changes the octets those parameters write, and the** `length` **their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; every padding site computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. Every site but `LOCATOR_SET`'s (the `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and the record lengths in `pcapkit/protocols/internet/hip.py`) now goes through one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make()`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other parameters move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). +- HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only `_make_http_*` method that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process()` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at each of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since each passes `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668), below) ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). - MPTCP error messages carried a doubled separator, rendering as `TCP: : [OptNo 30] 1: invalid flags combination`. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty field reads as a value that failed to interpolate. The sites were `_read_mptcp_join` and `_make_mptcp_join` for the invalid flag combination, and both guards of `_make_mptcp_dss` for missing required fields; every other message in the file was already spelled correctly, and none is doubled now. Pre-existing since 2023-04-10. The new test asserts each message in full, since the existing `assertIn` on the body passes either side of the change ([#649](https://github.com/JarryShaw/PyPCAPKit/issues/649)). - names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. Each is now in its module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as the sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values, and reports exactly those findings on the unfixed tree. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). -- every PCAP-NG packet block now carries the captured octets it declares. Every Enhanced Packet Block of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those block types, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack` extracted them from the block schema, and `ProtocolBase.__init__` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the packet types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). -- an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. The payload length callbacks for those frames in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. The same lambda also governs **packing**: an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte. The only other conditional of this shape, `TSOption.remainder` in `pcapkit/protocols/schema/internet/ipv4.py`, is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). -- the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register` was a bare `cls.__enum__[code] = schema`, and is the schema half of the public option registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of them takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its registry writes (`R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), among them) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). -- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). Another route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the unwrapped `length - N` spans and the `__option_padding__` fields: `_TextField.__call__` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Bare exceptions in `SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the last registrar with no overwrite guard: `Option.register` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and some short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). -- HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). Another defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690). `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. -- `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at these sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for each of those inputs. Each site now raises `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so any further bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787), [#789](https://github.com/JarryShaw/PyPCAPKit/pull/789)). +- every PCAP-NG packet block now carries the captured octets it declares. Every Enhanced Packet Block of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those block types, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack()` extracted them from the block schema, and `ProtocolBase.__init__()` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the packet types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__()` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). +- an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. The payload length callbacks for those frames in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack()` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. The same lambda also governs **packing**: an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte. The only other conditional of this shape, `TSOption.remainder` in `pcapkit/protocols/schema/internet/ipv4.py`, is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). +- the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register()` was a bare `cls.__enum__[code] = schema`, and is the schema half of the public option registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__()` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__()` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register()` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of them takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its registry writes (`R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), among them) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). +- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read()` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames()` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). Another route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the unwrapped `length - N` spans and the `__option_padding__` fields: `_TextField.__call__()` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Bare exceptions in `SystemdJournalExportBlock.post_process()`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the last registrar with no overwrite guard: `Option.register()` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack()` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and some short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). +- HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack()` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set()` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). Another defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690). `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. +- `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at these sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version()` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for each of those inputs. Each site now raises `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so any further bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787), [#789](https://github.com/JarryShaw/PyPCAPKit/pull/789)). - a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: `b'MESSAGE=hello'` plus three NULs returned `MESSAGE == 'hello\x00\x00\x00'`. The padding-only-line guard from [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) catches padding only on a line of its own, which needs the preceding field to have ended with a real newline; without one `readline()` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 octets, so at most the trailing three octets of a terminator-less line are stripped, with a `SchemaWarning`, and anything past that is left as data. The gate reads the *raw* line, since a real trailing newline, or any other raw last octet including ASCII whitespace, proves the padding is zero ([#794](https://github.com/JarryShaw/PyPCAPKit/issues/794)). `examples/generators/pcapng.py`'s `_journal_entry` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s `not line.strip(b'\x00')` guard and the mechanism ([#791](https://github.com/JarryShaw/PyPCAPKit/issues/791)). -- `httpv2.HTTP.read` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack`/ `FieldBase.length` machinery the schema modules share, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). -- `SystemdJournalExportBlock.post_process` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`; a missing or non-`b'\n'` octet ends the entry with a `SchemaWarning` instead of resynchronising at the wrong offset. `test_journal_fields_following_a_binary_field_are_not_discarded` fails against the unfixed code with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). +- `httpv2.HTTP.read()` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack()`/ `FieldBase.length` machinery the schema modules share, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). +- `SystemdJournalExportBlock.post_process()` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`; a missing or non-`b'\n'` octet ends the entry with a `SchemaWarning` instead of resynchronising at the wrong offset. `test_journal_fields_following_a_binary_field_are_not_discarded` fails against the unfixed code with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). - the same block's entries were split on a bare `b'\n\n'`, which shreds a binary field whose value contains that pair. Walking the entry instead of splitting it introduced a regression on first review: a `malformed` flag broke the *outer* per-block loop on a bad terminator, ending the whole block and dropping every well-formed entry behind it. Fixed by dropping `malformed`, leaving the per-entry `break` as the only effect. Another gap closed in the same change: a trailing separator on the block's last octet was indistinguishable from EOF and swallowed, so a rebuild wrote `length` one octet short ([#723](https://github.com/JarryShaw/PyPCAPKit/issues/723)). -- **a breaking change to** `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` each minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). Each now peeks at what `.get()` would consult and call it only once the value resolves; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and most other namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: some sample captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). +- **a breaking change to** `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` each minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). Each now peeks at what `.get()` would consult and call it only once the value resolves; a genuine miss gets `EnumField._unregistered_member()`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get()` merged `opt` over the requested namespace and most other namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: some sample captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). - `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the frame and the request left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). - `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and the genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: `TCP:80/8080` traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). -- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), above), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation` and `OptionType` convert their unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; those with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a later review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (an independent mint site of its own) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). -- `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. Every such `Raises:` clause already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). +- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), above), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation` and `OptionType` convert their unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; those with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a later review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options()`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (an independent mint site of its own) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). +- `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. Every such `Raises:` clause already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register()` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). - `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Other guards named the wrong option too: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). -- `httpv1.HTTP` refused every HTTP/1 message with no header fields, which [[RFC 9112 Section 2.1](https://datatracker.ietf.org/doc/html/rfc9112#section-2.1)] allows (`*( field-line CRLF )`): `b'GET / HTTP/1.0\r\n\r\n'` raised `ProtocolError: HTTP: invalid format`, because `_read_http_header` treated a header with no CRLF as malformed. So `HTTPv1(method='GET', uri='/', http_version='1.1')` failed to read back the bytes its own `make` built, and `HTTP._guess_version` answered `unknown HTTP version`, since `_test_start_line` mirrored the same check. A CRLF-less header is now a start line with zero fields, in the parser and the predicate alike. That check was also what refused the HTTP/2 connection preface, whose header is exactly `PRI * HTTP/2.0`; it is now refused by name instead, to the same message ([#1041](https://github.com/JarryShaw/PyPCAPKit/issues/1041)). +- `httpv1.HTTP` refused every HTTP/1 message with no header fields, which [[RFC 9112 Section 2.1](https://datatracker.ietf.org/doc/html/rfc9112#section-2.1)] allows (`*( field-line CRLF )`): `b'GET / HTTP/1.0\r\n\r\n'` raised `ProtocolError: HTTP: invalid format`, because `_read_http_header` treated a header with no CRLF as malformed. So `HTTPv1(method='GET', uri='/', http_version='1.1')` failed to read back the bytes its own `make` built, and `HTTP._guess_version()` answered `unknown HTTP version`, since `_test_start_line` mirrored the same check. A CRLF-less header is now a start line with zero fields, in the parser and the predicate alike. That check was also what refused the HTTP/2 connection preface, whose header is exactly `PRI * HTTP/2.0`; it is now refused by name instead, to the same message ([#1041](https://github.com/JarryShaw/PyPCAPKit/issues/1041)). - `httpv1.HTTP.from_data` could not rebuild an HTTP/1 message whose body is empty: `HTTP.from_data(info)` for `b'GET / HTTP/1.1\r\nHost: a\r\n\r\n'` raised `TypeError: can't concat NoneType to bytes`. `read` records an empty body as `None`, `_make_data` passed it through, and `make` concatenated it. `_make_data` now hands `make` `b''` for a `None` body; `info.body` is still `None`, so readers see no change. `httpv2` is unaffected: its empty-payload frames already round-trip ([#1050](https://github.com/JarryShaw/PyPCAPKit/issues/1050)). - `IPv6._read_ip_hextet` sliced the traffic class from `hex[0:2]` and the flow label from `hex[2:]`, both including the version nibble, so `6abcdef0` read as `(6, 0x6a, 0xbcdef0)`; it now reads `(6, 0xab, 0xcdef0)`. The parser reads these fields through the schema, so only a direct call was affected ([#1094](https://github.com/JarryShaw/PyPCAPKit/issues/1094)). -- the Mobility Header Message ID option wrote a zero fraction for every timestamp, and `MesgIDOption.post_process` ignored the fraction on read. `_make_opt_mesg_id` now writes `round(frac * 2**32)`, clamped to 32 bits, and the reader adds `fraction / 2**32` seconds, so `timestamp` keeps sub-second precision ([#1093](https://github.com/JarryShaw/PyPCAPKit/issues/1093)). +- the Mobility Header Message ID option wrote a zero fraction for every timestamp, and `MesgIDOption.post_process()` ignored the fraction on read. `_make_opt_mesg_id` now writes `round(frac * 2**32)`, clamped to 32 bits, and the reader adds `fraction / 2**32` seconds, so `timestamp` keeps sub-second precision ([#1093](https://github.com/JarryShaw/PyPCAPKit/issues/1093)). - HIP parameter defects. `HOST_ID` packed DI-Type and DI-Length as a 4-octet `BitField` where [RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401) section 5.2.9 has one 16-bit word, so every built record carried two surplus octets and read back the wrong algorithm ([#1118](https://github.com/JarryShaw/PyPCAPKit/issues/1118)); with it, `make_samples.py` builds every HIP parameter. `HIP_CIPHER` warned at six Cipher IDs, which section 5.2.8 allows; it now warns only above six and keeps the first six, as a recipient must. The maker is unchanged ([#1092](https://github.com/JarryShaw/PyPCAPKit/issues/1092)). And the HIPv2-only `DH_GROUP_LIST`, `HIP_CIPHER`, `HIT_SUITE_LIST` and `TRANSPORT_FORMAT_LIST` were built and parsed in a HIPv1 packet; at version 1 the reader and the maker now raise `ProtocolError`, as `R1_Counter` and `HIP_TRANSFORM` already do on a version mismatch ([#1133](https://github.com/JarryShaw/PyPCAPKit/issues/1133)). - the PCAP `Header` could not rebuild itself. `_make_magic`'s explicit-endian branch ignored `nanosecond` ([#1096](https://github.com/JarryShaw/PyPCAPKit/issues/1096)), and `_make_data` returned a `magic_number` that `make()` does not take -- one of the keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry above reports -- so `from_data` dropped it with an `UnknownFieldWarning` and rebuilt every header with the host-default `d4c3b2a1` magic. `_make_data` now returns `byteorder` and `nanosecond`, and every byte order and timestamp resolution rebuilds unchanged. `Header._read_protos` also read four octets whatever `size` asked for ([#1125](https://github.com/JarryShaw/PyPCAPKit/issues/1125)). And `_make_magic` honoured `lilendian` and `bigendian` only as a pair, so `Header(bigendian=True)` wrote `d4c3b2a1` on a little-endian host; either flag alone now picks the byte order and overrides `byteorder`, and equal flags still raise `EndianError` ([#1144](https://github.com/JarryShaw/PyPCAPKit/issues/1144)). - TCP's NS flag was written by `make()` but never parsed: it had been commented out of `Flags`, the reader and `_make_data`, with no reason recorded. Per the owner's ruling it is exposed rather than removed, so `info.flags.ns` exists and `from_data` keeps the bit ([#1098](https://github.com/JarryShaw/PyPCAPKit/issues/1098)). -- `from_data` round trips. `Frame._make_data` returned `ts_src` and `L2TPv2._make_data` `prio`, the other keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry reports, so `from_data` discarded both with an `UnknownFieldWarning` and an L2TPv2 priority bit was lost; they are now `ts_sec` and `priority` ([#1097](https://github.com/JarryShaw/PyPCAPKit/issues/1097)). And `ProtocolBase.from_data` took no keywords, so `Frame.from_data` and `PCAPNG.from_data`, which need `num`/`header` and `num`/`sct`/`ctx`, could not be called at all. Extra keywords now reach the construction, overriding `_make_data`'s, under the construction-keyword check, so a typo raises `UnsupportedCall` ([#1126](https://github.com/JarryShaw/PyPCAPKit/issues/1126)). `HTTP.from_data` always raised `invalid HTTP version: 0`, looking up a `version` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's `_make_data` also returned the payload `length`, which `ProtocolBase` takes as the parse length, so even a direct call failed as shorter than the 9-octet frame header; `make` now computes it, and every HTTP frame in the sample captures rebuilds byte for byte ([#1154](https://github.com/JarryShaw/PyPCAPKit/issues/1154)). +- `from_data` round trips. `Frame._make_data()` returned `ts_src` and `L2TPv2._make_data()` `prio`, the other keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry reports, so `from_data` discarded both with an `UnknownFieldWarning` and an L2TPv2 priority bit was lost; they are now `ts_sec` and `priority` ([#1097](https://github.com/JarryShaw/PyPCAPKit/issues/1097)). And `ProtocolBase.from_data()` took no keywords, so `Frame.from_data` and `PCAPNG.from_data`, which need `num`/`header` and `num`/`sct`/`ctx`, could not be called at all. Extra keywords now reach the construction, overriding `_make_data`'s, under the construction-keyword check, so a typo raises `UnsupportedCall` ([#1126](https://github.com/JarryShaw/PyPCAPKit/issues/1126)). `HTTP.from_data` always raised `invalid HTTP version: 0`, looking up a `version` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's `_make_data` also returned the payload `length`, which `ProtocolBase` takes as the parse length, so even a direct call failed as shorter than the 9-octet frame header; `make` now computes it, and every HTTP frame in the sample captures rebuilds byte for byte ([#1154](https://github.com/JarryShaw/PyPCAPKit/issues/1154)). - Quick-Start options did not round-trip. The IPv4, HOPOPT and IPv6-Opts readers took the function from `schema.func`, which only a parsed schema carries, so a built option raised `AttributeError`; HOPOPT and IPv6-Opts also sized the 8-octet option as 5, misreading the nonce with a `packet length < 0` warning ([#1091](https://github.com/JarryShaw/PyPCAPKit/issues/1091)). TCP's Quick-Start Response reader read one octet past the option, and its maker produced a negative rate code below 40 kbps; such a rate now rounds down to zero, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) permits ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)). - `Ethernet`, `ARP` and `PCAPNG` packed MAC addresses as ASCII hex text. `Ethernet._make_mac_addr` returned the separator-stripped hex string and `BytesField(length=6)` truncated it, so `Ethernet()` put `303030303030` on the wire for an all-zero address ([#1123](https://github.com/JarryShaw/PyPCAPKit/issues/1123)). `ARP._make_addr_resolve`, `PCAPNG._make_mac_addr` and `_make_eui_addr` did the same; all now decode the hex to octets, and `from_data` rebuilds each byte for byte ([#1134](https://github.com/JarryShaw/PyPCAPKit/issues/1134)). - TCP's User Timeout maker wrote `length=3` where [RFC 5482](https://datatracker.ietf.org/doc/html/rfc5482) and the reader require 4, so every option it built failed to parse with `ProtocolError` ([#1120](https://github.com/JarryShaw/PyPCAPKit/issues/1120)). - **a breaking change to** the HTTP/2 frame `length`: it counted the 9-octet frame header, where [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) section 4.1 counts the payload alone, so `make()` wrote `00000e` for a 5-octet PRIORITY frame and `read` rejected RFC-conformant frames -- and, since `_read_http_priority` checked for 9, pcapkit's own PRIORITY frame. The length and every per-type check now exclude the header. Captures written to the old convention no longer parse; the PRIORITY, RST_STREAM, SETTINGS (with and without ACK), PING, WINDOW_UPDATE, DATA, PUSH_PROMISE and GOAWAY frames that Scapy's `contrib.http2` packs now parse and rebuild byte for byte ([#1121](https://github.com/JarryShaw/PyPCAPKit/issues/1121)). - **a breaking change to** `ARP.make()`: `hlen` and `plen` are derived from the packed `sha` and `spa` when omitted, and an address that disagrees with its length field raises `ProtocolError` where it used to be truncated silently -- `ARP(ptype=IPv6, spa='::1')` wrote `plen=4` and four octets of the address. A non-IP `str` protocol address packs as hex octets rather than ASCII text, as hardware addresses do since [#1134](https://github.com/JarryShaw/PyPCAPKit/issues/1134), and non-hex input raises `ProtocolError` ([#1139](https://github.com/JarryShaw/PyPCAPKit/issues/1139)). - HTTP/2 `SETTINGS` built each setting pair as a `SchemaField` with no length, so a well-formed frame warned `packet length < 0` twice per pair. The item now has its fixed 6-octet size, as TCP SACK and SCTP gap-ack items already do ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140)). -- `HTTP._guess_version` took almost any binary for HTTP/2 on its fall-through: nine zero octets read as a DATA frame. The HTTP/2 trial now runs only on a header with a type [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) defines, no undefined flag, the reserved bit clear and a stream that type is sent on. Over 5,000 fuzzed zero-heavy buffers, HTTP/2 labels fell from 566 to 101, every one a legal DATA header. Explicit `version=2` parsing is unchanged ([#1141](https://github.com/JarryShaw/PyPCAPKit/issues/1141)). +- `HTTP._guess_version()` took almost any binary for HTTP/2 on its fall-through: nine zero octets read as a DATA frame. The HTTP/2 trial now runs only on a header with a type [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) defines, no undefined flag, the reserved bit clear and a stream that type is sent on. Over 5,000 fuzzed zero-heavy buffers, HTTP/2 labels fell from 566 to 101, every one a legal DATA header. Explicit `version=2` parsing is unchanged ([#1141](https://github.com/JarryShaw/PyPCAPKit/issues/1141)). - **a breaking change to** `TCP.make()`'s options: `_make_tcp_options` dropped the `NOP` and `EOOL` options it was given and padded each other option to 4 octets separately, so `from_data` rebuilt none of the segments in `http.pcap` with their original header octets. Options are now emitted as given and padded once at the end -- zero octets after a trailing `EOOL`, otherwise `NOP` options and an `EOOL` -- so a built header can be shorter than before; every TCP segment of the measured sample captures rebuilds its header octets, and `options-*.pcap` regenerate unchanged ([#1151](https://github.com/JarryShaw/PyPCAPKit/issues/1151)). - **a breaking change to** `from_data` on a capture truncated by snaplen: `IPv4`, `IPv6` and `UDP` rebuilt their length field from the payload at hand, writing the recorded length, so `big_endian.pcap` frame 3 came back as `45000052` where `450004a2` was on the wire. `_make_data` now passes the parsed length through to `make()`'s new `total_length` (IPv4, UDP) or `payload_length` (IPv6) keyword, which computes the length as before when omitted, so `IPv4(payload=...)` is unchanged ([#1155](https://github.com/JarryShaw/PyPCAPKit/issues/1155)). -- **a breaking change to** the layer chain of a capture cut inside a header: a next layer parsed from too few captured octets was zero-filled, and `from_data` rebuilt the fabricated octets. `IPv4.read` handed UDP the declared payload length when nothing past the IPv4 header was captured, so `test.pcapng` frame 3 rebuilt with 8 invented UDP octets ([#1167](https://github.com/JarryShaw/PyPCAPKit/issues/1167)). Every `_import_next_layer` now goes through one guard: no captured octets gives `NoPayload`, and fewer than the next layer's `__length_hint__` gives `Raw` unless the parsed header fits anyway, as a 12-octet L2TPv2 ZLB does. A header-only IPv6 packet is `IPv6` rather than `IPv6:UDP:Raw`, and frame 3 is `Ethernet:Raw`, rebuilt byte for byte ([#1170](https://github.com/JarryShaw/PyPCAPKit/issues/1170)). The guard needed honest hints, so `IPv6_Route` (4), `MH` (6), `HOPOPT` and `IPv6_Opts` (2) now hint their 8-octet minimum header ([#1173](https://github.com/JarryShaw/PyPCAPKit/issues/1173)), and so does the `IPv6_Ext` fallback, which hinted 2 ([#1178](https://github.com/JarryShaw/PyPCAPKit/issues/1178)). +- **a breaking change to** the layer chain of a capture cut inside a header: a next layer parsed from too few captured octets was zero-filled, and `from_data` rebuilt the fabricated octets. `IPv4.read()` handed UDP the declared payload length when nothing past the IPv4 header was captured, so `test.pcapng` frame 3 rebuilt with 8 invented UDP octets ([#1167](https://github.com/JarryShaw/PyPCAPKit/issues/1167)). Every `_import_next_layer` now goes through one guard: no captured octets gives `NoPayload`, and fewer than the next layer's `__length_hint__` gives `Raw` unless the parsed header fits anyway, as a 12-octet L2TPv2 ZLB does. A header-only IPv6 packet is `IPv6` rather than `IPv6:UDP:Raw`, and frame 3 is `Ethernet:Raw`, rebuilt byte for byte ([#1170](https://github.com/JarryShaw/PyPCAPKit/issues/1170)). The guard needed honest hints, so `IPv6_Route` (4), `MH` (6), `HOPOPT` and `IPv6_Opts` (2) now hint their 8-octet minimum header ([#1173](https://github.com/JarryShaw/PyPCAPKit/issues/1173)), and so does the `IPv6_Ext` fallback, which hinted 2 ([#1178](https://github.com/JarryShaw/PyPCAPKit/issues/1178)). - **a breaking change to** `L2TPv2`, against [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) section 3.1. `Type` was inverted; it is now `Data = 0` and `Control = 1`, regenerated in `pcapkit.const` from the fixed `pcapkit.vendor` table, so the same bytes parse to the other `flags.type`, and a default `make()` writes T=0 where it wrote T=1. A control message now gets `L` and a Length field automatically, and the offset pad, which `from_data` zero-filled, is kept as `info.padding` -- a new key in `info` and in dumped output ([#1175](https://github.com/JarryShaw/PyPCAPKit/issues/1175)). `from_data` also cleared `L` and dropped the Length field: `_make_data` returned it as `length`, which `ProtocolBase` takes as the parse length. `make()`'s `length` keyword is now `total_length`, written as given; a call still passing `length` loses `L` silently ([#1172](https://github.com/JarryShaw/PyPCAPKit/issues/1172)). `make()` also gains `length_flag` and `padding`, and raises `ProtocolError` for a `total_length` with `length_flag=False`. - **a breaking change to** `IPv6.from_data`: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden `__exthdr__` key, absent from `to_dict` and dumped output, and re-emitted ahead of the upper layer. Sample frames that rebuilt wrong now rebuild exactly, though some that had rebuilt with wrong bytes raised `AttributeError` instead ([#1177](https://github.com/JarryShaw/PyPCAPKit/issues/1177)), from older defects fixed with it: `HIP.alias` read `_info` before `read` had set it, which also broke a standalone HIP parse and `HIP.from_data` ([#1184](https://github.com/JarryShaw/PyPCAPKit/issues/1184)), and the MPL option's maker called `bit_length()` on the `IPv6Address` seed of an S=0 option. MPL in `HOPOPT` and `IPv6_Opts` now takes `S` from the parsed option or a new `seed_type` keyword, and a missing or oversized seed raises `ProtocolError` ([#1185](https://github.com/JarryShaw/PyPCAPKit/issues/1185)). With both fixed, every IPv6 frame of the `options-*.pcap` captures rebuilds byte for byte. - **a breaking change to** the SMF_DPD option's maker in `HOPOPT` and `IPv6_Opts`: it overwrote the parsed DPD mode, so H-DPD rebuilt as I-DPD, and inferred the TaggerID type from the Tagger ID's length, so a 4-octet `DEFAULT` Tagger ID rebuilt as IPv4. Both now come from the parsed option, or from `mode` and a new `tid_type` keyword. A Tagger ID that does not fit its type raises `ProtocolError`, so a 17-octet one does where it raised `FieldValueError`, and `tid=b''` builds a NULL Tagger ID where it raised `ValueError` ([#1190](https://github.com/JarryShaw/PyPCAPKit/issues/1190)). An H-DPD option with an empty Hash Assist Value raises `ProtocolError` where it raised a bare `IndexError` ([#1193](https://github.com/JarryShaw/PyPCAPKit/issues/1193)). @@ -305,19 +305,19 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - PCAP-NG: constructed and rebuilt blocks ([#1339](https://github.com/JarryShaw/PyPCAPKit/pull/1339)), lengths and padding ([#1373](https://github.com/JarryShaw/PyPCAPKit/pull/1373)), blocks and options as parsed ([#1378](https://github.com/JarryShaw/PyPCAPKit/pull/1378)), further follow-ups ([#1412](https://github.com/JarryShaw/PyPCAPKit/pull/1412)), malformed journal entries and short packet blocks ([#1420](https://github.com/JarryShaw/PyPCAPKit/pull/1420)), an SHB or IDB shorter than its fixed fields ([#1422](https://github.com/JarryShaw/PyPCAPKit/issues/1422), [#1425](https://github.com/JarryShaw/PyPCAPKit/pull/1425)), and captured padding octets ([#1441](https://github.com/JarryShaw/PyPCAPKit/pull/1441)). - a cut PCAP header or PCAP-NG block ([#1470](https://github.com/JarryShaw/PyPCAPKit/issues/1470), [#1475](https://github.com/JarryShaw/PyPCAPKit/pull/1475)). -- `Schema.pack` keeps option padding, `SchemaField` spans and iterated keys ([#1374](https://github.com/JarryShaw/PyPCAPKit/pull/1374)); a `SchemaField`'s unread octets survive `from_data` rather than rebuilding zero-filled ([#1380](https://github.com/JarryShaw/PyPCAPKit/issues/1380), [#1397](https://github.com/JarryShaw/PyPCAPKit/pull/1397)); a header cut inside a fixed-width field rebuilds as captured ([#1465](https://github.com/JarryShaw/PyPCAPKit/pull/1465)); and each non-final `Schema` subclass is finalised on its own state ([#1498](https://github.com/JarryShaw/PyPCAPKit/issues/1498), [#1499](https://github.com/JarryShaw/PyPCAPKit/pull/1499)). +- `Schema.pack()` keeps option padding, `SchemaField` spans and iterated keys ([#1374](https://github.com/JarryShaw/PyPCAPKit/pull/1374)); a `SchemaField`'s unread octets survive `from_data` rather than rebuilding zero-filled ([#1380](https://github.com/JarryShaw/PyPCAPKit/issues/1380), [#1397](https://github.com/JarryShaw/PyPCAPKit/pull/1397)); a header cut inside a fixed-width field rebuilds as captured ([#1465](https://github.com/JarryShaw/PyPCAPKit/pull/1465)); and each non-final `Schema` subclass is finalised on its own state ([#1498](https://github.com/JarryShaw/PyPCAPKit/issues/1498), [#1499](https://github.com/JarryShaw/PyPCAPKit/pull/1499)). - `from_data` rebuilds the payload from `info.to_dict()` ([#1447](https://github.com/JarryShaw/PyPCAPKit/issues/1447), [#1452](https://github.com/JarryShaw/PyPCAPKit/pull/1452)). - building a PCAP frame with `snaplen` 0 wrote a record with no packet data ([#1385](https://github.com/JarryShaw/PyPCAPKit/issues/1385), [#1410](https://github.com/JarryShaw/PyPCAPKit/pull/1410)); an out-of-range `if_tzone` is kept as captured and every `fromtimestamp` failure is caught ([#1340](https://github.com/JarryShaw/PyPCAPKit/issues/1340), [#1429](https://github.com/JarryShaw/PyPCAPKit/pull/1429)). -- registration: the parser registrars warn only when the entry really changes ([#1362](https://github.com/JarryShaw/PyPCAPKit/issues/1362), [#1400](https://github.com/JarryShaw/PyPCAPKit/pull/1400)); re-registering a stored `ModuleDescriptor` keeps it ([#1364](https://github.com/JarryShaw/PyPCAPKit/issues/1364), [#1415](https://github.com/JarryShaw/PyPCAPKit/pull/1415)); restoring a registration puts back the shipped descriptor, including through `PCAPNG.register` ([#1533](https://github.com/JarryShaw/PyPCAPKit/pull/1533), [#1573](https://github.com/JarryShaw/PyPCAPKit/pull/1573)); and unregistering a custom protocol removes its name from `__proto__` ([#1505](https://github.com/JarryShaw/PyPCAPKit/issues/1505), [#1530](https://github.com/JarryShaw/PyPCAPKit/pull/1530)). +- registration: the parser registrars warn only when the entry really changes ([#1362](https://github.com/JarryShaw/PyPCAPKit/issues/1362), [#1400](https://github.com/JarryShaw/PyPCAPKit/pull/1400)); re-registering a stored `ModuleDescriptor` keeps it ([#1364](https://github.com/JarryShaw/PyPCAPKit/issues/1364), [#1415](https://github.com/JarryShaw/PyPCAPKit/pull/1415)); restoring a registration puts back the shipped descriptor, including through `PCAPNG.register()` ([#1533](https://github.com/JarryShaw/PyPCAPKit/pull/1533), [#1573](https://github.com/JarryShaw/PyPCAPKit/pull/1573)); and unregistering a custom protocol removes its name from `__proto__` ([#1505](https://github.com/JarryShaw/PyPCAPKit/issues/1505), [#1530](https://github.com/JarryShaw/PyPCAPKit/pull/1530)). - fragments: an IPv6 extension header behind a first fragment is dissected as without it ([#1539](https://github.com/JarryShaw/PyPCAPKit/issues/1539), [#1540](https://github.com/JarryShaw/PyPCAPKit/pull/1540)), and a later IPv4 or IPv6 fragment's data is raw, not an upper-layer header ([#1545](https://github.com/JarryShaw/PyPCAPKit/issues/1545), [#1552](https://github.com/JarryShaw/PyPCAPKit/pull/1552), [#1553](https://github.com/JarryShaw/PyPCAPKit/issues/1553), [#1562](https://github.com/JarryShaw/PyPCAPKit/pull/1562)). - IPv4: a Total Length of 0 is read as TSO, and one inside the header as bogus ([#1547](https://github.com/JarryShaw/PyPCAPKit/issues/1547), [#1554](https://github.com/JarryShaw/PyPCAPKit/pull/1554)). - IPv6: a jumbogram's payload is dissected up to its Jumbo Payload Length ([#1434](https://github.com/JarryShaw/PyPCAPKit/issues/1434), [#1440](https://github.com/JarryShaw/PyPCAPKit/pull/1440)), a jumbogram [RFC 2675](https://datatracker.ietf.org/doc/html/rfc2675) calls an error is rejected ([#1687](https://github.com/JarryShaw/PyPCAPKit/issues/1687), [#1713](https://github.com/JarryShaw/PyPCAPKit/pull/1713)), and so is a header whose Version is not 6 ([#1680](https://github.com/JarryShaw/PyPCAPKit/issues/1680), [#1723](https://github.com/JarryShaw/PyPCAPKit/pull/1723)). - the warnings of a next layer replaced by `Raw` are dropped ([#1580](https://github.com/JarryShaw/PyPCAPKit/issues/1580), [#1588](https://github.com/JarryShaw/PyPCAPKit/pull/1588)). - IPv6-Route's data-type readers name the routing type in their `ProtocolError`, where each message read `[TypeNo ]` ([#442](https://github.com/JarryShaw/PyPCAPKit/issues/442)), and `HTTP(..., version=1)` parses from the frame's data rather than the drained file, where it raised `ValueError` ([#447](https://github.com/JarryShaw/PyPCAPKit/issues/447), [#451](https://github.com/JarryShaw/PyPCAPKit/pull/451)). -- `Schema.pack` accepts a `tuple` for a `ListField`, as the data models declare, so HIP and MH options that hold one round-trip ([#476](https://github.com/JarryShaw/PyPCAPKit/issues/476), [#480](https://github.com/JarryShaw/PyPCAPKit/pull/480)). +- `Schema.pack()` accepts a `tuple` for a `ListField`, as the data models declare, so HIP and MH options that hold one round-trip ([#476](https://github.com/JarryShaw/PyPCAPKit/issues/476), [#480](https://github.com/JarryShaw/PyPCAPKit/pull/480)). - a Mobility Header CGA extension reports its length with its 4-octet header, where every parsed one read 2 octets short ([#512](https://github.com/JarryShaw/PyPCAPKit/issues/512), [#517](https://github.com/JarryShaw/PyPCAPKit/issues/517), [#528](https://github.com/JarryShaw/PyPCAPKit/pull/528)). - TCP's SACK option enforces the `8n + 2` length [RFC 2018](https://datatracker.ietf.org/doc/html/rfc2018) gives it, as its docstring promised ([#519](https://github.com/JarryShaw/PyPCAPKit/issues/519), [#535](https://github.com/JarryShaw/PyPCAPKit/pull/535)). -- `IPv4.from_data()` rebuilds a parsed datagram: `Schema.pack` checks a payload against `ProtocolBase`, and option padding ends in an `EOOL` option rather than a bare wire code ([#506](https://github.com/JarryShaw/PyPCAPKit/issues/506), [#536](https://github.com/JarryShaw/PyPCAPKit/pull/536)). +- `IPv4.from_data()` rebuilds a parsed datagram: `Schema.pack()` checks a payload against `ProtocolBase`, and option padding ends in an `EOOL` option rather than a bare wire code ([#506](https://github.com/JarryShaw/PyPCAPKit/issues/506), [#536](https://github.com/JarryShaw/PyPCAPKit/pull/536)). - IPv4's SEC option sets the termination bit its own reader requires, sizes the bitmap from the bit count, and SID is 16 bits wide ([#534](https://github.com/JarryShaw/PyPCAPKit/issues/534), [#537](https://github.com/JarryShaw/PyPCAPKit/issues/537), [#543](https://github.com/JarryShaw/PyPCAPKit/pull/543)). - the MPTCP option header reads `length` from bit 8, per [RFC 8684 Section 3](https://datatracker.ietf.org/doc/html/rfc8684#section-3), rather than bit 1 ([#553](https://github.com/JarryShaw/PyPCAPKit/issues/553), [#558](https://github.com/JarryShaw/PyPCAPKit/pull/558)). - Mobility Header enumerations defined inline from their RFCs (`FastBindingAcknowledgmentStatus`, `IPv6AddressPrefixCode`, `LocalizedRoutingStatus`, `LMAAddressCode`) raise `EnumValueError` for an unassigned value rather than minting `Unassigned_N` ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#889](https://github.com/JarryShaw/PyPCAPKit/pull/889)), and each inherits `get` and its `default` from `EnumLookup` rather than overriding it ([#935](https://github.com/JarryShaw/PyPCAPKit/issues/935), [#940](https://github.com/JarryShaw/PyPCAPKit/pull/940)). @@ -370,8 +370,8 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r #### Fixed - `re.sub` sites under `pcapkit/vendor/` passed `re.MULTILINE` as the fourth *positional* argument, which is `count`, not `flags`: substitution capped at 8 matches, and a `DeprecationWarning` on Python 3.13+ (a future `TypeError`). Found by an AST sweep rather than grep, since most put the flag on a continuation line. The flag moves to `flags=` at every site, in `default.py`, `hip/eddsa_curve.py`, `http/frame.py`, `http/method.py`, `http/status_code.py`, `ipv4/router_alert.py`, `ipv6/option.py`, `ipv6/router_alert.py`, `ipv6/tagger_id.py`, `reg/ethertype.py`, `tcp/flags.py`, `tcp/mp_tcp_option.py` and `tcp/option.py`. Behaviour preserving: the shared pattern (`r'\r*\n'`) has no MULTILINE-sensitive anchor, so no `const/` regeneration was needed ([#796](https://github.com/JarryShaw/PyPCAPKit/issues/796)). -- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)) with `RecursionError` in `tests/const/test_const_enum_lookup.py`, `tests/const/test_const_enum_no_mint.py` and `tests/protocols/misc/test_pcapng_unit.py`. `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not these *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled only some regenerations, and neither crawler's was among them. Both crawlers now collapse to the single returning line the other registries use, and their const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process`'s shape legitimately breaks. The guard sees only some crawlers; the rest are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). -- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about failed sub-vendors without failing as a whole -- the pieces below land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). +- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)) with `RecursionError` in `tests/const/test_const_enum_lookup.py`, `tests/const/test_const_enum_no_mint.py` and `tests/protocols/misc/test_pcapng_unit.py`. `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not these *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled only some regenerations, and neither crawler's was among them. Both crawlers now collapse to the single returning line the other registries use, and their const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process()`'s shape legitimately breaks. The guard sees only some crawlers; the rest are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). +- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about failed sub-vendors without failing as a whole -- the pieces below land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path()`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). - follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: an unpinned invariant and inaccurate prose and lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: a test asserts that a `KeyboardInterrupt` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose `_dest_path` is an instance-method-only override fails the *class* call `_snapshot_and_restore` makes but not the *instance* call inside `__init__`, so that target runs unprotected and **succeeds silently**. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**: the belief that R1720 never applies to `try`/`except`/`else` was a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does), and under the `Makefile`'s flags, which CI runs, pylint flags the `else:`, which is deliberate because it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, since `copy2` reopens the path) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). - `TLSKeyLabel` tracked the Mozilla NSS key-log wiki, but `draft-ietf-opsawg-pcapng-06` section 4.7 now points at `[I-D.ietf-tls-keylogfile]` (draft-05), published as [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850), "The SSLKEYLOGFILE Format for TLS", whose section 4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, `TLSKeyLabel` was missing `ECH_SECRET` and `ECH_CONFIG` ([RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) sections 2.3 and 4.2), so `TLSKeyLabel('ECH_SECRET')` raised `ValueError` on a legitimate Decryption Secrets Block; both are added, `ECH_SECRET` keeping the file's `# nosec B105` bandit convention for `*_SECRET` members. `RSA` has no row in [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850)'s registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since removing a public member is breaking, with a docstring comment marking it the historical exception. `pcapkit/vendor/pcapng/secrets_type.py`'s stale `# NSS Key Log Format` comment is repointed at RFC 9850, and `TLSKeyLabel`'s docstring cites it. A new test pins the member set against RFC 9850 plus the `RSA` exception. Closes [#882](https://github.com/JarryShaw/PyPCAPKit/issues/882) ([#883](https://github.com/JarryShaw/PyPCAPKit/pull/883)). - both spellings of an IANA citation render as `:rfc:` roles in the generated `pcapkit.const` docstrings ([#1664](https://github.com/JarryShaw/PyPCAPKit/issues/1664), [#1673](https://github.com/JarryShaw/PyPCAPKit/pull/1673)). @@ -407,7 +407,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - `engine='pyshark'` had no CI job driving it through a real `tshark`-parsed capture. `pypcap-parity` installs `pyshark` but its `apt-get install` named no `tshark` package, so `PyShark.unsupported_reason()` declined the engine, the extractor fell back to the built-in parser, and the only real-extraction assertion, `assertGreater(extractor.length, 0)`, passed against the fallback's output: a vacuous pass invisible in every skip count. `pypcap-parity` now installs `tshark` with the debconf pre-seed `engine-tests` carries, and the test proves which engine ran (no `EngineWarning`, `extractor._exnam == 'pyshark'`, a `PyShark` instance), as the PyPCAP parity test does. Only `tests/integration/test_engine_runtime.py` and `test_engine_parity.py` change; the length assertion is kept ([#846](https://github.com/JarryShaw/PyPCAPKit/pull/846)). - `engine-tests` ran a Python-version leg per interpreter regardless of which engines that interpreter could install, so `pyproject.toml`'s `PyPCAPFile = ["pypcapfile; python_version < '3.12'"]` marker resolved to *nothing* on 3.12--3.14, yet the `Engines Python 3.12/3.13/3.14` legs passed and, once required, gated merges while exercising an engine that was never installed. Rebuilt as a Python x engine matrix in which every cell has an expected outcome: `supported` cells install through the real extra and must pass; `unsupported` cells (`PyPCAPFile` on 3.12--3.15, `PyShark` on 3.14/3.15) install by raw `pip install` to bypass the marker and assert the engine's own `PYTHON_CEILING` decline; and `not-installable` cells (`PyPCAP` on 3.12--3.15) log a genuine C-extension build failure and skip the test step. The install step never passes silently in either direction. Python 3.15 is `continue-on-error` throughout. Job names change shape (`Engines Python X` becomes `Engines Python X (engine)`), so the branch-protection ruleset's required-check list was updated in the same window, lest a required check that stopped existing block every merge ([#849](https://github.com/JarryShaw/PyPCAPKit/pull/849)). - Ruleset `23497679`'s `required_status_checks` names exact check contexts, and GitHub rulesets match literally with no wildcard, so the list needs hand-editing on every matrix change. The old `Engines Python ` names in it are dead since [#849](https://github.com/JarryShaw/PyPCAPKit/pull/849); the `Compat Python 3.10`-`3.14` names stay live, emitted by `python-compatibility.yml`, and are untouched. Adds a job, `required-checks` (context `Required checks passed`), depending on `[test, integration, engine-tests, pypcap-parity]` with `if: always() && inputs.gate-only != true` and an explicit per-dependency check that fails loudly, naming the job, on anything but `success`: a bare `needs:` list would leave it *skipped* rather than failed when a dependency failed, and GitHub's docs list a skipped required check as passing. That context is meant to replace the ruleset's `test`, `integration`, `Engines` and `pypcap-parity` entries once the ruleset is updated separately, as a repository setting outside the change; the `Compat` names are excluded from the replacement. Only `.github/workflows/unit-tests.yml` changes; no library or test code ([#856](https://github.com/JarryShaw/PyPCAPKit/pull/856)). -- the prose the post-[#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) defect wave added or touched, checked against the code; no code changes. `MesgIDOption.seconds` in the Mobility Header schema said seconds since 1970 where it holds an [RFC 1305](https://datatracker.ietf.org/doc/html/rfc1305) NTP count from 1900; `HTTP._guess_version` comments predated the HTTP/2 gate; and the `_parse_next_layer` note miscounted the captured octets. `Extractor._open_output` now says only the PCAP engine passes header fields, `Extractor.run` and the PCAP-NG engine's `run` list the `FormatError` for PCAP output from a PCAP-NG input, and the extraction and engine pages document the PCAP-output path ([#1179](https://github.com/JarryShaw/PyPCAPKit/pull/1179), [#1180](https://github.com/JarryShaw/PyPCAPKit/pull/1180), [#1186](https://github.com/JarryShaw/PyPCAPKit/pull/1186), [#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103)). +- the prose the post-[#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) defect wave added or touched, checked against the code; no code changes. `MesgIDOption.seconds` in the Mobility Header schema said seconds since 1970 where it holds an [RFC 1305](https://datatracker.ietf.org/doc/html/rfc1305) NTP count from 1900; `HTTP._guess_version()` comments predated the HTTP/2 gate; and the `_parse_next_layer` note miscounted the captured octets. `Extractor._open_output()` now says only the PCAP engine passes header fields, `Extractor.run()` and the PCAP-NG engine's `run` list the `FormatError` for PCAP output from a PCAP-NG input, and the extraction and engine pages document the PCAP-output path ([#1179](https://github.com/JarryShaw/PyPCAPKit/pull/1179), [#1180](https://github.com/JarryShaw/PyPCAPKit/pull/1180), [#1186](https://github.com/JarryShaw/PyPCAPKit/pull/1186), [#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103)). - **a breaking change to** the `all` extra: it carries the core addons only -- `cli`, `crypto` and `NGAP`. The third-party engines 1.4.x bundled in it (`dpkt`, `scapy` and `pyshark`) and the vendor crawler's `requests` and `beautifulsoup4` are no longer in it; the engines keep their own `DPKT`, `Scapy` and `PyShark` extras and the crawler its `vendor` extra, and `pypcapfile` has a new `PyPCAPFile` extra of its own. A new `dev` extra carries what CI needs ([#910](https://github.com/JarryShaw/PyPCAPKit/issues/910), [#914](https://github.com/JarryShaw/PyPCAPKit/pull/914)). - the `Scapy` and `dev` extras require `scapy>=2.6.0` ([#1572](https://github.com/JarryShaw/PyPCAPKit/issues/1572), [#1578](https://github.com/JarryShaw/PyPCAPKit/pull/1578)), and the `docs` extra `Sphinx>=7.3`, the first release with `versionremoved` ([#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)), and `sphinx-autodoc-typehints==3.15.0` ([#1856](https://github.com/JarryShaw/PyPCAPKit/issues/1856), [#1877](https://github.com/JarryShaw/PyPCAPKit/pull/1877)). The `test` extra gains `psutil`, `markdown-it-py` and `lxml`, which the scripts under `util/` use ([#1894](https://github.com/JarryShaw/PyPCAPKit/issues/1894), [#1896](https://github.com/JarryShaw/PyPCAPKit/pull/1896)). - the READMEs of `examples/benchmark` and `examples/legacy_smoke` are Markdown, `README.md` rather than `README.rst`, and `MANIFEST.in` ships them through `recursive-include examples *.md`. Stale build leftovers go: `setup.py`'s `bdist_wheel` override, since `build_py` and `install` already convert the tree the wheel archives, and `MANIFEST.in`'s `prune test`, which named no directory; `make profile` runs in `examples/legacy_smoke` ([#1773](https://github.com/JarryShaw/PyPCAPKit/issues/1773), [#1780](https://github.com/JarryShaw/PyPCAPKit/pull/1780)). `MANIFEST.in` also drops the lines that matched nothing -- `global-include *.rst`, `include pcapkit/_extern` and `exclude pcapkit/protocols/*/NotImplemented` -- and the source distribution and wheel are unchanged ([#1794](https://github.com/JarryShaw/PyPCAPKit/issues/1794), [#1813](https://github.com/JarryShaw/PyPCAPKit/pull/1813)). @@ -417,7 +417,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - places where a documentation page contradicted the code ([#413](https://github.com/JarryShaw/PyPCAPKit/pull/413)), ambiguous cross-references and autodoc signature failures ([#416](https://github.com/JarryShaw/PyPCAPKit/pull/416)), and `Extractor`'s documented exception plus phantom or stale `Args:` labels ([#501](https://github.com/JarryShaw/PyPCAPKit/pull/501)). - both halves of what the `Changelog drift` gate told an author, in `util/changelog_md.py`. `ResidualMarkupError` named the entry file but numbered its lines against the *converted* body, which rule 6 joins onto one line per block, so a reported number could not reach most of the file, and roles on separate source lines were all reported as one line. The conversion now carries a source map, so every complaint cites a line of the file the message names, one per construct, in the entry's order ([#588](https://github.com/JarryShaw/PyPCAPKit/issues/588)). Rule 2 accepted only the bare-number spelling of Sphinx's `:rfc:` role, so [RFC 6554 Section 3](https://datatracker.ietf.org/doc/html/rfc6554#section-3) fell past it and was reported as a role the rules do not cover; both spellings now convert, the anchor carried into the link target and the link text taken from Sphinx's own ([#592](https://github.com/JarryShaw/PyPCAPKit/issues/592)). Regenerating `CHANGELOG.md` is byte-identical over every committed entry: the fix changes what the gate *says*, not what it emits. -- `examples/generators/options.py`'s `TCP_BASE` spelled header fields with names `TCP.make` does not declare, so every option fixture in `examples/captures/options-tcp.pcap` was built from values the generator had not asked for. `make` ends in `**kwargs` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. `'seq': 1` was `seq_no`, so every captured frame carried sequence number 0; `'urgent_pointer'` was `urgent` (requested value and default both 0); and `'ack_flag': False` was `ack` while `'ack': 0` bound to `ack` itself, the acknowledgement *flag* rather than the number (`ack_no`), so that pair was wrong both ways. Each is renamed to the parameter `make` declares, keeping the intended values. `TCP(**TCP_BASE).info.seq` was 0 against a declared 1 and is now 1, and `options-tcp.pcap` changes only in the low octet of each frame's sequence number, with no case status or `EXPECTED_FAILURES` entry moving. The module's other base mappings were audited: the rest pass nothing their `make` does not declare, and HIP's `extension` is declared by `HIP.read`, this library's documented way of forwarding a read-path keyword, not another instance ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). +- `examples/generators/options.py`'s `TCP_BASE` spelled header fields with names `TCP.make()` does not declare, so every option fixture in `examples/captures/options-tcp.pcap` was built from values the generator had not asked for. `make` ends in `**kwargs` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. `'seq': 1` was `seq_no`, so every captured frame carried sequence number 0; `'urgent_pointer'` was `urgent` (requested value and default both 0); and `'ack_flag': False` was `ack` while `'ack': 0` bound to `ack` itself, the acknowledgement *flag* rather than the number (`ack_no`), so that pair was wrong both ways. Each is renamed to the parameter `make` declares, keeping the intended values. `TCP(**TCP_BASE).info.seq` was 0 against a declared 1 and is now 1, and `options-tcp.pcap` changes only in the low octet of each frame's sequence number, with no case status or `EXPECTED_FAILURES` entry moving. The module's other base mappings were audited: the rest pass nothing their `make` does not declare, and HIP's `extension` is declared by `HIP.read()`, this library's documented way of forwarding a read-path keyword, not another instance ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). - `util/bump_version.py` left `CITATION.cff` naming the previous release. Nothing else maintains that file, so every bump since [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) would have stranded the `version` and `date-released` that GitHub's "Cite this repository" button, citation managers, Zenodo and dependency inventories read. Both fields now move with `__version__`, together, since the file's header says both describe the newest *published* release and moving only one would assert that 1.5.0b5 was released on the day 1.5.0b4 was. The date is UTC, because seven of the thirty most recent bumps were made late evening US-Eastern, where a local date is a day behind the publish; the bump triggers `create-release.yml`, the median gap to the PyPI upload is three minutes, and the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment header, key ordering and each field's quoting survive (`cff-version` and a `references` entry's own `version` are anchored out at column zero), and the result is checked with `cffconvert --validate`. An absent file is reported on stderr and skipped rather than failing the vendor cron before its `git commit`, which would discard a whole registry crawl for a documentation file; a file with no `version` field raises before anything is written, since reporting success while rewriting nothing is the staleness being fixed. The script gains a `main()` guard (it ran the whole bump at import, hence no testable surface), and the `import pcapkit` fallback in its version reader, which returned `"1.5.0b4'\n"` (closing quote and newline, which `packaging` rejects), is fixed; it never worked and went unnoticed because the only caller installs the package first. A new gate asserts the committed file still names the packaged version, covering hand-made version changes that never run this script ([#625](https://github.com/JarryShaw/PyPCAPKit/pull/625)). - `LICENSE`'s copyright notice began the term at 2018, a year after the work it covers. The repository's first commit is `c57f7d0b7`, dated 2017-11-07, so the notice understated the term and contradicted `docs/source/conf.py`, whose Sphinx footer is `f'2017-{datetime.date.today().year}, Jarry Shaw'`. The wrong start year survived every maintenance pass because each looked only at the other end of the range: `bc836cfa2` (2020-05-31) introduced `2018-2020` when the BSD-3-Clause text replaced MPL 2.0, and hand bumps to `2018-2022`, `2018-2023` and `2018-2026` ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)) each copied `2018` forward. 2017 is **restored** rather than newly asserted: the initial MIT `LICENSE` read `Copyright (c) 2017 Jarry Shaw`, and the GPL v3 era carried `Copyright (C) 2017 Jarry Shaw` in its appendix; it was lost on 2018-12-08, when `4f3e00d5f` relicensed to Apache 2.0 under `Copyright 2018 Jarry Shaw` and `1c69341dc` replaced that with MPL 2.0 text with no author notice. The end year is **dropped** rather than automated: the notice ships inside the sdist and wheel, so its text is fixed at build time in every copy already downloaded and cannot be computed like the docs footer; copyright subsists from creation whether or not a notice names the current year; and BSD-3-Clause's canonical form is a singular `Copyright (c) `. A range was six years of hand maintenance buying nothing, and automation was considered and rejected, since automating a value that need not be current is worse than not carrying it. `conf.py` is untouched (a self-maintaining footer showing a range is conventional and correct). The licence body is unaltered: the change is confined to the copyright line, and the `DAMAGE.` that [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) repaired from `DAMAGE.s` is still clean ([#630](https://github.com/JarryShaw/PyPCAPKit/pull/630)). - `CITATION.cff` shipped in no source distribution. `MANIFEST.in` has an `include` each for `README.md`, `LICENSE` and `CHANGELOG.md` but none for the citation file, and its `global-include` patterns (`*.rst`, `*.py`) cannot match a `.cff`. Removing those `include` lines and rebuilding shows `README.md` and `LICENSE` shipping anyway (setuptools adds the latter from `license_files`, recorded as `License-File: LICENSE` in `PKG-INFO`) while `CHANGELOG.md` vanishes, so only `CHANGELOG.md` is load-bearing, and a citation file, which no packaging default covers, is in the same position. The gap is invisible from the web UI, because GitHub renders "Cite this repository" from the repository; citation managers, Zenodo and dependency inventories read the published artifact, the surface that was missing it. [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) flagged the omission and [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) did not address it. It matters because [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) taught `util/bump_version.py` to keep the file's `version` and `date-released` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets `RepositoryCitationTests` in `tests/project/test_bump_version.py` (the gate [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) added for hand-authored bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is not shipped in the source distribution". `python -m build --sdist` now differs from before only by the archive entry `pypcapkit-1.5.0b4/CITATION.cff`, the shipped copy is byte-identical, and `twine check --strict` passes on both ([#631](https://github.com/JarryShaw/PyPCAPKit/pull/631)). diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index c616c1712..0d9dbead5 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -23,22 +23,25 @@ pcapkit.const Changed ~~~~~~~ -* **a breaking change to** ``pcapkit.const.reg.apptype``. ``AppType`` becomes a - memberless base over one registry per transport protocol -- ``TCP``, ``UDP``, - ``SCTP`` and ``DCCP`` -- because ``aenum`` refuses to subclass an enumeration that - has members. ``isinstance(TCP.http, AppType)`` still holds and +* **a breaking change to** :mod:`pcapkit.const.reg.apptype`. + :class:`~pcapkit.const.reg.apptype.apptype.AppType` becomes a memberless base over one + registry per transport protocol -- :class:`~pcapkit.const.reg.apptype.tcp.TCP`, + :class:`~pcapkit.const.reg.apptype.udp.UDP`, + :class:`~pcapkit.const.reg.apptype.sctp.SCTP` and + :class:`~pcapkit.const.reg.apptype.dccp.DCCP` -- because ``aenum`` refuses to subclass + an enumeration that has members. ``isinstance(TCP.http, AppType)`` still holds and ``AppType.get(port, proto=...)`` proxies to the registry, so nothing under - ``pcapkit.protocols`` changes. **The rows IANA assigns no port, and those it - assigns no transport protocol, stop being members**; they become list-table - docstrings on the registry that owns them. ``__members_proto__`` is now a - per-registry ``MultiDict`` keyed on port, so every service IANA registers on - port 80 survives. ``get()`` answers with the canonical service, curated from - ``/etc/services`` because IANA names no precedence, and ``get_all()`` reaches the - aliases (:pr:`754`). + :mod:`pcapkit.protocols` changes. **The rows IANA assigns no port, and those it + assigns no transport protocol, stop being members**; they become list-table docstrings + on the registry that owns them. ``__members_proto__`` is now a per-registry + :class:`.MultiDict` keyed on port, so every service IANA registers on port 80 + survives. ``get()`` answers with the canonical service, curated from ``/etc/services`` + because IANA names no precedence, and ``get_all()`` reaches the aliases (:pr:`754`). * the ``%``-formatted ``raise ValueError(...)`` calls in the generated - ``pcapkit.const.reg.apptype.apptype``, left by :issue:`759`, are f-strings + :mod:`pcapkit.const.reg.apptype.apptype`, left by :issue:`759`, are f-strings (:issue:`792`). The remainder followed: the ``_missing_`` guards in - ``pcapkit.vendor.tcp.flags``, ``.ftp.command`` and ``.http.method``, ``AppType``'s + :mod:`pcapkit.vendor.tcp.flags`, ``.ftp.command`` and ``.http.method``, + :class:`~pcapkit.const.reg.apptype.apptype.AppType`'s ``__new__``/``__repr__``/``__str__``, and the ``extend_enum`` fallbacks in ``get()`` and in the span-handling tail. The ``consider-using-f-string`` disable drops from ``tcp.flags`` and ``reg.apptype.apptype`` and stays on ``ftp.command`` and @@ -61,32 +64,33 @@ Changed now exempts only a ``'..._0x%s' % hex(value)[2:].upper().zfill(4)`` expression, and the exact ``exempt_hits == 53`` bound stays. Test-only. Closes :issue:`879` (:pr:`881`). -* **a breaking change to** ``AppType.proto``: it is the single transport of the +* **a breaking change to** :attr:`.AppType.proto`: it is the single transport of the registry the member lives in, not every transport IANA assigned the service, and ``register_apptype`` takes its transports as varargs (:issue:`806`, :pr:`815`). Aliases show in ``repr()`` and ``str()``, and can be registered per transport (:issue:`807`, :pr:`837`). -* **a breaking change to** ``TransportProtocol``: its base moves from ``aenum.IntFlag`` - to ``aenum.IntEnum``, so ``|`` between two members gives a plain ``int`` rather - than a composite member (:issue:`808`, :pr:`836`). -* **a breaking change to** ``TLSKeyLabel``: the hand-written class in - ``pcapkit.protocols.misc.pcapng`` becomes the generated - ``pcapkit.const.pcapng.tls_key_label.TLSKeyLabel`` (``PCAPNG_TLSKeyLabel`` in - ``pcapkit.const.pcapng``), crawled from the IANA registry :rfc:`9850` creates +* **a breaking change to** :class:`.TransportProtocol`: its base moves from + ``aenum.IntFlag`` to ``aenum.IntEnum``, so ``|`` between two members gives a plain + ``int`` rather than a composite member (:issue:`808`, :pr:`836`). +* **a breaking change to** :class:`~pcapkit.const.pcapng.tls_key_label.TLSKeyLabel`: the + hand-written class in :mod:`pcapkit.protocols.misc.pcapng` becomes the generated + :class:`pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` (``PCAPNG_TLSKeyLabel`` in + :mod:`pcapkit.const.pcapng`), crawled from the IANA registry :rfc:`9850` creates (:issue:`886`, :pr:`890`), and the old path is removed (:issue:`1744`, :pr:`1754`). -* **a breaking change to** ``Method.get``: the method token is case-sensitive, per +* **a breaking change to** :meth:`.Method.get`: the method token is case-sensitive, per :rfc:`9110#section-9.1`, so ``Method.get('get')`` no longer resolves to ``GET`` (:issue:`896`, :pr:`907`). It also looks a key up by value, so ``BASELINE-CONTROL`` and ``VERSION-CONTROL`` resolve (:issue:`908`, :pr:`915`). -* **a breaking change to** ``ExtensionHeader``: it is generated from IANA's IPv6 - Extension Header Types registry rather than the Protocol Numbers registry's flag, - and the two disagree on 147 (``BIT-EMU``) (:issue:`925`, :pr:`926`). -* the TCP and UDP ``AppType`` registries load on first use rather than at import - (:issue:`1538`, :pr:`1560`). -* **a breaking change to** ``LLACode``: ``Wilcard`` is spelled ``Wildcard`` and ``MH`` - is ``MN``, the names :rfc:`5568` gives codes 0 and 2, with no alias for either old - name (:issue:`1848`, :pr:`1872`). +* **a breaking change to** + :class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader`: it is generated from + IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers + registry's flag, and the two disagree on 147 (``BIT-EMU``) (:issue:`925`, :pr:`926`). +* the TCP and UDP :class:`~pcapkit.const.reg.apptype.apptype.AppType` registries load on + first use rather than at import (:issue:`1538`, :pr:`1560`). +* **a breaking change to** :class:`~pcapkit.const.mh.lla_code.LLACode`: ``Wilcard`` is + spelled ``Wildcard`` and ``MH`` is ``MN``, the names :rfc:`5568` gives codes 0 and 2, + with no alias for either old name (:issue:`1848`, :pr:`1872`). Fixed ~~~~~ @@ -98,33 +102,36 @@ Fixed crashed on its own defaults; and the FTP ``_missing_`` overrides were plain methods rather than classmethods, so every unregistered value raised ``TypeError`` instead of extending the enumeration (:issue:`492`, :pr:`503`). -* the string-keyed ``get()`` in ``pcapkit.const.ftp.command`` and - ``pcapkit.const.http.method`` tested membership with the raw key but +* the string-keyed ``get()`` in :mod:`pcapkit.const.ftp.command` and + :mod:`pcapkit.const.http.method` tested membership with the raw key but registered ``key.upper()``, so the *first* lowercase or mixed-case token raised ``TypeError: 'RETR' already in use``. Reachable from wire data: - ``pcapkit.protocols.application.ftp`` matches with ``re.I`` and passes the + :mod:`pcapkit.protocols.application.ftp` matches with ``re.I`` and passes the match verbatim, and :rfc:`959#section-5` makes FTP commands case-insensitive, so ``retr file.txt`` was a valid request the library could not parse. ``get()`` and ``_missing_`` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate ``GET`` would have lacked the real one's ``safe`` and ``idempotent`` attributes (:issue:`582`, - :issue:`583`). ``http.method`` no longer folds case: :rfc:`9110#section-9.1` makes - the method token case-sensitive, so ``Method.get`` matches it exactly - (:issue:`896`, above). + :issue:`583`). :mod:`http.method ` no longer folds case: + :rfc:`9110#section-9.1` makes the method token case-sensitive, so :meth:`.Method.get` + matches it exactly (:issue:`896`, above). * ``get()``'s documented ``default`` was ignored on the integer path across the - generated ``pcapkit.const`` tree, because ``_missing_`` cannot see the caller's + generated :mod:`pcapkit.const` tree, because ``_missing_`` cannot see the caller's ``default`` -- ``Hardware.get(99999, 0)`` raised ``ValueError`` instead of returning ``0``. The integer path now consults ``default`` before letting the error escape. ``-1``, the placeholder the generated signature already carried, still separates "no default" from "a default"; :pr:`859` replaced it with an identity sentinel once the shared base :pr:`858` (below) put ``get()`` behind one call site. The sweep covers the integer registries generated from the shared template, not only those :issue:`584` - named; ``pcapng`` ``OptionType`` and ``reg`` ``AppType`` carry a bespoke integer - fallback and are deliberately left alone, since neither drops a default by raising. - Not reachable from wire data, so a contract fix rather than a parse fix - (:issue:`584`). -* ``_missing_`` in the Mobility Header flag enumerations (``BindingACKFlag``, - ``BindingUpdateFlag``, ``HandoverACKFlag``, ``HandoverInitiateFlag``) ended in + named; ``pcapng`` :class:`~pcapkit.const.pcapng.option_type.OptionType` and ``reg`` + :class:`~pcapkit.const.reg.apptype.apptype.AppType` carry a bespoke integer fallback + and are deliberately left alone, since neither drops a default by raising. Not + reachable from wire data, so a contract fix rather than a parse fix (:issue:`584`). +* ``_missing_`` in the Mobility Header flag enumerations + (:class:`~pcapkit.const.mh.binding_ack_flag.BindingACKFlag`, + :class:`~pcapkit.const.mh.binding_update_flag.BindingUpdateFlag`, + :class:`~pcapkit.const.mh.handover_ack_flag.HandoverACKFlag`, + :class:`~pcapkit.const.mh.handover_initiate_flag.HandoverInitiateFlag`) ended in ``return cls(value)``, the constructor that had just failed, so every in-range value that is not already a member re-entered ``_missing_`` unbounded and raised ``RecursionError`` -- ``F(0)``, no flags set, and every composite such as @@ -133,44 +140,44 @@ Fixed ``pcapkit/const/tcp/flags.py`` had none and so never had the defect (it has one now, from :issue:`647` below, with the same tail). Each now ends in ``return super()._missing_(value)``, as ``pcapkit/vendor/default.py`` emits and most - const modules (:issue:`647`'s new ones included) already did, so ``F(0)`` is an - empty flag and ``BindingACKFlag(0x06)`` is ``S|D``. Minting through - ``extend_enum`` was rejected for a flag type: naming ``0x06`` ``Unassigned_0x06`` - would hide the composite and add a member per bit pattern. The range guard is - untouched (:issue:`623`). + const modules (:issue:`647`'s new ones included) already did, so ``F(0)`` is an empty + flag and ``BindingACKFlag(0x06)`` is ``S|D``. Minting through ``extend_enum`` was + rejected for a flag type: naming ``0x06`` ``Unassigned_0x06`` would hide the composite + and add a member per bit pattern. The range guard is untouched (:issue:`623`). * constant registries rejected an invalid value differently from the built-in - ``enum``, or did not reject it at all. ``Flags``, + ``enum``, or did not reject it at all. :class:`~pcapkit.const.tcp.flags.Flags`, ``ftp.command.CommandType`` and ``reg.apptype.TransportProtocol`` are ``IntFlag`` types with no ``_missing_``, so ``aenum`` composed a pseudo-member for any integer: ``Flags(-1)`` returned ``65520`` (every TCP flag set, from a value no 16-bit field - can hold) and ``Flags(-65536)`` none. ``ftp.command.Command``, ``FEATCode`` and - ``http.method.Method`` are ``StrEnum`` types whose ``_missing_`` called - ``value.upper()`` before checking the type, so an integer raised ``AttributeError``. - These now raise a bare ``ValueError``, as ``enum.IntEnum`` does and the other - modules already did. :issue:`647`'s proposal to replace those with - ``pcapkit.utilities.exceptions.EnumError`` was deliberately rejected: ``EnumError`` - is ``(BaseError, TypeError)``, not a ``ValueError``, so it would diverge from the - built-in, walk past the ``except ValueError`` in every generated ``get()`` body - (silently undoing :issue:`584`), and log CRITICAL from ``BaseError.__init__`` once - per discarded default. The one deliberate divergence is kept and pinned: mutable - registries look up, miss, and ``extend_enum`` rather than raising, so - ``Method('FROBNICATE')``, ``ProtectionAuthority(1 << 70)`` and - ``AppType.get(65000, proto=tcp)`` still register; the :issue:`775` and - :issue:`860` entries below record how each came to return an - unregistered member instead. The flag guards end in - ``return super()._missing_(value)``, so in-range composites still decompose and - :issue:`623` stays fixed; ``Flags`` accepts ``0 <= value <= 0xFFFF``. A new - ``tests/const/test_const_enum_builtin_parity.py`` asserts that every - registry raises the same exception type as a stdlib ``enum.IntEnum``; it keys on - ``aenum.Enum`` because an ``IntFlag`` is not an ``IntEnum`` subclass, which is how - the ``IntFlag`` ones escaped the existing sweeps (:issue:`647`). -* **a breaking change to** ``AppType.get``: an out-of-range port is refused + can hold) and ``Flags(-65536)`` none. + :class:`ftp.command.Command `, :class:`.FEATCode` + and :class:`http.method.Method ` are ``StrEnum`` + types whose ``_missing_`` called ``value.upper()`` before checking the type, so an + integer raised ``AttributeError``. These now raise a bare ``ValueError``, as + ``enum.IntEnum`` does and the other modules already did. :issue:`647`'s proposal to + replace those with :exc:`pcapkit.utilities.exceptions.EnumError` was deliberately + rejected: :exc:`.EnumError` is ``(BaseError, TypeError)``, not a ``ValueError``, so it + would diverge from the built-in, walk past the ``except ValueError`` in every + generated ``get()`` body (silently undoing :issue:`584`), and log CRITICAL from + ``BaseError.__init__`` once per discarded default. The one deliberate divergence is + kept and pinned: mutable registries look up, miss, and ``extend_enum`` rather than + raising, so ``Method('FROBNICATE')``, ``ProtectionAuthority(1 << 70)`` and + ``AppType.get(65000, proto=tcp)`` still register; the :issue:`775` and :issue:`860` + entries below record how each came to return an unregistered member instead. The flag + guards end in ``return super()._missing_(value)``, so in-range composites still + decompose and :issue:`623` stays fixed; :class:`~pcapkit.const.tcp.flags.Flags` + accepts ``0 <= value <= 0xFFFF``. A new + ``tests/const/test_const_enum_builtin_parity.py`` asserts that every registry raises + the same exception type as a stdlib ``enum.IntEnum``; it keys on ``aenum.Enum`` + because an ``IntFlag`` is not an ``IntEnum`` subclass, which is how the ``IntFlag`` + ones escaped the existing sweeps (:issue:`647`). +* **a breaking change to** :meth:`.AppType.get`: an out-of-range port is refused rather than minted. Its ``except ValueError`` caught the range rejection from ``_missing_`` and minted regardless, so ``AppType.get(-1, proto='tcp')`` returned an unreachable ``PORT_-1_tcp`` while ``TCP(-1)`` raised. It now tests ``_missing_`` for ``None`` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until :pr:`874` (below) made it - an unregistered member. ``Transport._make_port`` + an unregistered member. :meth:`.Transport._make_port` **passes an unvalidated** ``int``, **so** ``TCP.make(srcport=99999)`` **now raises where it minted junk**; in 1.4.1, ``TCP(srcport=N)`` raised ``AttributeError`` for any bare ``int``. Separately, the crawler rendered one ``_missing_`` branch per @@ -180,17 +187,17 @@ Fixed ``6666/udp`` answered ``ircu`` rather than ``reserved``, and ``proto='sctp'`` minted ``x11`` where IANA assigns nothing. A branch naming a transport now tests ``cls.__transport__``; the branches naming none are unchanged (:pr:`764`). -* **a breaking change to** ``AppType._dispatch``: it resolved a ``proto`` naming +* **a breaking change to** :meth:`.AppType._dispatch`: it resolved a ``proto`` naming several transports by its lowest set bit, and ``tcp`` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since ``AppType.__eq__`` compares on ``port`` alone. - ``TransportProtocol`` was still an ``aenum.IntFlag`` and a member's ``proto`` + :class:`.TransportProtocol` was still an ``aenum.IntFlag`` and a member's ``proto`` carried the whole set IANA assigned the service, so ``tcp | udp`` was an ordinary - value to pass back in; the ``AppType.proto`` and ``TransportProtocol`` entries above - record how :issue:`806` and :issue:`808` later retired both. Over + value to pass back in; the :attr:`.AppType.proto` and :class:`.TransportProtocol` + entries above record how :issue:`806` and :issue:`808` later retired both. Over every multi-transport member this misrouted port 888 (``accessbuilder`` resolved to TCP's ``cddbp``) and port 999 (``puprouter`` resolved to TCP's ``garcon``). It now - raises ``ProtocolError``, a ``ValueError`` subclass, so the + raises :exc:`.ProtocolError`, a ``ValueError`` subclass, so the documented contract holds. No library caller passes a composite (:issue:`759`). * ``get()``'s string-key miss and every generated registry's ``_missing_`` @@ -202,17 +209,17 @@ Fixed ``_missing_`` fix changes behaviour only for those with a bounded-range ``extend_enum`` branch. ``pcapkit/toolkit/scapy.py`` and ``pcapkit/toolkit/pyshark.py`` fed ``get()`` a string key with no default and now - raise ``MissingKeyError`` instead of receiving a placeholder; the reader sites in + raise :exc:`.MissingKeyError` instead of receiving a placeholder; the reader sites in ``hopopt.py``/``ipv6_opts.py`` convert - the ``KeyError`` into ``ProtocolError``. Riding along: + the ``KeyError`` into :exc:`.ProtocolError`. Riding along: ``pcapkit/toolkit/pyshark.py`` gains a ``FILTER_NAME_TO_LINKTYPE`` map (``eth`` to ``ETHERNET``, ``tr`` to ``IEEE802_5``), consulted ahead of ``LinkType.get`` since pyshark reports a filter name, not a DLT name. The other minting call sites in ``_missing_``, ``get`` and the hand-written ``mh.py`` and ``ngap.py`` were deferred; the ``register``/``register_alias`` sites are not deferred but the by-design caller-named path (:pr:`838`). -* **a breaking change to** ``pcapkit.const.ipx.socket.Socket``'s ``_missing_``: archived - range branches that were strict subsets of a range tested earlier -- +* **a breaking change to** :class:`pcapkit.const.ipx.socket.Socket`'s ``_missing_``: + archived range branches that were strict subsets of a range tested earlier -- ``(0x0020, 0x003F)`` after ``(0x0001, 0x0BB8)``, and ``(0x4000, 0x4FFF)`` and ``(0x8000, 0xFFFF)`` after ``(0x0BB9, 0xFFFF)`` -- so they never fired. The generator ``pcapkit/vendor/ipx/socket.py`` now lists each subset range ahead of the @@ -240,88 +247,95 @@ Fixed ``Reserved [...]``, ``Reserved for Private/Experimental Use``, ``Unspecified in the IANA registry``, ``Deprecated``), which the ruling treats as a notation rather than an assignment. Some are mixed: - ``EtherType``'s ``DEC Unassigned`` and "Old Xerox Experimental values. Invalid as an - Ethertype since 1983." branches convert, while vendor-company names - (Xyplex, Motorola, Walker Richer & Quinn and others) keep minting, since a - proprietary protocol has no public name beyond the company's; + :class:`~pcapkit.const.reg.ethertype.EtherType`'s ``DEC Unassigned`` and "Old Xerox + Experimental values. Invalid as an Ethertype since 1983." branches convert, while + vendor-company names (Xyplex, Motorola, Walker Richer & Quinn and others) keep + minting, since a proprietary protocol has no public name beyond the company's; ``IEEE802_3_Length_Field`` and ``Berkeley_Trailer_encap_IP`` also keep minting but - are not company names. ``Socket`` (ipx)'s allocation-policy names (``Experimental``, - ``Dynamically Assigned Socket Numbers``, ``Statically Assigned Socket Numbers``, - ``Dynamically Assigned``) convert, while ``Registered by Xerox`` keeps minting, a - real ownership fact. Left alone are ``CGAType``, whose ``Tag_`` mint is not an + are not company names. :class:`~pcapkit.const.ipx.socket.Socket` (ipx)'s + allocation-policy names (``Experimental``, ``Dynamically Assigned Socket Numbers``, + ``Statically Assigned Socket Numbers``, ``Dynamically Assigned``) convert, while + ``Registered by Xerox`` keeps minting, a real ownership fact. Left alone are + :class:`~pcapkit.const.mh.cga_type.CGAType`, whose ``Tag_`` mint is not an IANA-style range, and the files holding the classes :issue:`860` moves onto - ``EnumRegistry`` (the ``AppType``, ``StatusCode``, ``ftp.return_code``, + :class:`.EnumRegistry` (the ``AppType``, ``StatusCode``, ``ftp.return_code``, ``ftp.command``, ``Method`` and ``OptionType`` modules; :pr:`869` and :pr:`874`, below). Each conversion changes both the vendor crawler and the generated file, per :pr:`838`'s and :pr:`858`'s precedent. The worked examples in ``docs/source/contributing/conventions/mint-criterion.rst`` keep ``Registered by Xerox`` under Mint (:pr:`861`). -* **a breaking change to** ``EtherType``'s ``_missing_`` ordering: values - ``0x0101``-``0x01FF`` now resolve to their Old Xerox Experimental name instead of an - ``IEEE802_3_Length_Field_0x0101``-style one. The generated method tested - ``0x0000 <= value <= 0x05DC`` (IEEE802.3 Length Field) first, and the Old Xerox - range sits wholly inside it, so that branch was unreachable. Unlike :issue:`841`'s - IPX fix, these bounds come from the live IANA CSV, so the fix is general: - ``pcapkit/vendor/reg/ethertype.py``'s ``process()`` collects range rows and a new - ``EtherType._insert_range`` places each ahead of the first already-placed range that - fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only - containment among the CSV's ranges. Closes :issue:`862` (:pr:`865`). -* **a breaking change to** ``Method``: ``__new__`` called ``str.__new__(cls)`` with no - argument, so every registered member's ``str`` payload was empty regardless of - value -- ``str(Method.GET) == ''`` and ``Method.GET == 'GET'`` was ``False``; - ``.value`` was always correct. :pr:`869` (above) had fixed the *unregistered* path - only, leaving ``str(Method('frob')) == 'frob'`` beside ``str(Method.GET) == ''``. - Fixed to ``str.__new__(cls, value)``, mirroring ``Command.__new__``, in the - generator (``pcapkit/vendor/http/method.py``) and the generated module. No sibling - hand-rolled ``__new__`` has the defect (``OptionType``/``AppType`` deliberately - store a display string). Consequences, all corrections: ``Method.GET == 'GET'`` is - ``True``; ``bool(Method.GET)`` flips ``False`` to ``True``; +* **a breaking change to** :class:`~pcapkit.const.reg.ethertype.EtherType`'s + ``_missing_`` ordering: values ``0x0101``-``0x01FF`` now resolve to their Old Xerox + Experimental name instead of an ``IEEE802_3_Length_Field_0x0101``-style one. The + generated method tested ``0x0000 <= value <= 0x05DC`` (IEEE802.3 Length Field) first, + and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike + :issue:`841`'s IPX fix, these bounds come from the live IANA CSV, so the fix is + general: ``pcapkit/vendor/reg/ethertype.py``'s ``process()`` collects range rows and a + new ``EtherType._insert_range`` places each ahead of the first already-placed range + that fully contains it, leaving non-overlapping pairs in CSV order; this pair is the + only containment among the CSV's ranges. Closes :issue:`862` (:pr:`865`). +* **a breaking change to** :class:`~pcapkit.const.http.method.Method`: ``__new__`` + called ``str.__new__(cls)`` with no argument, so every registered member's ``str`` + payload was empty regardless of value -- ``str(Method.GET) == ''`` and + ``Method.GET == 'GET'`` was ``False``; ``.value`` was always correct. :pr:`869` + (above) had fixed the *unregistered* path only, leaving + ``str(Method('frob')) == 'frob'`` beside ``str(Method.GET) == ''``. Fixed to + ``str.__new__(cls, value)``, mirroring ``Command.__new__``, in the generator + (``pcapkit/vendor/http/method.py``) and the generated module. No sibling hand-rolled + ``__new__`` has the defect + (:class:`~pcapkit.const.pcapng.option_type.OptionType`/:class:`~pcapkit.const.reg.apptype.apptype.AppType` + deliberately store a display string). Consequences, all corrections: + ``Method.GET == 'GET'`` is ``True``; ``bool(Method.GET)`` flips ``False`` to ``True``; ``json.dumps(Method.GET)`` emits ``"GET"`` not ``""``; ``sorted()`` is lexicographic rather than input-order-stable; and ``{m: ... for m in Method}`` gains an entry per member where it had one, because every member previously hashed as ``''`` and compared equal. Nothing under ``pcapkit/protocols`` relied on the old behaviour. Closes :issue:`870` (:pr:`871`). -* **a breaking change to** ``TransportProtocol`` and ``AppType``: step 2 - of :issue:`860`, the ``AppType`` half (the other bespoke registries were :pr:`869`, - above). ``AppType`` and its transport subclasses move onto ``EnumRegistry`` and - both mint sites -- ``_missing_``'s range branches and ``get()``'s own mint - (``PORT_{port}_{transport}`` -> ``'unknown'``) -- now build through a new - ``_unregistered_member`` override that reconstructs ``svc``/``port``/``proto``, per - the owner's ruling that only IANA-registered values are legitimate and that - ``get()`` lacks the information to create new ones. Every branch converts, - including those naming a real service assigned to a whole port span (``x11``, - ``active-net``, ``satvid-datalnk``, ``vrml-multi-use``, ``ircu``, - ``swx``, ``flex-lm``); unlike ``FEATCode``'s fix (:pr:`869`, above) there is no - import-time self-mutation to cure by declaring members statically, so these stay - deliberately unregistered lookups. ``AppType`` gains a working ``register()``; the - base's would have built a member with ``svc=''``. **The breaking part**: - ``TransportProtocol`` moves from power-of-two values - (``tcp=1, udp=2, sctp=4, dccp=8``) to sequential ``auto()`` (``sctp=3, dccp=4``), - per the owner's follow-up ruling once ``|``-joined composites stopped being parsed. - ``AppType.get`` treats whatever ``int`` it is given as a whole, so the renumbering - changes what specific integers *mean*: on ``AppType.get(80, proto=...)``, a bare - ``4`` resolved to ``SCTP.http`` on ``main`` and now **silently** resolves to - ``DCCP.unknown``, because a real member sits at ``4`` under both numberings; a bare +* **a breaking change to** :class:`.TransportProtocol` and + :class:`~pcapkit.const.reg.apptype.apptype.AppType`: step 2 of :issue:`860`, the + :class:`~pcapkit.const.reg.apptype.apptype.AppType` half (the other bespoke registries + were :pr:`869`, above). :class:`~pcapkit.const.reg.apptype.apptype.AppType` and its + transport subclasses move onto :class:`.EnumRegistry` and both mint sites -- + ``_missing_``'s range branches and ``get()``'s own mint (``PORT_{port}_{transport}`` + -> ``'unknown'``) -- now build through a new ``_unregistered_member`` override that + reconstructs ``svc``/``port``/``proto``, per the owner's ruling that only + IANA-registered values are legitimate and that ``get()`` lacks the information to + create new ones. Every branch converts, including those naming a real service assigned + to a whole port span (``x11``, ``active-net``, ``satvid-datalnk``, ``vrml-multi-use``, + ``ircu``, ``swx``, ``flex-lm``); unlike :class:`.FEATCode`'s fix (:pr:`869`, above) + there is no import-time self-mutation to cure by declaring members statically, so + these stay deliberately unregistered lookups. + :class:`~pcapkit.const.reg.apptype.apptype.AppType` gains a working ``register()``; + the base's would have built a member with ``svc=''``. **The breaking part**: + :class:`.TransportProtocol` moves from power-of-two values + (``tcp=1, udp=2, sctp=4, dccp=8``) to sequential ``auto()`` (``sctp=3, dccp=4``), per + the owner's follow-up ruling once ``|``-joined composites stopped being parsed. + :meth:`.AppType.get` treats whatever ``int`` it is given as a whole, so the + renumbering changes what specific integers *mean*: on ``AppType.get(80, proto=...)``, + a bare ``4`` resolved to :attr:`.SCTP.http` on ``main`` and now **silently** resolves + to ``DCCP.unknown``, because a real member sits at ``4`` under both numberings; a bare ``8`` used to mint ``DCCP.PORT_80_dccp`` and now raises ``ValueError``. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path (:pr:`874`). -* closes the remaining scope of :issue:`775`: ``EtherType`` and - ``Socket``, the last registries whose ``_missing_`` minted a permanent member, - now return a non-registering ``cls._unregistered_member(value, name)``. The issue's - AST census of ``extend_enum`` in ``_missing_``/``get`` is down to the - single ``CGAType`` carve-out (its mint is not an IANA-style range). - ``EtherType(0x0888)`` used to add a member to ``__members__`` - and return the identical object on a repeat; it now returns - ```` with ``__members__`` unchanged, absent from - ``_value2member_map_``, equal but not identical across lookups -- likewise - ``Socket(0x0010)``. The regenerated ``EtherType`` was not checked against the live - IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name +* closes the remaining scope of :issue:`775`: + :class:`~pcapkit.const.reg.ethertype.EtherType` and + :class:`~pcapkit.const.ipx.socket.Socket`, the last registries whose ``_missing_`` + minted a permanent member, now return a non-registering + ``cls._unregistered_member(value, name)``. The issue's AST census of ``extend_enum`` + in ``_missing_``/``get`` is down to the single + :class:`~pcapkit.const.mh.cga_type.CGAType` carve-out (its mint is not an IANA-style + range). ``EtherType(0x0888)`` used to add a member to ``__members__`` and return the + identical object on a repeat; it now returns ```` with + ``__members__`` unchanged, absent from ``_value2member_map_``, equal but not identical + across lookups -- likewise ``Socket(0x0010)``. The regenerated + :class:`~pcapkit.const.reg.ethertype.EtherType` was not checked against the live IANA + CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape ``test_const_enum_no_mint.py``'s - ``is_manufactured()`` sweep forbids; rather than weaken the predicate, a - narrow exemption covers exactly these files, since the collision the check guards - against cannot occur once ``_unregistered_member`` never registers (:pr:`878`). -* ``OptionType.register()`` and ``Command.register()`` built members that ``get()`` - could not find again, and ``OptionType.register_alias()`` raised + ``is_manufactured()`` sweep forbids; rather than weaken the predicate, a narrow + exemption covers exactly these files, since the collision the check guards against + cannot occur once ``_unregistered_member`` never registers (:pr:`878`). +* :meth:`.OptionType.register()` and :meth:`.Command.register()` built members that + ``get()`` could not find again, and :meth:`.OptionType.register_alias()` raised ``not a registered OptionType`` for every argument. Both registries now round-trip through ``get()``, in the generated files and the vendor templates (:pr:`1311`, :issue:`1330`, :pr:`1369`). @@ -330,26 +344,27 @@ Fixed :pr:`1353`). * every three-digit FTP reply and HTTP status code resolves, including those no registry row names (:issue:`1443`, :pr:`1444`). -* the ``AppType`` registry's dict union is defined only on interpreters whose - ``dict`` has one (:issue:`1694`, :pr:`1703`). -* NGAP's ``ProcedureCode`` and ``ProtocolIE`` are generated into - ``pcapkit.const.ngap`` from ``pycrate``'s compiled NGAP specification, with no - network fetch. An unseen value gets a non-registering member, where every unknown - key used to mint a member at the shared value ``-1`` and so aliased every other +* the :class:`~pcapkit.const.reg.apptype.apptype.AppType` registry's dict union is + defined only on interpreters whose ``dict`` has one (:issue:`1694`, :pr:`1703`). +* NGAP's :class:`~pcapkit.const.ngap.procedure_code.ProcedureCode` and + :class:`~pcapkit.const.ngap.protocol_ie.ProtocolIE` are generated into + :mod:`pcapkit.const.ngap` from ``pycrate``'s compiled NGAP specification, with no + network fetch. An unseen value gets a non-registering member, where every unknown key + used to mint a member at the shared value ``-1`` and so aliased every other (:issue:`880`, :pr:`899`). -* ``FEATCode.get`` matches case-insensitively, as :rfc:`5797#section-2` defines FEAT - codes; the registry-wide case-sensitivity audit found no other registry that +* :meth:`.FEATCode.get` matches case-insensitively, as :rfc:`5797#section-2` defines + FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold (:issue:`903`, :pr:`913`). * ``register`` refuses what ``get`` refuses. ``TCP.register(70000, 'x')`` and ``Hardware.register(70000, 'x')`` minted a member that ``get`` then returned, because no registry overrode the ``_validate_value`` hook ``register`` calls. Each - generated registry, ``AppType`` included, now overrides it with the check its - ``_missing_`` makes, raising ``EnumValueError``. ``get(value, default)`` on a value - the hook refuses still returns ``default`` and logs nothing (:issue:`1792`, - :pr:`1818`, :issue:`1816`, :pr:`1823`; the quiet miss is :issue:`1845`, - :pr:`1873`, under ``pcapkit.corekit``). -* ``Command.get`` refuses a key that is not a ``str`` with ``EnumValueError``, as - ``Method.get`` does. ``get(b'\x01')`` minted a member keyed by ``bytes``, and + generated registry, :class:`~pcapkit.const.reg.apptype.apptype.AppType` included, now + overrides it with the check its ``_missing_`` makes, raising :exc:`.EnumValueError`. + ``get(value, default)`` on a value the hook refuses still returns ``default`` and logs + nothing (:issue:`1792`, :pr:`1818`, :issue:`1816`, :pr:`1823`; the quiet miss is + :issue:`1845`, :pr:`1873`, under :mod:`pcapkit.corekit`). +* :meth:`.Command.get` refuses a key that is not a ``str`` with :exc:`.EnumValueError`, + as :meth:`.Method.get` does. ``get(b'\x01')`` minted a member keyed by ``bytes``, and ``get(-1)`` raised ``AttributeError`` whatever its ``default`` (:issue:`1824`, :pr:`1826`). @@ -367,32 +382,33 @@ Added plus guard rails for a full read and for a read against an empty buffer, the :issue:`431` behaviour the option and list loops depend on (:issue:`604`). -* ``EnumLookup``, the base carrying ``get`` and ``get_all`` that ``EnumRegistry`` and - the non-registry enumerations share (:issue:`877`, :pr:`906`, :pr:`921`, - :pr:`932`). Its ``get`` raises pcapkit exceptions in the shape of the stdlib - ``Enum``: a name miss raises ``EnumKeyError``, a ``KeyError`` (:issue:`923`, - :pr:`927`). -* ``MultiInfo`` and ``OrderedMultiInfo``, immutable ``Info`` subclasses that are also - a ``MultiDict`` and an ``OrderedMultiDict``, so a repeated key keeps every value. - The data models' option lists use them (:issue:`1484`, :pr:`1523`). -* ``pcapkit.corekit.fields.byteorder``, holding ``packet_byteorder`` and +* :class:`.EnumLookup`, the base carrying ``get`` and ``get_all`` that + :class:`.EnumRegistry` and the non-registry enumerations share (:issue:`877`, + :pr:`906`, :pr:`921`, :pr:`932`). Its ``get`` raises pcapkit exceptions in the shape + of the stdlib ``Enum``: a name miss raises :exc:`.EnumKeyError`, a ``KeyError`` + (:issue:`923`, :pr:`927`). +* :class:`.MultiInfo` and :class:`.OrderedMultiInfo`, immutable :class:`.Info` + subclasses that are also a :class:`.MultiDict` and an :class:`.OrderedMultiDict`, so a + repeated key keeps every value. The data models' option lists use them (:issue:`1484`, + :pr:`1523`). +* :mod:`pcapkit.corekit.fields.byteorder`, holding ``packet_byteorder`` and ``byteorder_callback``, the byte-order helpers the PCAP and PCAP-NG schemas share (:pr:`1528`, :issue:`1711`, :pr:`1715`). Changed ~~~~~~~ -* ``Probe``, ``CipherSuite`` and ``IntegritySuite`` are ``Info`` subclasses rather - than ``typing.NamedTuple``. They are Mappings now, so ``len()`` and iteration yield - field names rather than values. ``TCPSegment`` in ``pcapkit.toolkit.pcap`` stays a - ``typing.NamedTuple``, since it is built for every TCP frame and an ``Info`` costs - more to construct. -* ``ModuleDescriptor.klass`` reads an already-imported module out of ``sys.modules`` - rather than re-entering ``importlib.import_module``, since next layer dispatch - resolves a descriptor there on a per-frame path. Neither a registry *hit* holding a - ``ModuleDescriptor`` nor a *miss* writes the resolved class back: a memoised hit went - stale on ``importlib.reload`` (:issue:`1559`), and recording a miss in a class-level - ``collections.defaultdict`` is the defect +* :class:`.Probe`, :class:`.CipherSuite` and :class:`.IntegritySuite` are :class:`.Info` + subclasses rather than ``typing.NamedTuple``. They are Mappings now, so ``len()`` and + iteration yield field names rather than values. :class:`.TCPSegment` in + :mod:`pcapkit.toolkit.pcap` stays a ``typing.NamedTuple``, since it is built for every + TCP frame and an :class:`.Info` costs more to construct. +* :attr:`.ModuleDescriptor.klass` reads an already-imported module out of + ``sys.modules`` rather than re-entering ``importlib.import_module``, since next layer + dispatch resolves a descriptor there on a per-frame path. Neither a registry *hit* + holding a :class:`.ModuleDescriptor` nor a *miss* writes the resolved class back: a + memoised hit went stale on ``importlib.reload`` (:issue:`1559`), and recording a miss + in a class-level ``collections.defaultdict`` is the defect :issue:`421`/:pr:`426` fixed at this layer, :issue:`425`/:pr:`428` for the option, chunk and block registries, and :issue:`555`/:pr:`560` at the schema layer. A resolution now costs ~117 ns rather than ~436 ns, which is **not measurable in** @@ -401,7 +417,7 @@ Changed class after an ``importlib.reload`` forever and fail ``isinstance`` against the live one. Proposed by ``@Ts-Boom`` in :pr:`563`; the implementation differs because that one added a second, never-invalidated cache of resolved classes (:issue:`574`). -* ``SeekableReader.truncate()`` raises instead of returning a size, and the +* :meth:`.SeekableReader.truncate()` raises instead of returning a size, and the misspelled ``writeable()`` is now ``writable()``. Both break external callers; nothing inside the package calls either. ``io.IOBase`` documents one gate over two methods -- "If False, write() and truncate() will raise @@ -414,70 +430,70 @@ Changed buffered read paths must refuse, for which :issue:`643` and :issue:`644` landed tests. ``writeable()`` was never an override of anything, so ``io``, ``shutil`` and third parties never saw it, and the value reported is unchanged (:issue:`645`). -* **a breaking change to** ``@final``, now enforced at runtime on ``Info`` and - ``Schema``. ``info_final`` and ``schema_final`` both end ``return final(cls)``, so - every finalised class carried ``typing.final``'s ``__final__`` marker and nothing - read it. ``@final`` **alone on an** ``Info`` **or** ``Schema`` **subclass -- without** - ``@info_final`` **or** ``@schema_final`` **-- now raises** ``InfoError`` **or** - ``SchemaError`` **at first construction**, the class being marked final but never finalised and so - carrying no generated ``__init__``. ``__init_subclass__`` cannot catch that, since - ``final`` applies after class creation, so the check sits in the existing one-shot - ``FinalisedState.NONE`` branch and a finalised class pays nothing. **Deriving from a - finalised class raises too**, from new ``Info.__init_subclass__`` and - ``Schema.__init_subclass__`` hooks (before, only ``EnumSchema`` had one). - ``SchemaError`` **is a** ``ValueError``, so code catching ``TypeError`` around a - subclass declaration will not see it. ``@info_final @final`` and - ``@final @info_final`` both finalise silently; the re-entry check keys on - ``__finalised__`` rather than ``__final__``, because only the former records *the - decorator* having run. ``@info_final`` twice warns and returns the class unchanged. - ``pcapkit.utilities.compat`` takes ``final`` from ``typing_extensions`` below 3.11 - rather than 3.8, because ``typing.final`` only records ``__final__`` from 3.11. - The guard cannot fire on the library's own classes (:issue:`778`). -* **a breaking change to** ``Info.to_dict()``: it returns an ``OrderedMultiDict`` at - every level rather than a ``dict``, so a key that repeats, such as a second IPv6 - extension header of one type, survives a ``from_dict(to_dict())`` rebuild. - ``type(d) is dict`` no longer holds (:issue:`1453`, :pr:`1466`, :issue:`1484`, - :pr:`1523`). +* **a breaking change to** ``@final``, now enforced at runtime on :class:`.Info` and + :class:`.Schema`. ``info_final`` and ``schema_final`` both end ``return final(cls)``, + so every finalised class carried ``typing.final``'s ``__final__`` marker and nothing + read it. ``@final`` **alone on an** :class:`.Info` **or** :class:`.Schema` **subclass + -- without** ``@info_final`` **or** ``@schema_final`` **-- now raises** + :exc:`.InfoError` **or** :exc:`.SchemaError` **at first construction**, the class + being marked final but never finalised and so carrying no generated ``__init__``. + ``__init_subclass__`` cannot catch that, since ``final`` applies after class creation, + so the check sits in the existing one-shot :attr:`.FinalisedState.NONE` branch and a + finalised class pays nothing. **Deriving from a finalised class raises too**, from new + :meth:`.Info.__init_subclass__` and :meth:`.Schema.__init_subclass__` hooks (before, + only :class:`.EnumSchema` had one). :exc:`.SchemaError` **is a** ``ValueError``, so + code catching ``TypeError`` around a subclass declaration will not see it. + ``@info_final @final`` and ``@final @info_final`` both finalise silently; the re-entry + check keys on ``__finalised__`` rather than ``__final__``, because only the former + records *the decorator* having run. ``@info_final`` twice warns and returns the class + unchanged. :mod:`pcapkit.utilities.compat` takes ``final`` from ``typing_extensions`` + below 3.11 rather than 3.8, because ``typing.final`` only records ``__final__`` from + 3.11. The guard cannot fire on the library's own classes (:issue:`778`). +* **a breaking change to** :meth:`.Info.to_dict()`: it returns an + :class:`.OrderedMultiDict` at every level rather than a ``dict``, so a key that + repeats, such as a second IPv6 extension header of one type, survives a + ``from_dict(to_dict())`` rebuild. ``type(d) is dict`` no longer holds (:issue:`1453`, + :pr:`1466`, :issue:`1484`, :pr:`1523`). * **a breaking change to** ``pcapkit.utilities.decorators``: it moves to - ``pcapkit.corekit.decorators``, with no re-export. ``seekset``, ``beholder`` and - ``prepare`` are no longer importable from ``pcapkit.utilities``, and the module - logs as ``pcapkit.corekit.decorators`` (:issue:`1735`, :pr:`1737`). -* **a breaking change to** the sentinels: all live in ``pcapkit.corekit.sentinels``, + :mod:`pcapkit.corekit.decorators`, with no re-export. ``seekset``, ``beholder`` and + ``prepare`` are no longer importable from :mod:`pcapkit.utilities`, and the module + logs as :mod:`pcapkit.corekit.decorators` (:issue:`1735`, :pr:`1737`). +* **a breaking change to** the sentinels: all live in :mod:`pcapkit.corekit.sentinels`, which exports the objects ``NULL``, ``NO_VALUE`` and ``NO_DEFAULT`` and their - common base ``Sentinel``, but not their types, and ``NoValue`` is spelled + common base :class:`.Sentinel`, but not their types, and ``NoValue`` is spelled ``NO_VALUE`` (:pr:`916`, :issue:`911`, :pr:`922`, :issue:`937`, :pr:`939`). No moved name keeps a re-export, so - ``NoValueType`` has only that home, not its 1.4.x one in - ``pcapkit.corekit.fields.field`` (:issue:`1744`, :pr:`1754`). + :class:`.NoValueType` has only that home, not its 1.4.x one in + :mod:`pcapkit.corekit.fields.field` (:issue:`1744`, :pr:`1754`). Fixed ~~~~~ * stdlib exceptions leaking out where the library's own were promised: a malformed IP - field value raised a bare ``ValueError`` instead of ``FieldValueError`` + field value raised a bare ``ValueError`` instead of :exc:`.FieldValueError` (:issue:`465`); a ``bool`` address was silently packed as ``0.0.0.1`` or ``0.0.0.0``, ``bool`` being an ``int`` subclass (:issue:`491`, :pr:`500`); and ``@prepare`` discarded extra arguments silently and treated a *declared* zero length as end of stream, which is now distinguished from a genuinely exhausted one and - raises ``StreamEOFError`` (:issue:`454`, :issue:`458`). -* ``FieldBase.unpack`` zero-padded up to a field's declared ``length`` with + raises :exc:`.StreamEOFError` (:issue:`454`, :issue:`458`). +* :meth:`.FieldBase.unpack` zero-padded up to a field's declared ``length`` with ``rjust()`` however little data ``buffer`` held; a ~40-octet PCAP-NG Decryption Secrets Block with a bogus inner length forced a multi-gigabyte allocation, since ``length`` is often wire-derived. A declared length past 262144 octets -- libpcap's ``MAXIMUM_SNAPLEN`` and this package's default ``snaplen`` -- that - the buffer cannot back now raises ``FieldValueError``; the option and list + the buffer cannot back now raises :exc:`.FieldValueError`; the option and list loops' tolerance for a short read past a truncated area (:issue:`431`) is far under that ceiling and untouched (:issue:`554`). * that ceiling bounds one field, and a packet holds many, so the *sum* of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring ``secrets_length`` 262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing. - ``FieldBase.unpack`` now keeps a per-context ledger of octets supplied against octets - synthesised, and raises ``FieldValueError`` once the total of shortfalls *past 65,536 - octets* passes ``262144 + 16 * supplied``. The same 200 blocks retain 0.2 MiB (54.6x). - A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded - unconditionally and charged to nothing, so no snapshot-truncated capture is subject - to the budget; without that band the same legitimate frame parsed differently + :meth:`.FieldBase.unpack` now keeps a per-context ledger of octets supplied against + octets synthesised, and raises :exc:`.FieldValueError` once the total of shortfalls + *past 65,536 octets* passes ``262144 + 16 * supplied``. The same 200 blocks retain 0.2 + MiB (54.6x). A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, + is padded unconditionally and charged to nothing, so no snapshot-truncated capture is + subject to the budget; without that band the same legitimate frame parsed differently depending on what had been parsed before. **What this does not close**: a length declared by a 16-bit field can still be repeated without limit, and at this layer a crafted file is indistinguishable from a legitimate snapshot-truncated capture of @@ -485,17 +501,17 @@ Fixed ``incl_len``/``orig_len``. Nothing resets the ledger, so it bounds everything a context has parsed rather than one file (:issue:`573`). * which exception a malformed TCP SACK option raised depended on unrelated - process state: a clean interpreter raised ``ProtocolError`` as documented, but - a process that had popped ``pcapkit.corekit.fields.misc`` from ``sys.modules`` + process state: a clean interpreter raised :exc:`.ProtocolError` as documented, but + a process that had popped :mod:`pcapkit.corekit.fields.misc` from ``sys.modules`` (as the :issue:`439` ABC-cache regression tests do in every ``setUp``/``tearDown``) - raised ``FieldValueError`` from a different layer, before the documented check - (:issue:`525`). ``ListField.unpack`` resolved ``SchemaField`` through a + raised :exc:`.FieldValueError` from a different layer, before the documented check + (:issue:`525`). :meth:`.ListField.unpack` resolved :class:`.SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so ``isinstance`` misclassified the field and billed each item by its declared length instead of what it consumed. **Any - caller relying on the previously-observed** ``FieldValueError`` **now gets** - ``ProtocolError``, deterministically, matching the docstring. -* a ``NumberField`` whose ``length`` was a callable could not pack or parse at any + caller relying on the previously-observed** :exc:`.FieldValueError` **now gets** + :exc:`.ProtocolError`, deterministically, matching the docstring. +* a :class:`.NumberField` whose ``length`` was a callable could not pack or parse at any width ``struct`` has a native integer code for. ``length`` is ``-1`` until the callable resolves, ``-1`` has no native code, and the template builder raised ``_need_process`` for it and never cleared it -- a latch. Resolving the real width @@ -506,24 +522,24 @@ Fixed The flag is now recomputed from the width in force, which also keeps a callable resolving to a width with no native code (3 octets) byte-packed. **Every native width (1, 2, 4 and 8 octets) was affected, not only the 8 reported**, on - ``NumberField`` and ``EnumField`` alike. This made every extended 8-octet MPTCP - DSS form unbuildable, those widths being chosen at runtime from the DSS flags; + :class:`.NumberField` and :class:`.EnumField` alike. This made every extended 8-octet + MPTCP DSS form unbuildable, those widths being chosen at runtime from the DSS flags; :pr:`585` worked around it in the TCP schema alone (:issue:`591`). -* ``NumberField.pre_process`` sized a value with floor division dressed as a ceiling. - When a field is packed while ``length`` is still the ``-1`` placeholder, the width - came from ``math.ceil(value.bit_length() // 8)``; the ``//`` had already floored, so - ``math.ceil`` did nothing and the width was one octet short: ``256`` at one octet, - ``65536`` at two, ``16777216`` at three. **The reach is wider than "just past a - boundary"**: every bit length that is not a multiple of eight is wrong, so ``1`` - (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. - It is now the ceiling it was meant to be. The repair is reached only by packing a +* :meth:`.NumberField.pre_process` sized a value with floor division dressed as a + ceiling. When a field is packed while ``length`` is still the ``-1`` placeholder, the + width came from ``math.ceil(value.bit_length() // 8)``; the ``//`` had already + floored, so ``math.ceil`` did nothing and the width was one octet short: ``256`` at + one octet, ``65536`` at two, ``16777216`` at three. **The reach is wider than "just + past a boundary"**: every bit length that is not a multiple of eight is wrong, so + ``1`` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed + too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why :issue:`591`'s suite, whose values all had byte-aligned bit lengths, missed it; :issue:`591`'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack ``128``. The same typo in the ILNP nonce option builders is a separate change (:issue:`599`). -* ``FieldBase.unpack`` padded a short read on the wrong side. When a buffer falls +* :meth:`.FieldBase.unpack` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse (:issue:`431`) -- the shortfall was zero-filled with ``rjust()``, placing the zeros at the *front*, as if the octets never read were the leading ones; @@ -531,8 +547,8 @@ Fixed byte orders: one octet of a four-octet little-endian ``120`` read as ``2013265920``, and three octets of a four-octet big-endian ``0x01020304`` as ``0x10203``. A full read is untouched (:issue:`604`). -* ``SeekableReader.truncate`` put its padding where the reader's bookkeeping says the - content is, and ``read`` then returned one octet fewer than the buffer held. Same +* :meth:`.SeekableReader.truncate` put its padding where the reader's bookkeeping says + the content is, and ``read`` then returned one octet fewer than the buffer held. Same family as :issue:`604`, but separate defects. The buffer keeps content at ``[0:_buffer_cur]`` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; ``temp[-size:]`` sliced the buffer @@ -570,7 +586,7 @@ Fixed capture says so through ``captured_len`` and leaves its options whole, so it never trips this -- which the :pr:`571` ``len(buffer) < length`` rejection could not distinguish, and why it was declined. A new ``bounded_option`` clamps the payload to - the octets the area has left and warns with ``SchemaWarning``; it is applied to + the octets the area has left and warns with :exc:`.SchemaWarning`; it is applied to every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A companion ``bounded_area`` clamps a packet block's option @@ -581,45 +597,46 @@ Fixed areas keep the framing assumption; the general fix for a declared length reaching a read is :issue:`678`. Clamping rather than refusing keeps the :issue:`431` accommodation, since a PCAP-NG block read has no catch point above - ``FieldBase.unpack`` and one refusal would abort a whole extraction rather than one - block; and the clamp reads only the block's own declared framing, never a running + :meth:`.FieldBase.unpack` and one refusal would abort a whole extraction rather than + one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as ``'-2s'``. No clamp fires on the sample captures, and :issue:`678`'s uncaught ``ValueError`` on truncated input is unchanged (:issue:`594`). -* **a breaking change to** ``FieldBase.length``: it called ``struct.calcsize`` on +* **a breaking change to** :attr:`.FieldBase.length`: it called ``struct.calcsize`` on a template built from a negative resolved length (e.g. ``'-5s'``, from a ``length=`` callback whose running counter had been overdrawn), raising a bare ``struct.error`` that ordinary caller code does not catch. The chokepoint every - affected schema shares now raises ``ProtocolError``, covering + affected schema shares now raises :exc:`.ProtocolError`, covering ``application/{ftp,httpv1,httpv2,ngap}.py``, ``internet/{hip,ipv6_route,mh}.py``, ``link/ethernet.py``, ``misc/pcapng.py`` and ``transport/sctp.py`` with no other file changed; a non-negative length is unaffected. This closes the follow-on the - ``httpv2`` length-guard entry under ``pcapkit.protocols`` filed as out of scope + ``httpv2`` length-guard entry under :mod:`pcapkit.protocols` filed as out of scope (:issue:`805`). -* **a breaking change to** ``EnumRegistry.register()``: it now raises ``ValueError`` - naming the existing member and pointing at ``register_alias()``, where it used to - silently alias an already-registered value under the caller's new name -- ``aenum`` - treats a taken value as an alias request, so +* **a breaking change to** :meth:`.EnumRegistry.register()`: it now raises + ``ValueError`` naming the existing member and pointing at ``register_alias()``, where + it used to silently alias an already-registered value under the caller's new name -- + ``aenum`` treats a taken value as an alias request, so ``register(existing_value, 'TOTALLY_NEW_NAME')`` returned the existing member, minted nothing and raised nothing, contradicting its docstring. - ``pcapkit.corekit.enum.EnumRegistry`` is a new mixin implementing ``get``, + :class:`pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing ``get``, ``get_all``, ``register``, ``register_alias``, ``register_aliases`` and ``_unregistered_member`` once, per the owner's ruling on :issue:`842`; it first converts registries with no bespoke ``__new__`` - (``ipv6.extension_header.ExtensionHeader``, ``tcp.flags.Flags`` and the - ``mh.*_flag`` modules), and serves ``IntEnum``, ``IntFlag`` and ``StrEnum`` alike. - The guard reaches only those: the other const registries still carry the old - generator's bare ``extend_enum(cls, name, value)``, so ``pcapkit.const`` ships - contradictory ``register`` contracts until :pr:`858` (below). These also gain the - generated template's ``KeyError`` catching on a missing name, so - ``Flags.get('NOPE', 2048)`` returns ``Flags.ACK`` rather than raising; and a key - that is neither ``int`` nor ``str``, which raised ``TypeError`` on the ``IntFlag`` - registries (``Flags.get(3.5)``), now raises ``ValueError``, tried as a value. On - ``ExtensionHeader`` it raised ``KeyError``, the shift :pr:`858`'s entry attributes to - the other registries (:pr:`855`). -* **a breaking change to** ``EnumRegistry.register()``, extended: the const + (:class:`ipv6.extension_header.ExtensionHeader `, + :class:`tcp.flags.Flags ` and the ``mh.*_flag`` + modules), and serves ``IntEnum``, ``IntFlag`` and ``StrEnum`` alike. The guard reaches + only those: the other const registries still carry the old generator's bare + ``extend_enum(cls, name, value)``, so :mod:`pcapkit.const` ships contradictory + ``register`` contracts until :pr:`858` (below). These also gain the generated + template's ``KeyError`` catching on a missing name, so ``Flags.get('NOPE', 2048)`` + returns :attr:`.Flags.ACK` rather than raising; and a key that is neither ``int`` nor + ``str``, which raised ``TypeError`` on the ``IntFlag`` registries + (``Flags.get(3.5)``), now raises ``ValueError``, tried as a value. On + :class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader` it raised ``KeyError``, + the shift :pr:`858`'s entry attributes to the other registries (:pr:`855`). +* **a breaking change to** :meth:`.EnumRegistry.register()`, extended: the const registries still on the generated template's bare ``extend_enum`` now inherit the guard :pr:`855` (above) added, so ``TransType.register(6, 'TOTALLY_NEW_NAME')`` raises ``ValueError`` naming the value and the member that holds it and pointing at @@ -629,7 +646,7 @@ Fixed ``get``/``register``/``_unregistered_member``. The modules with a bespoke ``__new__`` (``ftp/command``, ``ftp/return_code``, ``http/method``, ``http/status_code``, ``pcapng/option_type`` and ``reg/apptype`` among them) are - left alone, so **every other class** inherits ``EnumRegistry`` (:pr:`855`). + left alone, so **every other class** inherits :class:`.EnumRegistry` (:pr:`855`). ``get()``'s dispatch shifts only for a key that is neither ``int`` nor ``str`` (``KeyError``, tried as a name; now ``ValueError``, tried as a value), a key type nothing in the library passes. ``pcapkit/corekit/enums.py`` is renamed ``enum.py``, @@ -641,7 +658,7 @@ Fixed and a supplied ``default`` overrode a good value match the code never attempted. It now falls back to a plain ``_value2member_map_`` lookup once the name lookup misses and before consulting ``default``. Deliberately not - ``cls(key)``: ``FEATCode._missing_`` (``pcapkit/const/ftp/command.py``) mints via + ``cls(key)``: :meth:`.FEATCode._missing_` (``pcapkit/const/ftp/command.py``) mints via ``extend_enum`` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches ``_missing_``. A name still wins when a @@ -663,76 +680,78 @@ Fixed fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to ``default`` only. So ``Hardware.get('Definitely-Not-A-Member', 40)``, - with ``40`` in ``Hardware``'s declared-but-unassigned range, now raises ``KeyError`` - naming the original key; its test waited on :pr:`865` (above), then landed once - :pr:`865` merged. Closes :issue:`864` (:pr:`868`). -* ``ProtoChain.__add__`` returned the left operand's cached ``protocols`` and + with ``40`` in :class:`~pcapkit.const.arp.hardware.Hardware`'s declared-but-unassigned + range, now raises ``KeyError`` naming the original key; its test waited on :pr:`865` + (above), then landed once :pr:`865` merged. Closes :issue:`864` (:pr:`868`). +* :meth:`.ProtoChain.__add__` returned the left operand's cached ``protocols`` and ``aliases``: once ``a.protocols`` had been read, ``a + ProtoChain(IPv4)`` reported ``chain == 'Ethernet:IPv4'`` but ``protocols == (Ethernet,)``. ``copy.copy(self)`` carried the cache across; the result is now built fresh, as ``from_list`` builds one, and ``+=`` follows, since it falls back to ``__add__`` (:issue:`1099`). -* a ``SchemaField`` with no declared length handed its ``-1`` placeholder to the nested - schema as that schema's length, so every item of such a list warned +* a :class:`.SchemaField` with no declared length handed its ``-1`` placeholder to the + nested schema as that schema's length, so every item of such a list warned ``packet length < 0``: a HIP ``LOCATOR_SET`` and an MH CGA - Parameters item among them. ``SchemaField.unpack`` now passes what is left in the + Parameters item among them. :meth:`.SchemaField.unpack` now passes what is left in the buffer as a declared length -- not ``None``, which would turn a truncated item's - ``FieldValueError`` into ``prepare()``'s quiet ``StreamEOFError``. Fixed-length items - are unchanged; the HTTP/2 ``SETTINGS`` pair, whose size is fixed, is given it in its - own schema instead (:issue:`1140`, below) (:issue:`1148`). -* a signed ``NumberField`` decoded as unsigned, a zero-length ``PayloadField`` - misread, and an empty ``Raw`` did not rebuild (:pr:`1331`). -* ``from_dict(to_dict())`` lost nested values, and ``len()`` of an ``Info`` counted - its hidden keys (:pr:`1334`). -* ``StringField`` packs back exactly the octets it unpacked, where a guessed charset, - a ``U+FFFD`` substitution or a character-count length used to change them; the - decoded text is unchanged (:issue:`1293`, :pr:`1336`). A charset that rejects + :exc:`.FieldValueError` into ``prepare()``'s quiet :exc:`.StreamEOFError`. + Fixed-length items are unchanged; the HTTP/2 ``SETTINGS`` pair, whose size is fixed, + is given it in its own schema instead (:issue:`1140`, below) (:issue:`1148`). +* a signed :class:`.NumberField` decoded as unsigned, a zero-length + :class:`.PayloadField` misread, and an empty :class:`~pcapkit.protocols.misc.raw.Raw` + did not rebuild (:pr:`1331`). +* ``from_dict(to_dict())`` lost nested values, and ``len()`` of an :class:`.Info` + counted its hidden keys (:pr:`1334`). +* :class:`.StringField` packs back exactly the octets it unpacked, where a guessed + charset, a ``U+FFFD`` substitution or a character-count length used to change them; + the decoded text is unchanged (:issue:`1293`, :pr:`1336`). A charset that rejects ``'replace'`` falls back to UTF-8 (:issue:`1706`, :pr:`1728`), and Python 3.8 unquotes through ``pcapkit.utilities.compat.unquote`` (:issue:`1693`, :pr:`1707`). -* ``IPv4InterfaceField`` keeps a mask that is not a netmask, and a deep copy of an - ``OrderedMultiDict`` keeps its order (:pr:`1346`). +* :class:`.IPv4InterfaceField` keeps a mask that is not a netmask, and a deep copy of an + :class:`.OrderedMultiDict` keeps its order (:pr:`1346`). * **a breaking change to** padding: non-zero padding octets are kept on rebuild rather than zeroed (:issue:`1223`, :pr:`1436`). -* fields keep the bits and octets they captured on rebuild: a ``BitField``'s unnamed - bits (:issue:`1487`, :pr:`1495`), a ``NumberField``'s bits outside ``bit_length`` - (:issue:`1488`, :pr:`1497`), and the bytes an untyped ``ListField`` unpacked - (:issue:`1489`, :pr:`1494`). An unnamed ``ListField``, ``PayloadField`` or - ``SchemaField`` gets a default name (:issue:`1491`, :pr:`1493`). -* each non-final ``Info`` subclass is finalised on its own state (:issue:`1490`, +* fields keep the bits and octets they captured on rebuild: a :class:`.BitField`'s + unnamed bits (:issue:`1487`, :pr:`1495`), a :class:`.NumberField`'s bits outside + ``bit_length`` (:issue:`1488`, :pr:`1497`), and the bytes an untyped + :class:`.ListField` unpacked (:issue:`1489`, :pr:`1494`). An unnamed + :class:`.ListField`, :class:`.PayloadField` or :class:`.SchemaField` gets a default + name (:issue:`1491`, :pr:`1493`). +* each non-final :class:`.Info` subclass is finalised on its own state (:issue:`1490`, :pr:`1496`), and its constructor reads only the class's own annotations (:issue:`1692`, :pr:`1702`). * copying or pickling a stream proxy no longer recurses (:issue:`1524`, :pr:`1544`). * ``@prepare`` reads ``length`` and ``packet`` by keyword as well as by position, so ``Schema.unpack(data)`` and ``Schema.unpack(data, length=2)`` work where they raised ``IndexError`` (:issue:`444`, :pr:`450`). -* ``ListField`` unpacks a schema item from the per-item copy its callback and +* :class:`.ListField` unpacks a schema item from the per-item copy its callback and ``length_callback`` configured, not from the unconfigured original (:issue:`433`, :pr:`453`). * an ``*IntField`` given a ``signed=`` that contradicts its fixed sign, such as - ``UInt8Field(signed=True)``, raises ``FieldValueError``; the argument used to be + ``UInt8Field(signed=True)``, raises :exc:`.FieldValueError`; the argument used to be discarded silently (:issue:`545`, :pr:`549`). -* negative lengths raise ``ProtocolError`` rather than a bare stdlib error: - ``FieldBase.length`` tells a malformed template from a negative resolved length - (:issue:`825`, :pr:`827`), and ``NumberField`` checks a negative resolved +* negative lengths raise :exc:`.ProtocolError` rather than a bare stdlib error: + :attr:`.FieldBase.length` tells a malformed template from a negative resolved length + (:issue:`825`, :pr:`827`), and :class:`.NumberField` checks a negative resolved ``length`` and a negative ``bit_length`` (:issue:`828`, :pr:`829`, :issue:`831`, :pr:`834`). -* ``ModuleDescriptor.klass`` raises ``ProtocolError`` for a class name that resolves - to nothing, or one never given, rather than a bare ``AttributeError``; the +* :attr:`.ModuleDescriptor.klass` raises :exc:`.ProtocolError` for a class name that + resolves to nothing, or one never given, rather than a bare ``AttributeError``; the registry modules share one ``NULL`` sentinel compared by identity, where each had its own string (:issue:`832`, :issue:`833`, :pr:`835`). -* ``PortEnumField`` given a ``proto`` with no ``.name`` -- a string such as +* :class:`.PortEnumField` given a ``proto`` with no ``.name`` -- a string such as ``'tcp'``, or a composite such as ``TransportProtocol.tcp | TransportProtocol.udp``, which is a bare ``int`` -- raised ``AttributeError`` on a port the registry does not hold. The miss takes its transport from the registry's ``__transport__``, as - ``AppType.get`` does (:issue:`1774`, :pr:`1781`). -* ``EnumLookup.get`` answers a value miss quietly, whether ``default`` answers it or - the miss is raised: a registry's ``_validate_value`` rejection logs nothing and - installs no exception hook there, while ``EnumRegistry.register`` still reports the - same rejection loudly (:issue:`1845`, :pr:`1873`). + :meth:`.AppType.get` does (:issue:`1774`, :pr:`1781`). +* :meth:`.EnumLookup.get` answers a value miss quietly, whether ``default`` answers it + or the miss is raised: a registry's ``_validate_value`` rejection logs nothing and + installs no exception hook there, while :meth:`.EnumRegistry.register` still reports + the same rejection loudly (:issue:`1845`, :pr:`1873`). * each sentinel is a singleton, as ``None`` is: calling its type, ``copy``, ``deepcopy`` and ``pickle`` at every protocol give back the one object, and ``NO_VALUE`` reprs as its own name rather than as an object address. Their new - common base, ``Sentinel``, is public; a sentinel's type cannot be subclassed, and - neither it nor the sentinel takes an attribute write. A pickle written before this + common base, :class:`.Sentinel`, is public; a sentinel's type cannot be subclassed, + and neither it nor the sentinel takes an attribute write. A pickle written before this change at protocol 0 or 1 still loads, though not as the singleton (:issue:`1841`, :issue:`1842`, :pr:`1876`). @@ -748,15 +767,16 @@ Fixed interpolated the name unguarded, but a ``Flag`` value composed entirely of undeclared bits has ``name is None``, so the literal ``None`` landed in the name half: ``Flags(0)`` rendered as ``Flags::None [0]`` in ``json``, ``tree``, ``text``, - ``txt``, ``plist`` and ``xml``, out of both ``Extractor`` and ``TraceFlow``. A - nameless member now renders with the value's decimal spelling, so ``Flags(0)`` gives - ``Flags::0 [0]`` and ``Flags(8)`` gives ``Flags::8 [8]``. That is the spelling the - enumeration libraries use for an undeclared residue (``Flags(2057).name`` is - ``ACK|9``), and it cannot be mistaken for a member name, since an identifier may not - begin with a digit, whereas ``NONE`` is a real declared name elsewhere. The guard is - on ``name is None`` and not on the value, because the defect was never about zero - (``Flags(1)`` and ``Flags(9)`` are equally nameless). Surfaced by :pr:`634`, which - made a flagless TCP segment reach this path (:issue:`648`). + ``txt``, ``plist`` and ``xml``, out of both :class:`.Extractor` and + :class:`.TraceFlow`. A nameless member now renders with the value's decimal spelling, + so ``Flags(0)`` gives ``Flags::0 [0]`` and ``Flags(8)`` gives ``Flags::8 [8]``. That + is the spelling the enumeration libraries use for an undeclared residue + (``Flags(2057).name`` is ``ACK|9``), and it cannot be mistaken for a member name, + since an identifier may not begin with a digit, whereas ``NONE`` is a real declared + name elsewhere. The guard is on ``name is None`` and not on the value, because the + defect was never about zero (``Flags(1)`` and ``Flags(9)`` are equally nameless). + Surfaced by :pr:`634`, which made a flagless TCP segment reach this path + (:issue:`648`). * ``tests/dumpkit/test_nameless_enum_rendering_unit.py`` (added by :pr:`670`) carried probes assuming every flag-enum registry is 16 bits wide: ``test_scalar_return_renders_a_nameless_member_as_its_value`` used one fixed tuple @@ -764,17 +784,18 @@ Fixed every registry but only against ``registry(0)``. Widths differ: ``ftp.command.CommandType`` 3 bits, ``reg.apptype.TransportProtocol`` 4 (computed at runtime), the ``mh`` ack/initiate flags 8, and - ``mh.binding_update_flag.BindingUpdateFlag`` and ``tcp.flags.Flags`` 16. + :class:`mh.binding_update_flag.BindingUpdateFlag ` + and :class:`tcp.flags.Flags ` 16. ``tcp.flags.Flags(65536)`` correctly raises (``0 <= value <= 0xFFFF``), so the sweep - was failing on correct behaviour. The probes now derive each registry's bound from - its declared members, and a new test pins that one past the widest in-field value is + was failing on correct behaviour. The probes now derive each registry's bound from its + declared members, and a new test pins that one past the widest in-field value is refused. Test-only (:issue:`702`). * the plist writer emitted mapping keys unescaped, so a key carrying ``&``, ``<`` or ``>`` produced a report that is not well-formed XML. ``dictdumper``'s ``_append_dict`` interpolates the key into ``'{item}'`` and calls - ``_encode_value`` on the value only, so both branches of ``pcapkit.dumpkit.common`` + ``_encode_value`` on the value only, so both branches of :mod:`pcapkit.dumpkit.common` that build a mapping now escape their keys through one ``escape_key`` helper -- a - ``MultiDict``, where :issue:`575` escaped an enum-derived key inline and nothing + :class:`.MultiDict`, where :issue:`575` escaped an enum-derived key inline and nothing else, and a plain dict, where nothing was escaped. The json, tree and text writers are untouched. A non-``str`` key is escaped too: a PCAP-NG decryption secrets block keys TLS key log entries by a raw bytes client random whose repr can carry any of @@ -783,15 +804,15 @@ Fixed ``JarryShaw/DictDumper#121`` (:issue:`772`). * the JSON writer emitted mapping keys unescaped, so the report of ``test.pcapng`` did not parse (``Expecting ':' delimiter``): its TLS key log is keyed by a ``bytes`` client - random whose repr holds ``"`` and ``\``. ``escape_key`` in ``pcapkit.dumpkit.common`` - now JSON-escapes keys for JSON output, as :issue:`772` made it XML-escape them for - plist, and the JSON report of every ``examples/captures/`` file now parses. The - upstream defect, ``JarryShaw/DictDumper#121``, stands (:issue:`1152`). The PCAP-NG - end-to-end test of that report no longer skips; test-only (:issue:`1795`, - :pr:`1809`). + random whose repr holds ``"`` and ``\``. ``escape_key`` in + :mod:`pcapkit.dumpkit.common` now JSON-escapes keys for JSON output, as :issue:`772` + made it XML-escape them for plist, and the JSON report of every ``examples/captures/`` + file now parses. The upstream defect, ``JarryShaw/DictDumper#121``, stands + (:issue:`1152`). The PCAP-NG end-to-end test of that report no longer skips; test-only + (:issue:`1795`, :pr:`1809`). * **a breaking change to** the dump schema: the output writers dropped or corrupted - what the packets carried. An ``OrderedMultiDict``, which every option list is, now - dumps as a list of single-key entries in wire order + what the packets carried. An :class:`.OrderedMultiDict`, which every option list is, + now dumps as a list of single-key entries in wire order (:pr:`1305`). Inside an object the writers fall back on, a class, module or function is written as its qualified name rather than expanded, and an object met again on its own expansion path as ```` (:issue:`1372`, @@ -817,12 +838,15 @@ Added or flow tracing, and ``pypcapfile`` has no IPv6 decoder. Install only **one** of ``pypcap`` and ``pcap-ct``: both own the top-level ``pcap`` module, and with both present ``pcap-ct`` wins the import and the other becomes unselectable. The - missing interface constants ``PyPCAP``, ``PCAP_CT`` and ``PyPCAPFile`` are now - exported alongside ``DPKT``, ``Scapy``, ``PyShark`` and ``PCAPKit`` (:pr:`412`). - Python 3.11 is the last interpreter on which every built-in engine can run, and - even there ``pypcap`` and ``pcap-ct`` cannot coexist. -* ``EngineBase.unsupported_reason``, a preflight every engine answers and - ``Extractor.run`` consults before anything is imported. An engine that cannot + missing interface constants :data:`~pcapkit.interface.core.PyPCAP`, + :data:`~pcapkit.interface.core.PCAP_CT` and :data:`~pcapkit.interface.core.PyPCAPFile` + are now exported alongside :data:`~pcapkit.interface.core.DPKT`, + :data:`~pcapkit.interface.core.Scapy`, :data:`~pcapkit.interface.core.PyShark` and + :data:`~pcapkit.interface.core.PCAPKit` (:pr:`412`). Python 3.11 is the last + interpreter on which every built-in engine can run, and even there ``pypcap`` and + ``pcap-ct`` cannot coexist. +* :meth:`.EngineBase.unsupported_reason`, a preflight every engine answers and + :meth:`.Extractor.run` consults before anything is imported. An engine that cannot run in the current environment now gives one warning naming the real cause (a Python version, a missing ``tshark`` or ``libpcap``, the wrong ``pcap`` distribution) and a clean fall back to ``pcapkit``'s own parser, rather than an @@ -843,8 +867,8 @@ Changed dispatches on the engine type, where both branches of the old test were dead and the native adapter ran against every engine's frames (:pr:`402`). * reassembly and flow-tracing defaults moved (:pr:`435`), visible to - a caller. ``Datagram.completed`` widened from ``bool`` to a ``Completion`` enumeration - (``COMPLETE``, ``PARTIAL``, ``TIMEOUT``); only ``COMPLETE`` is truthy, so + a caller. ``Datagram.completed`` widened from ``bool`` to a :class:`.Completion` + enumeration (``COMPLETE``, ``PARTIAL``, ``TIMEOUT``); only ``COMPLETE`` is truthy, so ``if datagram.completed:`` is unaffected but ``datagram.completed == True`` no longer holds. TCP flow tracing is **bidirectional by default**, which merges each flow's two halves and closes one only once both have FINed -- 331 flows become 111 @@ -861,43 +885,44 @@ Changed :rfc:`791` specifies the opposite resolution, and records the disagreement instead (:pr:`482`). * renames with no compatibility alias left behind: ``HoleDiscriptor`` is spelled - ``HoleDescriptor`` and its package alias ``TCP_HoleDiscriptor`` is + :class:`.HoleDescriptor` and its package alias ``TCP_HoleDiscriptor`` is ``TCP_HoleDescriptor`` (:pr:`350`). -* subclass registration is **opt-in** for ``Engine``, ``Reassembly``, ``TraceFlow`` - and ``dumpkit``'s ``Dumper`` (:issue:`514`). Each registers if and only if its - registry keyword is given -- ``engine=`` for ``Engine``, ``protocol=`` for - ``Reassembly`` and ``TraceFlow``, ``fmt=`` for ``Dumper``. Previously an absent - keyword fell back to the class' own name, so *every* subclass of the public class - was registered, and declining meant subclassing the parallel ``*Base`` class under - an alias, which every built-in does (hence no public class had a subclass, while - every built-in derives from its ``*Base`` class). **This breaks out-of-tree code - that subclasses one of these classes and relies on the derived key**; pass the - keyword, or call the matching ``register_*`` function. Nothing the library ships is - affected, and the ``*Base`` classes remain importable. Silent failures are now loud: - an unrecognised class keyword raises ``UnsupportedCall`` instead of being swallowed by - ``**kwargs`` (passing ``name=`` to a ``Reassembly`` subclass silently ignored the - key, since ``protocol=`` is the real one), and so does ``Dumper``'s ``ext=`` without - ``fmt=``. A class attribute is not an opt-in: ``__engine_name__`` and - ``__protocol_name__`` still set the name a class reports, registered or not. Each - metaclass gained a class-level ``registry`` property mirroring - ``EnumSchema.registry``, and a ``Dumper`` subclass no longer touches the filesystem - while its ``class`` statement runs. ``Engine``'s keyword is ``engine=`` rather than - the ``name=`` first shipped, because ``mcls``, ``name``, ``bases`` and ``namespace`` - collide with ``abc.ABCMeta.__new__``'s parameters on Python 3.10 and raise - ``TypeError`` before the hook runs. There is no ``name=`` alias: a keyword that works - on some interpreters and not others is the trap being removed. -* **a breaking change to** ``Extractor.register_engine``, ``register_reassembly`` and - ``register_traceflow``, and the ``register_extractor_*`` wrappers over them: each - now accepts only a subclass of the public ``Engine``, ``Reassembly`` or - ``TraceFlow`` (:issue:`1016`). They had accepted a subclass of the matching +* subclass registration is **opt-in** for :class:`.Engine`, :class:`.Reassembly`, + :class:`.TraceFlow` and ``dumpkit``'s :class:`.Dumper` (:issue:`514`). Each registers + if and only if its registry keyword is given -- ``engine=`` for :class:`.Engine`, + ``protocol=`` for :class:`.Reassembly` and :class:`.TraceFlow`, ``fmt=`` for + :class:`.Dumper`. Previously an absent keyword fell back to the class' own name, so + *every* subclass of the public class was registered, and declining meant subclassing + the parallel ``*Base`` class under an alias, which every built-in does (hence no + public class had a subclass, while every built-in derives from its ``*Base`` class). + **This breaks out-of-tree code that subclasses one of these classes and relies on the + derived key**; pass the keyword, or call the matching ``register_*`` function. Nothing + the library ships is affected, and the ``*Base`` classes remain importable. Silent + failures are now loud: an unrecognised class keyword raises :exc:`.UnsupportedCall` + instead of being swallowed by ``**kwargs`` (passing ``name=`` to a + :class:`.Reassembly` subclass silently ignored the key, since ``protocol=`` is the + real one), and so does :class:`.Dumper`'s ``ext=`` without ``fmt=``. A class attribute + is not an opt-in: ``__engine_name__`` and ``__protocol_name__`` still set the name a + class reports, registered or not. Each metaclass gained a class-level ``registry`` + property mirroring :attr:`.EnumSchema.registry`, and a :class:`.Dumper` subclass no + longer touches the filesystem while its ``class`` statement runs. :class:`.Engine`'s + keyword is ``engine=`` rather than the ``name=`` first shipped, because ``mcls``, + ``name``, ``bases`` and ``namespace`` collide with ``abc.ABCMeta.__new__``'s + parameters on Python 3.10 and raise ``TypeError`` before the hook runs. There is no + ``name=`` alias: a keyword that works on some interpreters and not others is the trap + being removed. +* **a breaking change to** :meth:`.Extractor.register_engine`, ``register_reassembly`` + and ``register_traceflow``, and the ``register_extractor_*`` wrappers over them: each + now accepts only a subclass of the public :class:`.Engine`, :class:`.Reassembly` or + :class:`.TraceFlow` (:issue:`1016`). They had accepted a subclass of the matching ``*Base`` class since :issue:`513`, which was a tolerance for pcapkit's own built-ins (every built-in derives from the base, none from the public class) rather than a contract: ``*Base`` is documented as internal, and the public class is the one - carrying the registration hook. Third-party code that subclasses ``EngineBase``, - ``ReassemblyBase`` or ``TraceFlowBase`` directly and registers through these - functions now gets ``RegistryError`` naming the public class; subclass the public - class instead. The error messages are unchanged. The built-ins never pass through a - registrar; ``Extractor._register_internal_engine`` and its sibling internal + carrying the registration hook. Third-party code that subclasses :class:`.EngineBase`, + :class:`.ReassemblyBase` or :class:`.TraceFlowBase` directly and registers through + these functions now gets :exc:`.RegistryError` naming the public class; subclass the + public class instead. The error messages are unchanged. The built-ins never pass + through a registrar; ``Extractor._register_internal_engine`` and its sibling internal registrars check the base for internal registration, and are what the :issue:`513` regression test now exercises. ``register_protocol`` is untouched (:pr:`1020`). * extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical @@ -905,27 +930,27 @@ Changed rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% (IP reassembly submits a datagram for every frame, fragmented or not) (:pr:`424`). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow - dumper handed each record to a ``Frame`` constructor that re-dissected the whole - stack to return the bytes it had just been given; options are no longer parsed twice - either (:pr:`427`). -* **a breaking change to** ``Extractor.engine`` and ``Extractor.record_header``: both - are now declared to return ``EngineBase[_P]`` rather than ``Engine``, and the private - ``_exeng`` attribute follows. The built-in engines subclass ``EngineBase`` directly, - so none was ever an ``Engine``, which adds only the ``__init_subclass__`` - registration hook. A type checker sees the change: code passing the - result to an ``Engine``-typed parameter must now accept ``EngineBase``; and because - the old declarations were *unparameterised*, ``engine.read_frame()`` and - ``record_header().read_frame()`` used to be ``Any`` and are now the extractor's own - frame type, so an assignment that relied on ``Any`` there no longer type-checks - (:issue:`1022`). + dumper handed each record to a :class:`~pcapkit.protocols.misc.pcap.frame.Frame` + constructor that re-dissected the whole stack to return the bytes it had just been + given; options are no longer parsed twice either (:pr:`427`). +* **a breaking change to** :attr:`.Extractor.engine` and + :meth:`.Extractor.record_header`: both are now declared to return ``EngineBase[_P]`` + rather than :class:`.Engine`, and the private ``_exeng`` attribute follows. The + built-in engines subclass :class:`.EngineBase` directly, so none was ever an + :class:`.Engine`, which adds only the ``__init_subclass__`` registration hook. A type + checker sees the change: code passing the result to an :class:`.Engine`-typed + parameter must now accept :class:`.EngineBase`; and because the old declarations were + *unparameterised*, ``engine.read_frame()`` and ``record_header().read_frame()`` used + to be ``Any`` and are now the extractor's own frame type, so an assignment that relied + on ``Any`` there no longer type-checks (:issue:`1022`). * the documentation of the engine base classes: the member stubs (``name``, ``module``, ``registry``, ``extractor``, ``unsupported_reason``, ``run``, - ``read_frame``, ``close`` and the rest) moved from ``Engine`` to ``EngineBase``, - where they are defined, and ``Engine`` now documents only the - ``__init_subclass__`` registration hook it adds, so the ``EngineBase[_P]`` that - ``Extractor.engine`` returns leads to an API. Cross-references to ``Engine.run``, + ``read_frame``, ``close`` and the rest) moved from :class:`.Engine` to + :class:`.EngineBase`, where they are defined, and :class:`.Engine` now documents only + the ``__init_subclass__`` registration hook it adds, so the ``EngineBase[_P]`` that + :attr:`.Extractor.engine` returns leads to an API. Cross-references to ``Engine.run``, ``Engine.read_frame``, ``Engine.close`` and ``Engine.unsupported_reason`` now point - at ``EngineBase``, which stays out of ``__all__``, as :issue:`1016` settled + at :class:`.EngineBase`, which stays out of ``__all__``, as :issue:`1016` settled (:issue:`1024`). * **a breaking change to** ``register_mh_extension``: ``schema`` is keyword-only, as in the other schema-taking registrars (:issue:`516`, :pr:`526`). @@ -940,11 +965,11 @@ Fixed * ``format='text'`` raised ``AttributeError`` before writing anything, naming a ``dictdumper.Text`` that has never existed. It now points at ``Tree``, as the ``'txt'`` alias beside it already did. -* ``Extractor`` had the ownership of its input stream inverted: a handle it opened +* :class:`.Extractor` had the ownership of its input stream inverted: a handle it opened itself, from ``fin`` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. ``_cleanup`` closed under ``not self._flag_s`` (the flag that gated the ``open()``), and the - ``SeekableReader`` wrapping a non-seekable input asked for + :class:`.SeekableReader` wrapping a non-seekable input asked for ``stream_closing=not self._flag_s`` in the same wrong direction. The leak was one descriptor per extraction, the root cause of the ``ResourceWarning`` that reddened :pr:`577`, :pr:`596` and :pr:`600` (:issue:`606` had only made the test immune). @@ -952,7 +977,7 @@ Fixed finaliser releases the handle of an extraction *abandoned* before end of file (``auto=False``, iterated part way and dropped); the files :issue:`610` names emit no such warning. ``__exit__`` deliberately still - closes unconditionally, since a caller who scopes an ``Extractor`` with ``with`` + closes unconditionally, since a caller who scopes an :class:`.Extractor` with ``with`` asked for that (:issue:`610`). * ``extract(..., no_eof=True)`` never returned. End of stream was detected and ``ExtractionWarning: EOF reached`` fired, but every loop that handles it @@ -979,35 +1004,36 @@ Fixed ``except Exception`` handlers are never entered during the spin, and the memory is the retry loop's ``EOF reached`` records, which the runner retains (:issue:`620`). * ``register_protocol`` displaced one protocol class with another and said nothing. - The registry behind ``pcapkit.protocols.__proto__`` is keyed on + The registry behind :data:`pcapkit.protocols.__proto__` is keyed on ``cls.__name__.upper()``, and several dispatchable classes are named ``HTTP`` - (the base ``pcapkit.protocols.application.http.HTTP`` and the ``httpv1`` and + (the base :class:`pcapkit.protocols.application.http.HTTP` and the ``httpv1`` and ``httpv2`` implementations), so a bare assignment decided the winner: ``register_protocol(httpv2.HTTP)`` moved ``__proto__['HTTP']`` to the ``httpv2`` - class with no exception, log line or ``RegistryWarning``, and + class with no exception, log line or :exc:`.RegistryWarning`, and ``ProtocolBase.expand_comp('HTTP')`` then resolved to whichever registered last. The - overwrite now raises a ``RegistryWarning`` naming the displaced and the replacing + overwrite now raises a :exc:`.RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: - ``Protocol.__init_subclass__`` registers unconditionally, so merely *defining* a - subclass of the public ``Protocol`` under a name a built-in holds displaced that - built-in. A plain ``import pcapkit`` emits zero new warnings: the built-in - ``HTTP`` classes derive from ``ProtocolBase``, so ``__init_subclass__`` never fires - for them, and import-time seeding keys off the distinct names ``HTTP``, ``HTTPv1`` - and ``HTTPv2``. The guard deliberately reads "key present and the incumbent is a - different class", not mere presence (:pr:`726` later gave its sibling - registrars the same identity guard): this function is the funnel every wrapper - registrar ends in, so registering one class under two codes (``register_tcp`` then - ``register_udp``, a documented path) reaches it twice with nothing displaced. The - collision is reported, not resolved: re-keying is a registry-format change that none - of the bare-name readers can absorb (each degrades *silently* to a string or ``Raw`` - on a miss), so it belongs to the registry redesign in :issue:`514`. The issue called - this the one registrar with no warning; in fact none warned by itself, the siblings - only by delegating to a guarded classmethod (:issue:`675`). + :meth:`.Protocol.__init_subclass__` registers unconditionally, so merely *defining* a + subclass of the public :class:`~pcapkit.protocols.protocol.Protocol` under a name a + built-in holds displaced that built-in. A plain ``import pcapkit`` emits zero new + warnings: the built-in ``HTTP`` classes derive from :class:`.ProtocolBase`, so + ``__init_subclass__`` never fires for them, and import-time seeding keys off the + distinct names ``HTTP``, ``HTTPv1`` and ``HTTPv2``. The guard deliberately reads "key + present and the incumbent is a different class", not mere presence (:pr:`726` later + gave its sibling registrars the same identity guard): this function is the funnel + every wrapper registrar ends in, so registering one class under two codes + (``register_tcp`` then ``register_udp``, a documented path) reaches it twice with + nothing displaced. The collision is reported, not resolved: re-keying is a + registry-format change that none of the bare-name readers can absorb (each degrades + *silently* to a string or :class:`~pcapkit.protocols.misc.raw.Raw` on a miss), so it + belongs to the registry redesign in :issue:`514`. The issue called this the one + registrar with no warning; in fact none warned by itself, the siblings only by + delegating to a guarded classmethod (:issue:`675`). * the ``#:`` autodoc comments in ``pcapkit/foundation/traceflow/traceflow.py`` that named a bare ``Type``, which Sphinx resolves against every class named ``Type`` in the project rather than ``typing.Type``, and silently linked to - ``pcapkit.const.l2tp.type.Type``, an L2TP field-type enum; once :issue:`684` - rendered ``TraceFlow._foutio``, the built docs pointed at the wrong class with no + :class:`pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once :issue:`684` + rendered :attr:`.TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. These sites now spell it ``~typing.Type[Dumper]``, among the sites :issue:`709` lists. The other bare ``Type`` sites, in hand-written ``:type:`` fields under ``docs/source/pcapkit/foundation/``, are fixed separately by @@ -1026,59 +1052,60 @@ Fixed outside the wording of :issue:`718`'s identity guard (``incumbent is not None and incumbent is not new``, landed by :pr:`726`), whose issue named only code-keyed registrars: ``register_engine``, ``register_reassembly`` and ``register_traceflow`` - on ``Extractor``, and ``register_dumper`` on both ``Extractor`` and ``TraceFlow``. - Each now compares the incumbent by identity + on :class:`.Extractor`, and ``register_dumper`` on both :class:`.Extractor` and + :class:`.TraceFlow`. Each now compares the incumbent by identity before warning; both ``register_dumper`` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings (:issue:`739`). -* **the class registrars listed below** now raise ``RegistryError`` for a non-class +* **the class registrars listed below** now raise :exc:`.RegistryError` for a non-class argument, where a bare ``TypeError`` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the ``register_dumper`` methods, and from inside ``abc`` for the others. They are ``register_dumper``, ``register_engine``, - ``register_reassembly`` and ``register_traceflow`` on ``Extractor``, - ``TraceFlow.register_dumper`` (reached + ``register_reassembly`` and ``register_traceflow`` on :class:`.Extractor`, + :meth:`.TraceFlow.register_dumper` (reached through ``register_traceflow_dumper``), and - ``pcapkit.foundation.registry.protocols.register_protocol``. Each guard gains an + :func:`pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that - accept a ``ModuleDescriptor`` it runs after the descriptor is unwrapped, so a + accept a :class:`.ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while ``register_protocol`` takes no descriptor. No guard's target class changes, and a wrong class still raises - ``RegistryError`` as before. ``RegistryError`` subclasses ``TypeError``, so + :exc:`.RegistryError` as before. :exc:`.RegistryError` subclasses ``TypeError``, so ``except TypeError`` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocol ``register`` classmethods is fixed separately by :issue:`1026` (:issue:`1021`). -* a missing extraction engine package now gives a single ``EngineWarning`` rather than - a duplicate (:issue:`1045`, :pr:`1046`). ``Extractor.import_test`` warned that the - package was absent, then ``Extractor.run`` warned again when it fell back to the - default engine; ``run`` now stays silent there, and ``import_test`` says what ``run`` - used to: ``engine () is not installed; using default engine instead``, - the module name in backticks. Only code that matched ``import_test``'s old wording, or +* a missing extraction engine package now gives a single :exc:`.EngineWarning` rather + than a duplicate (:issue:`1045`, :pr:`1046`). :meth:`.Extractor.import_test` warned + that the package was absent, then :meth:`.Extractor.run` warned again when it fell + back to the default engine; ``run`` now stays silent there, and ``import_test`` says + what ``run`` used to: + ``engine () is not installed; using default engine instead``, the + module name in backticks. Only code that matched ``import_test``'s old wording, or that expects the duplicate warning, breaks. -* ``Extractor`` defects: ``Extractor.output``'s ``UnsupportedCall`` named +* :class:`.Extractor` defects: :attr:`.Extractor.output`'s :exc:`.UnsupportedCall` named ``format`` rather than ``output``; ``make_name`` looked an unknown format up in the ``__output__`` ``defaultdict`` and so inserted it; and the Scapy engine never set - ``_offmt``, so ``Extractor.format`` raised ``AttributeError`` under + ``_offmt``, so :attr:`.Extractor.format` raised ``AttributeError`` under ``engine='scapy'`` (:issue:`1095`). -* the DPKT, Scapy and PyShark engines call ``Extractor.record_header()`` at setup +* the DPKT, Scapy and PyShark engines call :meth:`.Extractor.record_header()` at setup again, as they did until 2023, so the global-header step is shared with the default engine and sets ``_offmt`` for each of them. **Their dumped output changes**: JSON, plist and tree reports gain the leading ``Global Header`` record (``Section Header 1`` for PCAP-NG) the default engine already writes, identical to its, and ``files=True`` - writes that record's file too; frame records are unchanged. ``Extractor.format`` now - also answers on a capture with no frames. ``format='pcap'`` failing on every engine is - a separate defect, :issue:`1127` (below) (:issue:`1104`). + writes that record's file too; frame records are unchanged. :attr:`.Extractor.format` + now also answers on a capture with no frames. ``format='pcap'`` failing on every + engine is a separate defect, :issue:`1127` (below) (:issue:`1104`). * ``format='pcap'`` failed on every engine with a ``TypeError`` from - ``PCAPIO.__init__()``, missing ``protocol``: ``Extractor.__init__`` created the writer - before any global header was read, so ``PCAPIO`` had no link type. The header step - now creates it through ``Extractor._open_output``, and ``run()`` opens it for engines - with no such step; a PCAP writer takes the frame itself and skips the - ``Global Header`` record, so the default engine writes ``in.pcap`` back byte for + ``PCAPIO.__init__()``, missing ``protocol``: :meth:`.Extractor.__init__` created the + writer before any global header was read, so :class:`.PCAPIO` had no link type. The + header step now creates it through :meth:`.Extractor._open_output`, and ``run()`` + opens it for engines with no such step; a PCAP writer takes the frame itself and skips + the ``Global Header`` record, so the default engine writes ``in.pcap`` back byte for byte. The third-party engines hand the writer no frame octets, so they fall back to - JSON with a ``FormatWarning``, as their flow tracing already does, and a PCAP-NG - input raises ``FormatError`` (:issue:`1127`). The written global header also + JSON with a :exc:`.FormatWarning`, as their flow tracing already does, and a PCAP-NG + input raises :exc:`.FormatError` (:issue:`1127`). The written global header also dropped the input's ``thiszone``, ``sigfigs`` and ``snaplen``, so most sample PCAP files did not come back byte for byte; the PCAP engine now passes those fields - through ``_open_output`` to ``PCAPIO``, and every one does (:issue:`1165`). + through ``_open_output`` to :class:`.PCAPIO`, and every one does (:issue:`1165`). * IP reassembly reports only the octets it holds: a truncated fragment, a strict run and a zero-length datagram no longer report octets from the declared Total Length (:pr:`1368`). Only the final fragment's partial block counts, and a datagram too @@ -1123,7 +1150,7 @@ pcapkit.interface Added ~~~~~ -* ``pcapkit.interface.registry``, making public the registry-entry helpers +* :mod:`pcapkit.interface.registry`, making public the registry-entry helpers ``same_entry``, ``restore_shipped`` and ``restore_shipped_dumper`` (:issue:`1519`, :pr:`1568`). @@ -1136,9 +1163,9 @@ Added * ESP parsing and construction [:rfc:`4303`], with optional payload decryption and ICV verification through ``cryptography`` (``pip install pypcapkit[crypto]``) (:pr:`378`). Keys reach the dissector through a new caller-state channel, - ``pcapkit.corekit.context``, surfaced as the ``context=`` keyword on ``extract()`` - and ``Extractor``, carrying an ``esp.SecurityAssociation``. Nothing here raises: - with no association the SPI and sequence number are still reported and the + :mod:`pcapkit.corekit.context`, surfaced as the ``context=`` keyword on ``extract()`` + and :class:`.Extractor`, carrying an :class:`.esp.SecurityAssociation`. Nothing here + raises: with no association the SPI and sequence number are still reported and the ciphertext is left opaque with ``status=NO_SA``, and a failed decryption or ICV check is recorded on the parsed result the same way. Extended Sequence Numbers, TFC padding and anti-replay are not implemented, and an unsupported cipher or MAC is @@ -1166,42 +1193,51 @@ Added option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the CGA extensions round-trip end to end. * dispatch entries for dissectors that existed but were reachable from no registry - (:pr:`436`): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, ``L2TPv2`` - on UDP 1701 and OSPF at ``TransType`` 89. ``VLAN`` became an abstract base with - ``C_Tag`` (802.1Q) and ``S_Tag`` (802.1ad) as concrete subclasses, so a Q-in-Q - frame no longer collapses into one opaque ``Raw``; ``L2TP`` likewise became a - base, with ``L2TPv2`` carrying the :rfc:`2661` implementation. -* ``Protocol``/``ProtocolBase`` gain a ``code=`` class keyword for the next-layer - dispatch registries (``Link.__proto__``, ``Internet.__proto__``, ``TCP.__proto__``, - ``UDP.__proto__``, ``SCTP.__proto__``, ``Frame.__proto__``, ``PCAPNG.__proto__``), - the opt-in treatment :issue:`514` gave ``Engine``, ``Reassembly``, ``TraceFlow`` and - ``Dumper`` above, extended to the one family that needed a registration key - invented. Omitting ``code`` leaves a subclass unregistered, as before; the built-in - tables are still filled by literal assignment in each layer module, so nothing the - library ships moves. ``code`` accepts a bare enum member, whose *type* infers the - destination (``EtherType`` means ``Link``, ``TransType`` means ``Internet``, - ``PayloadProtocolIdentifier`` means ``SCTP``, and ``LinkType`` means *both* - ``Frame`` **and** ``PCAPNG``, mirroring ``register_linktype``), or a + (:pr:`436`): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, + :class:`.L2TPv2` on UDP 1701 and OSPF at + :class:`~pcapkit.const.reg.transtype.TransType` 89. + :class:`~pcapkit.protocols.link.vlan.VLAN` became an abstract base with + :class:`.C_Tag` (802.1Q) and :class:`.S_Tag` (802.1ad) as concrete subclasses, so a + Q-in-Q frame no longer collapses into one opaque + :class:`~pcapkit.protocols.misc.raw.Raw`; :class:`~pcapkit.protocols.link.l2tp.L2TP` + likewise became a base, with :class:`.L2TPv2` carrying the :rfc:`2661` implementation. +* :class:`~pcapkit.protocols.protocol.Protocol`/:class:`.ProtocolBase` gain a ``code=`` + class keyword for the next-layer dispatch registries (:attr:`.Link.__proto__`, + :attr:`.Internet.__proto__`, :attr:`.TCP.__proto__`, :attr:`.UDP.__proto__`, + :attr:`.SCTP.__proto__`, :attr:`.Frame.__proto__`, :attr:`.PCAPNG.__proto__`), the + opt-in treatment :issue:`514` gave :class:`.Engine`, :class:`.Reassembly`, + :class:`.TraceFlow` and :class:`.Dumper` above, extended to the one family that needed + a registration key invented. Omitting ``code`` leaves a subclass unregistered, as + before; the built-in tables are still filled by literal assignment in each layer + module, so nothing the library ships moves. ``code`` accepts a bare enum member, whose + *type* infers the destination (:class:`~pcapkit.const.reg.ethertype.EtherType` means + :class:`.Link`, :class:`~pcapkit.const.reg.transtype.TransType` means + :class:`.Internet`, + :class:`~pcapkit.const.sctp.payload_protocol_identifier.PayloadProtocolIdentifier` + means :class:`~pcapkit.protocols.transport.sctp.SCTP`, and + :class:`~pcapkit.const.reg.linktype.LinkType` means *both* + :class:`~pcapkit.protocols.misc.pcap.frame.Frame` **and** + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`, mirroring ``register_linktype``), or a ``{destination: key}`` mapping, required for a raw ``int`` such as a port number, - which cannot say which transport it belongs to. Either form may appear in an - iterable, so one declaration can register a class into several registries (e.g. a - ``L2TP`` subclass reachable by IP protocol number and by a UDP port), and the - mapping form is accepted even where inference would work. Inference refuses rather - than guesses: an enum member whose type names no known destination raises - ``RegistryError``, and an unrecognised class keyword raises ``UnsupportedCall``, - matching the other registrar families. Backed by - ``pcapkit.foundation.registry.protocols.register_protocol_code``, also callable + which cannot say which transport it belongs to. Either form may appear in an iterable, + so one declaration can register a class into several registries (e.g. a + :class:`~pcapkit.protocols.link.l2tp.L2TP` subclass reachable by IP protocol number + and by a UDP port), and the mapping form is accepted even where inference would work. + Inference refuses rather than guesses: an enum member whose type names no known + destination raises :exc:`.RegistryError`, and an unrecognised class keyword raises + :exc:`.UnsupportedCall`, matching the other registrar families. Backed by + :func:`pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the - mechanism :issue:`548` needs; :issue:`548` turned out to need a missing class - instead (115 is :rfc:`3931` L2TPv3 over IP, not :rfc:`2661`; see the corresponding - **Fixed** entry below), so its worked example names ``L2TPv3`` (:pr:`570`). + mechanism :issue:`548` needs; :issue:`548` turned out to need a missing class instead + (115 is :rfc:`3931` L2TPv3 over IP, not :rfc:`2661`; see the corresponding **Fixed** + entry below), so its worked example names ``L2TPv3`` (:pr:`570`). * ``tests/protocols/test_dispatch_reachability_unit.py``, the coverage :issue:`548` - asked for: every ``ProtocolBase`` descendant whose ``__index__`` returns an enum + asked for: every :class:`.ProtocolBase` descendant whose ``__index__`` returns an enum member must be reachable under that code in the registry its enum *type* designates, read from the same ``_CODE_DESTINATIONS`` table backing ``code=`` so the two cannot drift. It walks the classes rather than the registry entries, so it - catches a class nothing registered, the shape in which ``OSPF`` once shipped - (:issue:`548`). + catches a class nothing registered, the shape in which + :class:`~pcapkit.protocols.application.ospf.OSPF` once shipped (:issue:`548`). * ``tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py``, covering all three MP_JOIN layouts through the *public* constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the @@ -1210,7 +1246,7 @@ Added one. The stale ``tcp-mptcp/MP_JOIN`` entry is deleted from ``EXPECTED_FAILURES`` (:issue:`587`). * ``tests/protocols/test_option_generator_tcp_base_unit.py``, asserting that every - ``TCP_BASE`` key is a parameter ``TCP.make`` declares (derived from + ``TCP_BASE`` key is a parameter :meth:`.TCP.make` declares (derived from ``inspect.signature``) and that the built segment carries the stated header in both data model and packed octets (:issue:`602`). * ILNP nonce sizing coverage in @@ -1234,131 +1270,145 @@ Added Changed ~~~~~~~ -* **a breaking change to** ``OSPF``, ``RARP`` and ``DRARP``: they move from - ``pcapkit.protocols.link`` to ``pcapkit.protocols.application``, with no deprecated - re-export at the old path (:issue:`719`). Layer is decided by designed function, with - the IETF as the single source of truth -- :rfc:`1812#section-7` places routing - protocols in the application layer and :rfc:`1122#section-1.1.3` lists RARP there -- - and the rule is recorded in ``docs/source/contributing/conventions/protocol-layer-placement.rst``. +* **a breaking change to** :class:`~pcapkit.protocols.application.ospf.OSPF`, + :class:`.RARP` and :class:`.DRARP`: they move from :mod:`pcapkit.protocols.link` to + :mod:`pcapkit.protocols.application`, with no deprecated re-export at the old path + (:issue:`719`). Layer is decided by designed function, with the IETF as the single + source of truth -- :rfc:`1812#section-7` places routing protocols in the application + layer and :rfc:`1122#section-1.1.3` lists RARP there -- and the rule is recorded in + ``docs/source/contributing/conventions/protocol-layer-placement.rst``. Import paths, before to after: - * ``pcapkit.protocols.link.ospf`` to ``pcapkit.protocols.application.ospf`` - * ``pcapkit.protocols.data.link.ospf`` to ``pcapkit.protocols.data.application.ospf`` - * ``pcapkit.protocols.schema.link.ospf`` to ``pcapkit.protocols.schema.application.ospf`` - * ``pcapkit.protocols.link.rarp`` to ``pcapkit.protocols.application.rarp`` + * ``pcapkit.protocols.link.ospf`` to :mod:`pcapkit.protocols.application.ospf` + * ``pcapkit.protocols.data.link.ospf`` to + :mod:`pcapkit.protocols.data.application.ospf` + * ``pcapkit.protocols.schema.link.ospf`` to + :mod:`pcapkit.protocols.schema.application.ospf` + * ``pcapkit.protocols.link.rarp`` to :mod:`pcapkit.protocols.application.rarp` ``from pcapkit import OSPF`` and ``from pcapkit.protocols import OSPF`` (likewise - ``RARP`` and ``DRARP``) are unaffected; ``from pcapkit.protocols.link import OSPF`` - and the deep paths above break. RARP has no ``data`` or ``schema`` module of its own, as - it reuses ARP's. + :class:`.RARP` and :class:`.DRARP`) are unaffected; + ``from pcapkit.protocols.link import OSPF`` and the deep paths above break. RARP has + no ``data`` or ``schema`` module of its own, as it reuses ARP's. ``layer`` changes from ``'Link'`` to ``'Application'`` for OSPF, RARP and DRARP. - ``OSPF`` now takes ``Application`` as its only base, and ``RARP`` becomes - ``class RARP(Application, ARP)`` -- layer base first, because ``ARP``'s chain reaches - ``Link``, which owns ``__layer__``. ``ARP`` and ``InARP`` stay ``'Link'``. Dispatch - keys are unchanged: OSPF is still reached from ``Internet.__proto__`` at - ``TransType.OSPFIGP`` and RARP from ``Link.__proto__`` by EtherType. Leaving ``Link`` - repoints ``OSPF.__proto__`` at ``ProtocolBase.__proto__``, so OSPF no longer sees - Link's EtherType entries -- which nothing used, since it is reached by protocol - number, and a ``-1`` lookup falls back to - ``Raw`` in either registry without inserting. ``register`` and ``_read_protos`` still - resolve, on ``ProtocolBase`` rather than ``Link``. ``RARP`` keeps Link's through ``ARP``. + :class:`~pcapkit.protocols.application.ospf.OSPF` now takes :class:`.Application` as + its only base, and :class:`.RARP` becomes ``class RARP(Application, ARP)`` -- layer + base first, because :class:`~pcapkit.protocols.link.arp.ARP`'s chain reaches + :class:`.Link`, which owns ``__layer__``. :class:`~pcapkit.protocols.link.arp.ARP` and + :class:`.InARP` stay ``'Link'``. Dispatch keys are unchanged: OSPF is still reached + from :attr:`.Internet.__proto__` at :attr:`.TransType.OSPFIGP` and RARP from + :attr:`.Link.__proto__` by EtherType. Leaving :class:`.Link` repoints + ``OSPF.__proto__`` at :attr:`.ProtocolBase.__proto__`, so OSPF no longer sees Link's + EtherType entries -- which nothing used, since it is reached by protocol number, and a + ``-1`` lookup falls back to :class:`~pcapkit.protocols.misc.raw.Raw` in either + registry without inserting. ``register`` and ``_read_protos`` still resolve, on + :class:`.ProtocolBase` rather than :class:`.Link`. :class:`.RARP` keeps Link's through + :class:`~pcapkit.protocols.link.arp.ARP`. Layer-limited extraction: ``layer='link'`` no longer sets the termination flag on OSPF and ``layer='application'`` now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every ``layer=`` value, including ``'internet'`` (``IPv4:OSPFIGP`` before and after, as IPv4 ends the chain itself). -* renames with no compatibility alias left behind: PCAP-NG ``Option`` subclasses spell - the namespace class keyword ``ns=`` instead of ``namespace=`` (:issue:`439`). +* renames with no compatibility alias left behind: PCAP-NG + :class:`~pcapkit.protocols.schema.misc.pcapng.Option` subclasses spell the namespace + class keyword ``ns=`` instead of ``namespace=`` (:issue:`439`). * ``tests/protocols/transport/test_tcp_udp_unit.py`` now reaches the MP_JOIN - dispatchers through ``TCP()`` itself, instead of assigning a Python ``set`` to - ``_flags`` on a bare ``TCP.__new__(TCP)``. A ``set`` answers the same membership - tests, so every branch ran while the attribute had neither the type production - assigns nor the ordering that governs when it exists -- how :issue:`587` stayed - invisible. Reverting :issue:`587`'s hoist, or restoring the + dispatchers through :class:`TCP() ` itself, + instead of assigning a Python ``set`` to ``_flags`` on a bare ``TCP.__new__(TCP)``. A + ``set`` answers the same membership tests, so every branch ran while the attribute had + neither the type production assigns nor the ordering that governs when it exists -- + how :issue:`587` stayed invisible. Reverting :issue:`587`'s hoist, or restoring the ``cast('Enum_Flags', 0)`` seed, now fails the file. Test-only (:issue:`603`). * building a protocol through its constructor with a keyword that names nothing now - raises ``UnsupportedCall`` instead of discarding it. **This is a behaviour change to - a public API**: every ``make`` in the tree ends its signature with ``**kwargs`` and + raises :exc:`.UnsupportedCall` instead of discarding it. **This is a behaviour change + to a public API**: every ``make`` in the tree ends its signature with ``**kwargs`` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in :issue:`602`, - :issue:`541` and :issue:`556`. The check sits in ``ProtocolBase.__init__`` rather + :issue:`541` and :issue:`556`. The check sits in :meth:`.ProtocolBase.__init__` rather than ``make``, because ``__post_init__`` passes one ``**kwargs`` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by - ``read`` legitimately travels through ``make`` (``HIP.read`` declares + ``read`` legitimately travels through ``make`` (:meth:`.HIP.read` declares ``extension``). The accepted set is the union of every keyword-taking parameter of ``make``, ``read``, ``pack``, ``unpack``, ``__post_init__`` and ``__init__`` across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new ``__keywords__`` opts a class out: a set, for a keyword read - out of ``**kwargs`` by name (``ESP.read`` with ``packet``); and ``None``, for a + out of ``**kwargs`` by name (:meth:`.ESP.read` with ``packet``); and ``None``, for a dispatcher whose real signature belongs to a class chosen at call time (only - ``HTTP.make``). The message names the near neighbour (``seq`` reports *did you mean - 'seq_no'?*). ``from_data`` warns ``UnknownFieldWarning`` rather than raising for a key - its own ``_make_data`` returned, which the caller cannot fix; a keyword passed to - ``from_data`` itself raises, as for a direct construction (:pr:`1147`). The leftover - copies of :issue:`602`'s misspelled TCP base, in - ``examples/generators/dispatch.py`` and ``tests/protocols/transport/``, are fixed. - Some ``_make_data`` keys that ``make`` does not declare are reported, not fixed: - ``Frame._make_data`` returns ``ts_src`` where ``make`` declares ``ts_sec``, - ``L2TPv2._make_data`` returns ``prio`` where it declares ``priority``, and - ``Header._make_data`` returns a ``magic_number`` that ``Header.make`` does not take. - A *direct* ``SomeProtocol.make(...)`` call is not checked and still discards - silently (:issue:`617`). -* **a breaking change to** ``Application``: it no longer forbids payload dispatch + :meth:`HTTP.make `). The message names + the near neighbour (``seq`` reports *did you mean 'seq_no'?*). ``from_data`` warns + :exc:`.UnknownFieldWarning` rather than raising for a key its own ``_make_data`` + returned, which the caller cannot fix; a keyword passed to ``from_data`` itself + raises, as for a direct construction (:pr:`1147`). The leftover copies of + :issue:`602`'s misspelled TCP base, in ``examples/generators/dispatch.py`` and + ``tests/protocols/transport/``, are fixed. Some ``_make_data`` keys that ``make`` does + not declare are reported, not fixed: :meth:`.Frame._make_data` returns ``ts_src`` + where ``make`` declares ``ts_sec``, :meth:`.L2TPv2._make_data` returns ``prio`` where + it declares ``priority``, and :meth:`.Header._make_data` returns a ``magic_number`` + that :meth:`.Header.make` does not take. A *direct* ``SomeProtocol.make(...)`` call is + not checked and still discards silently (:issue:`617`). +* **a breaking change to** :class:`.Application`: it no longer forbids payload dispatch outright. It encodes "no further protocol layer above this one", which undissected trailer bytes are not, so ``_decode_next_layer`` and ``_import_next_layer`` accept - the ``-1`` sentinel -- the rest of the packet, resolving to ``Raw``, or to - ``NoPayload`` when nothing remains -- and ``__post_init__`` keeps a payload that - ``read`` set instead of overwriting it with ``NoPayload``. A real protocol number - (any ``proto`` other than ``-1``) is still refused with ``UnsupportedCall``. A - subclass that calls either method with ``-1`` changes from raising to succeeding; - no subclass in the package does (:issue:`719`). -* the ``Protocol`` and ``ProtocolMeta`` docstrings. ``Protocol`` claimed to be the base - of every protocol family, though nothing in the package subclasses it, and - ``ProtocolMeta`` claimed to generate attributes its empty body does not. ``Protocol`` - is now documented as the base for third-party protocols, and ``ProtocolBase`` as the - one the families derive from (:issue:`1101`). + the ``-1`` sentinel -- the rest of the packet, resolving to + :class:`~pcapkit.protocols.misc.raw.Raw`, or to + :class:`~pcapkit.protocols.misc.null.NoPayload` when nothing remains -- and + ``__post_init__`` keeps a payload that ``read`` set instead of overwriting it with + :class:`~pcapkit.protocols.misc.null.NoPayload`. A real protocol number (any ``proto`` + other than ``-1``) is still refused with :exc:`.UnsupportedCall`. A subclass that + calls either method with ``-1`` changes from raising to succeeding; no subclass in the + package does (:issue:`719`). +* the :class:`~pcapkit.protocols.protocol.Protocol` and :class:`.ProtocolMeta` + docstrings. :class:`~pcapkit.protocols.protocol.Protocol` claimed to be the base of + every protocol family, though nothing in the package subclasses it, and + :class:`.ProtocolMeta` claimed to generate attributes its empty body does not. + :class:`~pcapkit.protocols.protocol.Protocol` is now documented as the base for + third-party protocols, and :class:`.ProtocolBase` as the one the families derive from + (:issue:`1101`). * **a breaking change to** ``OSPF_CrytographicAuthentication``: the misspelling is corrected to ``OSPF_CryptographicAuthentication``, and ``CrytographicAuthentication`` - in ``pcapkit.protocols.schema.application.ospf`` and - ``pcapkit.protocols.data.application.ospf`` to ``CryptographicAuthentication``, with - no alias left for the old spelling. Both OSPF names also move to the application group - of ``schema.__all__`` and ``data.__all__``, after :issue:`719`'s move (:issue:`1090`). + in :mod:`pcapkit.protocols.schema.application.ospf` and + :mod:`pcapkit.protocols.data.application.ospf` to ``CryptographicAuthentication``, + with no alias left for the old spelling. Both OSPF names also move to the application + group of ``schema.__all__`` and ``data.__all__``, after :issue:`719`'s move + (:issue:`1090`). * **a breaking change to** a declared length that runs past the data: it is refused - with ``ProtocolError`` rather than accepted and zero-filled on rebuild, so a + with :exc:`.ProtocolError` rather than accepted and zero-filled on rebuild, so a malformed option looks different in ``info`` (:issue:`1325`, :pr:`1398`). * dissection is bounded: a frame at 64 layers (:issue:`1610`, :pr:`1620`) and an IPv6 extension-header chain at 32 headers (:issue:`1604`, :pr:`1608`). Both limits are ``PCAPKIT_FRAME_LAYER_LIMIT`` and ``PCAPKIT_EXTENSION_HEADER_LIMIT`` in - ``pcapkit.utilities.config`` (:pr:`1743`). Rebuilding a + :mod:`pcapkit.utilities.config` (:pr:`1743`). Rebuilding a frame cut at the chain bound warns once (:issue:`1619`, :pr:`1630`). * **a breaking change to** IPv6 extension headers: an unrecognised or unparsable one - is parsed generically, per :rfc:`6564`, rather than collapsing the packet to ``Raw`` - (:issue:`891`, :pr:`904`). ``IPv6_Ext`` does it, and is also the shared base of - every extension header (:issue:`917`, :pr:`924`). + is parsed generically, per :rfc:`6564`, rather than collapsing the packet to + :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`891`, :pr:`904`). + :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` does it, and is also the shared + base of every extension header (:issue:`917`, :pr:`924`). * **a breaking change to** ``pack()``: it no longer keeps the header it builds on the instance it is called on, which in 1.4.1 too left ``schema`` disagreeing with ``info``: ``UDP(srcport=1234).pack()`` reset ``schema.srcport`` to 0 while ``info.srcport`` kept 1234. Once an instance has been parsed or constructed, ``pack()`` builds on a shallow copy, so ``schema``, ``info`` and what ``make()`` - sets, such as TCP's ``connection``, stay as they were. The ``Frame``, ``PCAPNG`` - and ``HTTPv2`` overrides do the same, and re-packing a PCAP-NG block that carries - an option allowed only once no longer raises. Code that relied on ``pack()`` to - re-sync ``schema`` calls ``make()`` or constructs a new instance (:issue:`1769`, - :pr:`1791`). + sets, such as TCP's ``connection``, stay as they were. The + :class:`~pcapkit.protocols.misc.pcap.frame.Frame`, + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` and ``HTTPv2`` overrides do the same, + and re-packing a PCAP-NG block that carries an option allowed only once no longer + raises. Code that relied on ``pack()`` to re-sync ``schema`` calls ``make()`` or + constructs a new instance (:issue:`1769`, :pr:`1791`). * **a breaking change to** the HOPOPT, IPv6-Opts and Mobility Header padding: two octets are one ``PadN`` with no data (``01 00``) rather than two ``Pad1`` options, as :rfc:`8200#section-4.2` and :rfc:`6275#section-6.2.2` ask. Only a freshly built header with exactly two octets of padding changes; a parsed one that carried two ``Pad1`` options still rebuilds byte for byte (:issue:`1793`, :pr:`1812`). -* **a breaking change to** ``NTPTimestamp``: it and ``PMIPv6Timestamp`` are ``Info`` - classes in ``pcapkit.protocols.data.internet.mh`` rather than tuples, and - ``NTPTimestamp`` keeps no alias at its 1.4.x home in - ``pcapkit.protocols.internet.mh``. As Mappings they no longer index, unpack, compare - or hash as a tuple does; ``.seconds`` and ``.fraction`` still work, and the dumpers - write both as keys (:issue:`1886`, :pr:`1887`). +* **a breaking change to** :class:`.NTPTimestamp`: it and :class:`.PMIPv6Timestamp` are + :class:`.Info` classes in :mod:`pcapkit.protocols.data.internet.mh` rather than + tuples, and :class:`.NTPTimestamp` keeps no alias at its 1.4.x home in + :mod:`pcapkit.protocols.internet.mh`. As Mappings they no longer index, unpack, + compare or hash as a tuple does; ``.seconds`` and ``.fraction`` still work, and the + dumpers write both as keys (:issue:`1886`, :pr:`1887`). Fixed ~~~~~ @@ -1367,34 +1417,36 @@ Fixed registries, so a lookup miss inserted the key into class-level state shared by every later instance, after which a legitimate ``register_*`` call warned that the code was already registered. Every read now goes through a lookup that does - not grow the table, and ``IPv4.__option__`` and ``HIP.__parameter__`` became + not grow the table, and :attr:`.IPv4.__option__` and :attr:`.HIP.__parameter__` became inspectable class attributes rather than names assembled at call time (:pr:`426`, :pr:`428`, :issue:`429`, :pr:`434`). What breaks: a tuple-registered handler pair written to the documented ``OptionParser``/``OptionConstructor`` signature now works where it could never be called, and a pair written with an explicit leading ``self`` -- the only shape that used to work -- now does not. -* the identical defect one layer up, in the schema layer's ``EnumSchema.registry``: +* the identical defect one layer up, in the schema layer's :attr:`.EnumSchema.registry`: ``Option.registry[code]`` for an unregistered ``code`` inserted the default schema under that code, so a single lookup made an unassigned TCP option number such as ``156`` read back as registered for the rest of the process. - ``EnumSchema.__enum__`` is now built (or, when a subclass seeds it manually, as + :attr:`.EnumSchema.__enum__` is now built (or, when a subclass seeds it manually, as ``PCAPNG.Option``'s namespaced mapping and ``TCP.MPTCP``'s plain one do) as a retention-safe mapping that still returns the registered default on a miss but stops recording it; ``.registry`` returns the same object as before, so nothing holding a reference is affected (:issue:`555`). -* on Python 3.10 and older, no ``Schema`` subclass got its own ``_abc_impl``: all +* on Python 3.10 and older, no :class:`.Schema` subclass got its own ``_abc_impl``: all fell through to ``collections.abc.Mapping``'s, so a single ``isinstance`` or ``issubclass`` answer poisoned every later question about that class for the - rest of the process. A terminating PCAP-NG ``EndRecord`` tested ``True`` as an - ``IPv4Record`` (:issue:`439`). + rest of the process. A terminating PCAP-NG + :class:`~pcapkit.protocols.schema.misc.pcapng.EndRecord` tested ``True`` as an + :class:`~pcapkit.protocols.schema.misc.pcapng.IPv4Record` (:issue:`439`). * construction, broken in several places at once: the generated typed ``__init__`` was never installed, so ``__post_init__`` did not run and a schema built from a subset of its fields could not be packed -- ``UDP(srcport=53, dstport=5353)`` now packs (:pr:`430`); IPv6 and Mobility Header option padding was wrong, leaving - construction wholly broken (:pr:`398`); ``HTTP.make`` called the versioned - ``make`` unbound, so every real call raised ``TypeError`` (:issue:`452`, - :pr:`462`); and ``IPv4._make_data`` returned the fragment offset in octets where - the wire wants 8-octet units, and read ``data.options`` on a packet that has none + construction wholly broken (:pr:`398`); + :meth:`HTTP.make ` called the versioned + ``make`` unbound, so every real call raised ``TypeError`` (:issue:`452`, :pr:`462`); + and :meth:`.IPv4._make_data` returned the fragment offset in octets where the wire + wants 8-octet units, and read ``data.options`` on a packet that has none (:issue:`494`, :pr:`499`). * a truncated or under-declared area no longer parses "successfully", and no longer wedges the process. The option and list loops could spin forever with no @@ -1403,8 +1455,8 @@ Fixed least one octet, and the error names the option, the offset and the octets remaining (:issue:`431`, :pr:`432`). Separately, wire-derived lengths in ``ipv6_opts``, CALIPSO, MPL, REG_INFO and HIP list callbacks - underflowed below zero, which ``ListField``'s own ``while length > 0`` turned into a - silent empty list; they are floored and raise instead (:pr:`449`, :pr:`456`, + underflowed below zero, which :class:`.ListField`'s own ``while length > 0`` turned + into a silent empty list; they are floored and raise instead (:pr:`449`, :pr:`456`, :pr:`460`, :issue:`463`). * field widths and units, each measured against the specification: HIP's ``TRANSPORT_FORMAT_LIST``, ``NAT_TRAVERSAL_MODE`` and ``ESP_TRANSFORM`` list @@ -1418,66 +1470,72 @@ Fixed * the remaining call sites where a ``_make_*`` method converts an address itself, ahead of the schema, so the same ``bool``-as-``int`` mistake reached them, unguarded by the schema-layer fix above: ``ARP._make_proto_resolve``, - ``IPv6_Route._make_data_type_rpl`` (which also derived its RPL compression + :meth:`.IPv6_Route._make_data_type_rpl` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the ``dst`` - parameter of ``IPv6_Route.make``, and the latent ``OSPF._make_id_numbers``. Each + parameter of :meth:`.IPv6_Route.make`, and the latent ``OSPF._make_id_numbers``. Each now goes through ``parse_ip_address``, the helper :pr:`539` and :issue:`552` route their own sites through, completing the programme :pr:`481` and - :issue:`508` started. Pinning ``version=6`` on the ``IPv6_Route`` sites is a - further, smaller behaviour change: both converted a plain integer through the - family-inferring ``ipaddress.ip_address``, so ``ip=[258]`` packed a 4-octet IPv4 - address (``0.0.1.2``) inside an IPv6-only header; it now packs the 16-octet - ``::102`` (:issue:`540`). + :issue:`508` started. Pinning ``version=6`` on the + :class:`~pcapkit.protocols.internet.ipv6_route.IPv6_Route` sites is a further, smaller + behaviour change: both converted a plain integer through the family-inferring + ``ipaddress.ip_address``, so ``ip=[258]`` packed a 4-octet IPv4 address (``0.0.1.2``) + inside an IPv6-only header; it now packs the 16-octet ``::102`` (:issue:`540`). * PCAP and PCAP-NG output and parsing: ``bytes(frame)`` returned the *next* frame's octets, ``files=True`` wrote names like ``Frame 1..json``, and a PCAP-NG ``timestamp_epoch`` was shifted by the reading host's timezone (:pr:`403`); further parser defects (:issue:`341`--:issue:`347`, :pr:`371`) and, on the write path, more - block-parsing ones (:pr:`388`); ``BitField`` packed every named + block-parsing ones (:pr:`388`); :class:`.BitField` packed every named bit as set (:issue:`359`, :pr:`374`); the extension-header walk failed to advance past the last IPv6 extension header (:issue:`348`, :pr:`373`); and IPv6 fragment offsets went unscaled, with reassembly keyed on the flow label -- optional, and routinely zero, so distinct datagrams collapsed together -- rather than on the fragment identification (:pr:`389`). -* ``TCP._make_mptcp_addaddr`` could not build an ``ADD_ADDR`` option end to end: its - ``kind=``/``length=`` arguments were rejected with ``UnknownFieldWarning`` and +* :meth:`.TCP._make_mptcp_addaddr` could not build an ``ADD_ADDR`` option end to end: + its ``kind=``/``length=`` arguments were rejected with :exc:`.UnknownFieldWarning` and dropped, and ``.pack()`` then raised ``KeyError: 'length'`` from ``port``'s own - condition, ``pkt['length'] in (10, 22)``. The cause was one layer up: ``MPTCP``, the - base of every Multipath TCP subtype schema, declared ``kind`` and ``length`` only - under ``typing.TYPE_CHECKING`` rather than as real fields, unlike ``Option``. That - dropped ``kind=``/``length=`` for every ``_make_mptcp_*`` constructor, so ``MPTCP`` - now declares both for real (:issue:`541`). The missing fields broke parsing too: a - subtype schema's leading field read the ``kind`` octet itself, an off-by-two in - field alignment rather than a wire-format change (a correct sender's octets were - always right). Spec-correct ``ADD_ADDR`` and ``MP_PRIO`` options failed with - ``FieldError: TCP: [OptNo 30] 3 invalid IP version`` and ``KeyError: 'length'``; - both parse now. + condition, ``pkt['length'] in (10, 22)``. The cause was one layer up: + :class:`~pcapkit.protocols.schema.transport.tcp.MPTCP`, the base of every Multipath + TCP subtype schema, declared ``kind`` and ``length`` only under + ``typing.TYPE_CHECKING`` rather than as real fields, unlike + :class:`~pcapkit.protocols.schema.transport.tcp.Option`. That dropped + ``kind=``/``length=`` for every ``_make_mptcp_*`` constructor, so + :class:`~pcapkit.protocols.schema.transport.tcp.MPTCP` now declares both for real + (:issue:`541`). The missing fields broke parsing too: a subtype schema's leading field + read the ``kind`` octet itself, an off-by-two in field alignment rather than a + wire-format change (a correct sender's octets were always right). Spec-correct + ``ADD_ADDR`` and ``MP_PRIO`` options failed with + ``FieldError: TCP: [OptNo 30] 3 invalid IP version`` and ``KeyError: 'length'``; both + parse now. * dropped-keyword/wrong-cast defects flagged in review during this release and filed only as :issue:`556`: HIP's ``_make_param_encrypted`` passed ``cipher=`` to a schema with no such field, so an AES-cipher ``ENCRYPTED`` parameter built through - ``make`` packed without its IV; and IPv6-Route's ``RPL.post_process``, which runs - on every ``Schema.pack``, assumed ``self.addresses`` was still the concatenated + ``make`` packed without its IV; and IPv6-Route's :meth:`.RPL.post_process`, which runs + on every :meth:`.Schema.pack`, assumed ``self.addresses`` was still the concatenated ``bytes`` a parse leaves, and raised slicing the ``list[bytes]`` a ``make``-built multi-address header holds (:issue:`556`). * further defects :issue:`541` exposed by letting construction reach code that had - never run. ``MPTCP.subtype`` was still ``typing.TYPE_CHECKING``-only, so - ``TCP(options=[(Enum_Option.Multipath_TCP, ...)])`` raised - ``AttributeError: ... has no attribute 'subtype'`` for every subtype but + never run. + :attr:`MPTCP.subtype ` was still + ``typing.TYPE_CHECKING``-only, so ``TCP(options=[(Enum_Option.Multipath_TCP, ...)])`` + raised ``AttributeError: ... has no attribute 'subtype'`` for every subtype but ``MP_JOIN``: the convenience constructor reads the schema straight back through - ``_read_mptcp_*`` with no byte round trip, so ``_MPTCP.post_process``, the only code - that set ``subtype``, never ran. Fixed on the construction path - (``TCP._make_mode_mp``) rather than by making ``subtype`` a real field too, the way - ``kind``/``length`` became in :issue:`541`: ``subtype`` is already packed as 4 bits of - each subtype's own ``test`` bitfield, and a second field for the same bits would - double-encode them or need a "derive, don't pack" field kind this library lacks + ``_read_mptcp_*`` with no byte round trip, so :meth:`._MPTCP.post_process`, the only + code that set ``subtype``, never ran. Fixed on the construction path + (:meth:`.TCP._make_mode_mp`) rather than by making ``subtype`` a real field too, the + way ``kind``/``length`` became in :issue:`541`: ``subtype`` is already packed as 4 + bits of each subtype's own ``test`` bitfield, and a second field for the same bits + would double-encode them or need a "derive, don't pack" field kind this library lacks (:issue:`566`). Separately, ``_make_mptcp_capable`` wrote ``length=20 if rkey is None else 32`` where :rfc:`8684` section 3.1 gives 12 and 20, - and ``MPTCPCapable.rkey``'s condition (``pkt['length'] != 32``) dropped the - receiver's key for exactly the length the maker used for "key present": a key-absent - MP_CAPABLE could not be built, and a key-present one lost its key on the wire. Both, - and the guard in ``_read_mptcp_capable``, now agree on 12/20. **This changes - MP_CAPABLE's packed output**: a 20-octet key-present option built or parsed under - the old code becomes 12 octets with no key, or 20 octets with the key present, - depending on which the caller meant (:issue:`567`). + and + :attr:`MPTCPCapable.rkey `'s + condition (``pkt['length'] != 32``) dropped the receiver's key for exactly the length + the maker used for "key present": a key-absent MP_CAPABLE could not be built, and a + key-present one lost its key on the wire. Both, and the guard in + ``_read_mptcp_capable``, now agree on 12/20. **This changes MP_CAPABLE's packed + output**: a 20-octet key-present option built or parsed under the old code becomes 12 + octets with no key, or 20 octets with the key present, depending on which the caller + meant (:issue:`567`). * TCP's MP_CAPABLE now parses and builds all five lengths of :rfc:`8684` section 3.1 -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where ``_read_mptcp_capable`` raised @@ -1487,10 +1545,11 @@ Fixed (2 octets, ``length > 22``) are new. ``_make_mptcp_capable`` takes ``dl_len`` and ``checksum`` too, where it used to drop them silently through ``**kwargs``, and picks the length from how many of ``skey``, ``rkey``, ``dl_len`` and ``checksum`` are not - ``None``. A field given without the one before it raises ``ProtocolError``. The data - model ``MPTCPCapable`` gains ``dl_len`` and ``checksum``, and ``skey`` becomes - ``Optional[int]``: code that constructs it directly must now pass both new fields, - and ``to_dict()`` carries the new keys (:issue:`1042`). + ``None``. A field given without the one before it raises :exc:`.ProtocolError`. The + data model :class:`~pcapkit.protocols.data.transport.tcp.MPTCPCapable` gains + ``dl_len`` and ``checksum``, and ``skey`` becomes ``Optional[int]``: code that + constructs it directly must now pass both new fields, and ``to_dict()`` carries the + new keys (:issue:`1042`). * IPv6-Route's RPL routing data was five octets wide at the front where :rfc:`6554` section 3 gives four, with further defects behind it. ``CmprI``, ``CmprE`` and ``Pad`` are each 4 bits sharing a 32-bit word with a 20-bit ``Reserved``, but the @@ -1505,7 +1564,7 @@ Fixed :pr:`489` for want of a working RPL round trip to validate against. It is replaced by section 4.2's address-count arithmetic, ``n = (((Hdr Ext Len * 8) - Pad - (16 - CmprE)) / (16 - CmprI)) + 1``, which the - reader now requires to be non-negative and whole. ``RPL.post_process`` subtracted + reader now requires to be non-negative and whole. :meth:`.RPL.post_process` subtracted ``pad_len`` a second time from a buffer whose length callback had already taken it off, losing one address per ``16 - CmprI`` octets of padding, and set ``ip`` only when it had parsed octets, so ``IPv6_Route(type=..., data={'ip': [...]})`` raised a @@ -1517,37 +1576,38 @@ Fixed ``16 - CmprI`` zero). ``ipv6-route-type/RPL_Source_Route_Header`` round-trips and its ``EXPECTED_FAILURES`` entry is deleted; none of this has been checked against a real RPL capture (:issue:`564`). -* ``L2TPv2`` parsed any version nibble, so an L2TPv3 datagram was reported as v2 +* :class:`.L2TPv2` parsed any version nibble, so an L2TPv3 datagram was reported as v2 with a tunnel and session ID read out of v3's Control Connection ID. :rfc:`2661` - §3.1 fixes ``Ver`` at 2 and reserves 1 for L2F, and ``L2TPv2.version`` documented + §3.1 fixes ``Ver`` at 2 and reserves 1 for L2F, and :attr:`.L2TPv2.version` documented that any other value "is a different protocol reached through a different class", but nothing enforced it, so the hard-coded ``Literal[2]`` property and ``info.version`` answered 2 and 3 on the same octets. ``read`` now raises - ``ProtocolError``, degrading the payload to ``Raw`` through the existing - ``beholder`` path with the reason recorded. **This affects real captures**: - :rfc:`3931` §4.1.2 puts L2TPv3 on port 1701 too, the port ``UDP.__proto__`` - binds, so ``Ethernet:IPv4:UDP:L2TPv2:Raw`` with invented field values becomes + :exc:`.ProtocolError`, degrading the payload to + :class:`~pcapkit.protocols.misc.raw.Raw` through the existing ``beholder`` path with + the reason recorded. **This affects real captures**: :rfc:`3931` §4.1.2 puts L2TPv3 on + port 1701 too, the port :attr:`.UDP.__proto__` binds, so + ``Ethernet:IPv4:UDP:L2TPv2:Raw`` with invented field values becomes ``Ethernet:IPv4:UDP:Raw`` with the octets preserved. - This resolves :issue:`548`, which reported ``TransType.L2TP`` (115) as registered - nowhere and proposed binding ``L2TPv2`` there. That binding is wrong rather than + This resolves :issue:`548`, which reported :attr:`.TransType.L2TP` (115) as registered + nowhere and proposed binding :class:`.L2TPv2` there. That binding is wrong rather than awkward: :rfc:`3931` §4.1.1 gives 115 to *L2TPv3 over IP*, whose session header carries **no version nibble at all**, so a v2 parser cannot detect that the datagram is not its own. 115 is a missing *class*, not a missing registration, and stays unbound until an ``L2TPv3`` class exists; no dissector was invented to fill it. - The reasoning is recorded in ``pcapkit.protocols.link.l2tp``, and - ``register_protocol_code``'s worked example, which named ``L2TPv2`` at 115, names - ``L2TPv3`` instead (:issue:`548`). + The reasoning is recorded in :mod:`pcapkit.protocols.link.l2tp`, and + ``register_protocol_code``'s worked example, which named :class:`.L2TPv2` at 115, + names ``L2TPv3`` instead (:issue:`548`). * building any ``MP_JOIN`` option raised - ``AttributeError: 'TCP' object has no attribute '_flags'``. ``TCP.make`` constructed - the options *before* assigning the ``self._flags`` that the option makers read, and - ``_make_mptcp_join`` branches on it to choose among the three layouts of :rfc:`8684` - section 3.2 (figure 5, SYN, 12 octets; figure 6, SYN/ACK, 16; figure 7, ACK, 24). - The parse path was never affected, since ``read`` assigns the flags first. Fixed by - hoisting the flag resolution above the ``_make_tcp_options`` call, leaving only the - data-offset computation, which needs the options' total length, after it. **Not** - fixed by a zero default for ``_flags``, which would have been worse: - ``Protocol.pack`` is public and calls ``make``, so an instance that had parsed a + ``AttributeError: 'TCP' object has no attribute '_flags'``. :meth:`.TCP.make` + constructed the options *before* assigning the ``self._flags`` that the option makers + read, and ``_make_mptcp_join`` branches on it to choose among the three layouts of + :rfc:`8684` section 3.2 (figure 5, SYN, 12 octets; figure 6, SYN/ACK, 16; figure 7, + ACK, 24). The parse path was never affected, since ``read`` assigns the flags first. + Fixed by hoisting the flag resolution above the ``_make_tcp_options`` call, leaving + only the data-offset computation, which needs the options' total length, after it. + **Not** fixed by a zero default for ``_flags``, which would have been worse: + :meth:`.Protocol.pack` is public and calls ``make``, so an instance that had parsed a segment had the attribute set and silently built the option for the segment it had *read*. A parsed MP_JOIN-SYN instance asked to pack an MP_JOIN-ACK segment emitted an ACK header carrying figure 5's 12-octet SYN option, with the caller's 20-octet @@ -1556,28 +1616,29 @@ Fixed nor ACK: the accumulator was seeded with ``cast('Enum_Flags', 0)``, and ``typing.cast`` being a runtime no-op, ``self._flags`` stayed a plain ``int`` and the first membership test raised ``TypeError`` rather than the documented - ``ProtocolError``. It is now seeded with ``Enum_Flags(0)``, a real flagless member + :exc:`.ProtocolError`. It is now seeded with ``Enum_Flags(0)``, a real flagless member that equals ``0`` and ORs identically. The read path's seed is unchanged: a flagless MP_JOIN is rejected by ``mptcp_data_selector`` before ``_read_mptcp_join`` runs (:issue:`587`). -* the ILNP Nonce option builders in ``HOPOPT`` and ``IPv6_Opts`` sized the option - with ``math.ceil(nonce.bit_length() // 8)``, floor division dressed as a ceiling - (``//`` floors and ``math.ceil`` of an ``int`` is a no-op). The nonce is packed by - a ``NumberField`` whose width *is* the declared ``len``, so an under-declared - length silently truncated the nonce on the wire. Every nonce whose bit length was - not a multiple of eight was affected, and **small values were the worst case - rather than boundary values**: any nonce below 256 was declared as *zero* octets - and dropped (``nonce=9`` packed to ``b'\x8b\x00'`` and parsed back as ``0``). Fixed to - ``max(1, math.ceil(nonce.bit_length() / 8))``, the form used at the other - sizing sites in ``hip.py`` and ``mh.py``. The one-octet floor is the ``mh.py`` - convention and is load-bearing because ``nonce`` defaults to ``0``, whose bit - length is ``0``: without it the default builds an option with no Nonce Value - field at all, collapsing "the nonce is 0" into "there is no nonce" when - :rfc:`6744` gives the option that field. The read path takes its width from the - ``len`` octet on the wire and was never affected (:issue:`601`). +* the ILNP Nonce option builders in :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` + and :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` sized the option with + ``math.ceil(nonce.bit_length() // 8)``, floor division dressed as a ceiling (``//`` + floors and ``math.ceil`` of an ``int`` is a no-op). The nonce is packed by a + :class:`.NumberField` whose width *is* the declared ``len``, so an under-declared + length silently truncated the nonce on the wire. Every nonce whose bit length was not + a multiple of eight was affected, and **small values were the worst case rather than + boundary values**: any nonce below 256 was declared as *zero* octets and dropped + (``nonce=9`` packed to ``b'\x8b\x00'`` and parsed back as ``0``). Fixed to + ``max(1, math.ceil(nonce.bit_length() / 8))``, the form used at the other sizing sites + in ``hip.py`` and ``mh.py``. The one-octet floor is the ``mh.py`` convention and is + load-bearing because ``nonce`` defaults to ``0``, whose bit length is ``0``: without + it the default builds an option with no Nonce Value field at all, collapsing "the + nonce is 0" into "there is no nonce" when :rfc:`6744` gives the option that field. The + read path takes its width from the ``len`` octet on the wire and was never affected + (:issue:`601`). * ``httpv1``'s ``_RE_METHOD`` was unanchored and ``re.match`` anchors only at the start, so it prefix-matched, and the request-line reader passed the whole - ``para1`` to ``Method.get`` rather than the captured ``method`` group. So + ``para1`` to :meth:`.Method.get` rather than the captured ``method`` group. So ``b'Get'`` matched on the single character ``G``, satisfied the guard that decides a start-line is a request, and handed the mixed-case token to a lookup that raised on it. Fixing either half alone still gives a wrong answer (normalising the lookup @@ -1591,9 +1652,9 @@ Fixed auditing ``_RE_METHOD``'s siblings, and escaped as the wrong exception type. The pattern is only a guard (the value comes from ``int(para2)`` on the raw token), so a prefix match let a malformed status past the guard and out of ``int()`` uncaught, - where ``_read_http_header`` documents ``ProtocolError``: ``200x`` raised + where ``_read_http_header`` documents :exc:`.ProtocolError`: ``200x`` raised ``ValueError: invalid literal for int()`` and ``2000`` raised - ``ValueError: 2000 is not a valid StatusCode``; both are now ``ProtocolError``. + ``ValueError: 2000 is not a valid StatusCode``; both are now :exc:`.ProtocolError`. :rfc:`9112#section-4` gives ``status-code = 3DIGIT``, exactly three, so the anchor is what the grammar said (the production lives in HTTP/1.1 because ``status-code`` is part of its ``status-line``; :rfc:`9110#section-15` covers code semantics and the @@ -1610,31 +1671,33 @@ Fixed large allocation is attempted, and all three end in the ordinary handled parse failure (:issue:`604`). * reading a big-endian classic PCAP byte-swapped every record header field, and then - crashed. ``Frame.unpack`` seeded the file's byte order under the key ``bytesorder`` - where the schema's ``byteorder_callback`` reads ``byteorder``, so the lookup never - found it and fell back to ``sys.byteorder``, the host's order rather than the - file's. On a little-endian host reading a little-endian capture that is right by + crashed. :meth:`.Frame.unpack` seeded the file's byte order under the key + ``bytesorder`` where the schema's ``byteorder_callback`` reads ``byteorder``, so the + lookup never found it and fell back to ``sys.byteorder``, the host's order rather than + the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of ``big_endian.pcap`` read ``incl_len=1241513984`` for a record whose real value is ``74``. ``incl_len`` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising ``ValueError: read length must be non-negative or -1`` -- the reported crash, and - the *second* symptom. The sibling ``Frame.pack`` spelled the + the *second* symptom. The sibling :meth:`.Frame.pack` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; ``byteorder_callback`` now records that it defines the key and why the fallback hides a typo (:issue:`605`). * documentation. ``mptcp_dss_ack_selector``'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to - ``pcapkit.corekit.fields.numbers``, which is where :pr:`598` then fixed it; the + :mod:`pcapkit.corekit.fields.numbers`, which is where :pr:`598` then fixed it; the same paragraph sat in ``test_tcp_mptcp_length_arithmetic_unit.py``'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public - ``TCP()`` constructor at all", true only until :issue:`587`. A callable-length - ``NumberField`` packs and unpacks both DSS widths now, and wire *absence* was never - the obstacle: ``MPTCPDSS.ssn``, ``dl_len`` and ``checksum`` have always been - ``ConditionalField`` on the sibling ``M`` flag. The ``SwitchField`` form is kept - for the narrower reason the note now gives: ``ConditionalField``'s ``length`` - forwards to the wrapped field without consulting the condition, whereas a - ``SwitchField`` always resolves to a concrete field (:issue:`603`). + :class:`TCP() ` constructor at all", true only + until :issue:`587`. A callable-length :class:`.NumberField` packs and unpacks both DSS + widths now, and wire *absence* was never the obstacle: + :attr:`MPTCPDSS.ssn `, ``dl_len`` + and ``checksum`` have always been :class:`.ConditionalField` on the sibling ``M`` + flag. The :class:`.SwitchField` form is kept for the narrower reason the note now + gives: :class:`.ConditionalField`'s ``length`` forwards to the wrapped field without + consulting the condition, whereas a :class:`.SwitchField` always resolves to a + concrete field (:issue:`603`). * the assertion :issue:`604` left pinning the old padding side, which :pr:`621` turned red on ``main``. ``TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length`` @@ -1654,9 +1717,9 @@ Fixed ``RHASH_len`` is a whole number of octets. For an arbitrary ``bit_length()`` the builder emitted an *odd* contents width, which its own reader refused: ``_read_param_solution``'s ``(schema.len - 4) % 2`` guard exists because - ``SolutionParameter`` splits that width into two equal halves. ``random=0x1`` with - ``solution=0xfff`` declared ``len=7`` and raised - ``ProtocolError: HIPv2: [ParamNo 321] invalid format`` on input the library had + :class:`~pcapkit.protocols.schema.internet.hip.SolutionParameter` splits that width + into two equal halves. ``random=0x1`` with ``solution=0xfff`` declared ``len=7`` and + raised ``ProtocolError: HIPv2: [ParamNo 321] invalid format`` on input the library had produced. The undersized length was worse: both fields take their width from that ``len``, so ``solution=0xfff`` was packed into one octet and came back as ``0xff``, silently. Fixed to ``4 + 2 * math.ceil(max(...) / 8)``, the form the sibling @@ -1664,38 +1727,40 @@ Fixed rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value (:issue:`608`). -* ``TCP.read`` seeded its connection-flag accumulator with ``cast('Enum_Flags', 0)``. - ``typing.cast`` is a runtime no-op, so the accumulator began as the plain ``int`` - ``0``, and only the ``|=`` promoting it to a ``pcapkit.const.tcp.flags.Flags`` - member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) - left ``self._flags`` an ``int``, so a membership test raised +* :meth:`.TCP.read` seeded its connection-flag accumulator with + ``cast('Enum_Flags', 0)``. ``typing.cast`` is a runtime no-op, so the accumulator + began as the plain ``int`` ``0``, and only the ``|=`` promoting it to a + :class:`pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an + all-zero flags octet (an nmap NULL scan) left ``self._flags`` an ``int``, so a + membership test raised ``TypeError: argument of type 'int' is not a container or iterable``, and - ``TCP.connection`` returned an ``int`` where it annotates ``Flags``. Any flag masked + :attr:`TCP.connection ` returned an + ``int`` where it annotates :class:`~pcapkit.const.tcp.flags.Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is ``Flags(0)``, as :pr:`597` already did for the sibling accumulator in ``make``; - ``Flags`` declares no ``_missing_``, so ``Flags(0)`` is an ordinary - ``aenum.IntFlag`` pseudo-member and does not meet the ``RecursionError`` the same - construction hits on the ``pcapkit.const.mh`` flag enumerations (:issue:`623`). - The MP_JOIN path was latent: ``mptcp_data_selector`` rejects a flagless MP_JOIN - before :rfc:`8684` section 3.2's three layouts are chosen. A dump-format change is - observable: a flagless segment's ``connection`` renders as a string where it - rendered as the number ``0``, so it no longer switches JSON type with the flags. That - string read ``Flags::None [0]`` until the ``pcapkit.dumpkit`` fix above - (:issue:`616`). -* **a breaking change to a public attribute.** ``Frame.len`` and ``Frame.cap_len`` - were filled from *opposite* wire fields depending on the container format, so - ``frame.len`` meant the captured length out of a ``.pcap`` and the on-wire length - out of a ``.pcapng``. The PCAP reader moved to match PCAP-NG: ``len`` is the on-wire - length (the record header's ``orig_len``) and ``cap_len`` the octets stored - (``incl_len``). **Code reading** ``frame.len`` **or** ``frame.cap_len`` **from a** ``.pcap`` - **gets the other field's value than before**, and for a truncated frame that is a - different number rather than a relabelling. Seniority did not settle it: the PCAP - spelling is the *older* (``c43892af``, 2022-01-11) and PCAP-NG's arrived in - ``25f216f4`` (2023-04-27), both internally consistent. The *names* settle it, being + :class:`~pcapkit.const.tcp.flags.Flags` declares no ``_missing_``, so ``Flags(0)`` is + an ordinary ``aenum.IntFlag`` pseudo-member and does not meet the ``RecursionError`` + the same construction hits on the :mod:`pcapkit.const.mh` flag enumerations + (:issue:`623`). The MP_JOIN path was latent: ``mptcp_data_selector`` rejects a + flagless MP_JOIN before :rfc:`8684` section 3.2's three layouts are chosen. A + dump-format change is observable: a flagless segment's ``connection`` renders as a + string where it rendered as the number ``0``, so it no longer switches JSON type with + the flags. That string read ``Flags::None [0]`` until the :mod:`pcapkit.dumpkit` fix + above (:issue:`616`). +* **a breaking change to a public attribute.** :attr:`.Frame.len` and + :attr:`.Frame.cap_len` were filled from *opposite* wire fields depending on the + container format, so ``frame.len`` meant the captured length out of a ``.pcap`` and + the on-wire length out of a ``.pcapng``. The PCAP reader moved to match PCAP-NG: + ``len`` is the on-wire length (the record header's ``orig_len``) and ``cap_len`` the + octets stored (``incl_len``). **Code reading** ``frame.len`` **or** ``frame.cap_len`` + **from a** ``.pcap`` **gets the other field's value than before**, and for a truncated + frame that is a different number rather than a relabelling. Seniority did not settle + it: the PCAP spelling is the *older* (``c43892af``, 2022-01-11) and PCAP-NG's arrived + in ``25f216f4`` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: ``frame.len`` is "Frame length on the wire" and ``frame.cap_len`` "Frame length stored into the capture file", and ``frame.len_lt_caplen`` expert info flags ``frame_len < cap_len`` as malformed, which it could not be if ``len`` were - the smaller, captured one. ``Frame.read`` hands + the smaller, captured one. :meth:`.Frame.read` hands ``_decode_next_layer`` the octets *present*, now ``frame.cap_len``; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of :issue:`554`, :issue:`573` and :issue:`594` on every snapped frame. The dumpers are @@ -1703,16 +1768,16 @@ Fixed existed until :pr:`614` (:issue:`618`). * the HIP ``PUZZLE`` and ``SOLUTION`` builders derived wire-format values from the payload value instead of the data model, and each was wrong. **This changes the** - ``PuzzleParameter`` **and** ``SolutionParameter`` **data models and the octets both - parameters emit.** The width of ``Random #I`` and ``Puzzle solution #J`` came from - ``int.bit_length()`` alone, so every leading zero octet was dropped on - re-serialisation: a ``SOLUTION`` read with - ``Length = 20`` rebuilt as ``Length = 6``, a ``PUZZLE`` with ``Length = 12`` as - ``Length = 5``, silently. That width is ``RHASH_len / 8`` octets - [:rfc:`7401#section-5.2.4`, :rfc:`7401#section-5.2.5`], a property of the - Responder's HIT Suite, so both data models now carry it as ``rhash_len`` and both - builders prefer it; a new ``rhash_len=`` keyword states it for a build from - scratch, which under HIPv2 is the only place it can come from, since + :class:`~pcapkit.protocols.data.internet.hip.PuzzleParameter` **and** + :class:`~pcapkit.protocols.data.internet.hip.SolutionParameter` **data models and the + octets both parameters emit.** The width of ``Random #I`` and ``Puzzle solution #J`` + came from ``int.bit_length()`` alone, so every leading zero octet was dropped on + re-serialisation: a ``SOLUTION`` read with ``Length = 20`` rebuilt as ``Length = 6``, + a ``PUZZLE`` with ``Length = 12`` as ``Length = 5``, silently. That width is + ``RHASH_len / 8`` octets [:rfc:`7401#section-5.2.4`, :rfc:`7401#section-5.2.5`], a + property of the Responder's HIT Suite, so both data models now carry it as + ``rhash_len`` and both builders prefer it; a new ``rhash_len=`` keyword states it for + a build from scratch, which under HIPv2 is the only place it can come from, since ``RSA,DSA/SHA-256`` is the REQUIRED HIT Suite [:rfc:`7401#section-5.2.10`] and makes the field 32 octets rather than 8. On the mandatory path about one parameter in 256 has a zero top octet and lost it (:issue:`653`). ``SOLUTION``'s second @@ -1723,7 +1788,7 @@ Fixed ``0x2b`` into a field the RFC requires to be zero, and could not write that zero: a conformant ``SOLUTION`` with ``Reserved`` ``0x00`` parsed to ``timedelta(0)`` and could not be re-serialised, escaping a bare ``ValueError`` from ``math.log2(0.0)`` - that was not a ``BaseError``. The field is renamed ``reserved``, defaults to zero, + that was not a :exc:`.BaseError`. The field is renamed ``reserved``, defaults to zero, and is carried verbatim across a round trip (:issue:`654`). And neither builder read its own ``version`` keyword, so ``version=1`` and ``version=2`` computed identical lengths, where :rfc:`5201#section-5.2.4` and :rfc:`5201#section-5.2.5` @@ -1732,15 +1797,16 @@ Fixed anything narrower, a parameter this library's reader rejects -- the shape of :issue:`608`, in the ``PUZZLE`` builder that :pr:`629` never opened (:issue:`655`). The remaining ``math.log2`` sites, in ``PUZZLE`` where a lifetime is real, - raise ``ProtocolError`` (``(BaseError, ValueError)``, so a caller written around + raise :exc:`.ProtocolError` (``(BaseError, ValueError)``, so a caller written around the bare exception still catches it). Migration: ``SolutionParameter.lifetime`` is - now ``reserved`` and an ``int`` rather than a ``timedelta``; ``PuzzleParameter`` and - ``SolutionParameter`` gained a required ``rhash_len``; and ``_make_param_solution`` - no longer takes ``lifetime=``. Each was reachable end to end only once - :issue:`608` was fixed in :pr:`629`, which removed the parity guard that had failed - these rebuilds loudly; the round trip then stopped raising and started quietly - emitting a different parameter, which is why they were fixed together - (:issue:`653`, :issue:`654`, :issue:`655`). + now ``reserved`` and an ``int`` rather than a ``timedelta``; + :class:`~pcapkit.protocols.data.internet.hip.PuzzleParameter` and + :class:`~pcapkit.protocols.data.internet.hip.SolutionParameter` gained a required + ``rhash_len``; and ``_make_param_solution`` no longer takes ``lifetime=``. Each was + reachable end to end only once :issue:`608` was fixed in :pr:`629`, which removed the + parity guard that had failed these rebuilds loudly; the round trip then stopped + raising and started quietly emitting a different parameter, which is why they were + fixed together (:issue:`653`, :issue:`654`, :issue:`655`). * every HIP parameter pcapkit emitted was ``4 (mod 8)`` octets long, because the padding aligned the *contents* rather than the record. Corrected for **every HIP parameter but** ``LOCATOR_SET``, excluded below. **This changes the octets those @@ -1756,31 +1822,32 @@ Fixed short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. Every site - but ``LOCATOR_SET``'s (the ``PaddingField`` callbacks in + but ``LOCATOR_SET``'s (the :class:`.PaddingField` callbacks in ``pcapkit/protocols/schema/internet/hip.py`` and the record lengths in ``pcapkit/protocols/internet/hip.py``) now goes through one ``parameter_total_len`` and one ``parameter_padding_len`` (:issue:`651`). ``LOCATOR_SET`` kept the old expression because defects there cancelled exactly: its padding callback never - received the parameter's ``len`` (the nested ``Locator`` schemas share a packet - context whose own ``len`` shadowed it, so it saw 4 for an IPv6 locator), and the - parameter's ``len`` was written in 4-octet units where the RFC's ``Length`` is a byte - count; the result, ``4 + 24n + 4``, equalled the RFC total ``24n + 8``. :issue:`679` - later fixed both sites. ``EncryptedParameter``'s ``data`` length callback is fixed in - the same change: it subtracted the sixteen ``iv`` octets but not the four ``reserved`` - ones, which cancelled the padding's four at ``Length % 8`` in ``{0, 5, 6, 7}``, so - correcting the padding alone would have made ``ENCRYPTED`` four octets too long at - all eight residues. ``HIP.make``'s ``len = total_length // 8 + 4`` is not part of - the defect: :rfc:`7401#section-5.1.3` defines Header Length in 8-byte units - excluding the first 8 bytes, so the floor division is exact once each parameter is a - multiple of eight. Migration: a ``SEQ`` parameter's ``length`` is now 8 where it was - 12, and the other parameters move likewise, so code comparing stored output byte for - byte or asserting on ``Data_*Parameter.length`` sees the RFC's values. ``LOCATOR_SET`` - is unchanged in both respects until :issue:`679` (below). - ``examples/generators/options.py``'s ``HIP_COPIES`` stayed at two for - ``R1_COUNTER``'s four-octet ``counter`` where :rfc:`7401#section-5.2.3` requires - eight, which this defect had masked, until :issue:`672` widened ``counter`` and - :issue:`679` fixed ``LOCATOR_SET``'s ``Length`` unit, after which :issue:`689` - dropped ``HIP_COPIES`` to one (:issue:`651`). + received the parameter's ``len`` (the nested + :class:`~pcapkit.protocols.schema.internet.hip.Locator` schemas share a packet context + whose own ``len`` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's + ``len`` was written in 4-octet units where the RFC's ``Length`` is a byte count; the + result, ``4 + 24n + 4``, equalled the RFC total ``24n + 8``. :issue:`679` later fixed + both sites. :class:`~pcapkit.protocols.schema.internet.hip.EncryptedParameter`'s + ``data`` length callback is fixed in the same change: it subtracted the sixteen ``iv`` + octets but not the four ``reserved`` ones, which cancelled the padding's four at + ``Length % 8`` in ``{0, 5, 6, 7}``, so correcting the padding alone would have made + ``ENCRYPTED`` four octets too long at all eight residues. :meth:`.HIP.make`'s + ``len = total_length // 8 + 4`` is not part of the defect: :rfc:`7401#section-5.1.3` + defines Header Length in 8-byte units excluding the first 8 bytes, so the floor + division is exact once each parameter is a multiple of eight. Migration: a ``SEQ`` + parameter's ``length`` is now 8 where it was 12, and the other parameters move + likewise, so code comparing stored output byte for byte or asserting on + ``Data_*Parameter.length`` sees the RFC's values. ``LOCATOR_SET`` is unchanged in both + respects until :issue:`679` (below). ``examples/generators/options.py``'s + ``HIP_COPIES`` stayed at two for ``R1_COUNTER``'s four-octet ``counter`` where + :rfc:`7401#section-5.2.3` requires eight, which this defect had masked, until + :issue:`672` widened ``counter`` and :issue:`679` fixed ``LOCATOR_SET``'s ``Length`` + unit, after which :issue:`689` dropped ``HIP_COPIES`` to one (:issue:`651`). * HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** ``_make_http_data`` was the only ``_make_http_*`` method @@ -1789,9 +1856,9 @@ Fixed saying the stream continues where the capture said it ended [:rfc:`9113#section-6.1`]. ``PADDED`` was re-derived from ``pad_len``, and the sibling builders already restored theirs, so this was a missing read-back - (:issue:`652`). Separately, ``FrameType.post_process`` seeded its flag accumulator - with a bare ``0``, which ``|=`` promotes only as a side effect, so a frame with - **no** bit set left ``__flags__`` a plain ``int`` where the schema and data model + (:issue:`652`). Separately, :meth:`.FrameType.post_process` seeded its flag + accumulator with a bare ``0``, which ``|=`` promotes only as a side effect, so a frame + with **no** bit set left ``__flags__`` a plain ``int`` where the schema and data model declare a ``Flags``, and ``END_STREAM in __value__`` raised ``TypeError: argument of type 'int' is not a container or iterable`` rather than answering ``False``. That is the common case: a ``0x00`` flags octet is every @@ -1800,15 +1867,19 @@ Fixed In the dump, ``__value__`` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, ``"Flags::None [0]"`` where it read ``0``, the same change :issue:`616` made to - TCP's ``connection``. The seed is guarded because ``FrameType.Flags`` declares no - members and, from Python 3.11, a memberless ``enum.Flag`` subclass refuses - ``Flags(0)``, so the frame schemas that inherit it (``UnassignedFrame``, - ``PriorityFrame``, ``RSTStreamFrame``, ``GoawayFrame``, ``WindowUpdateFrame``) keep - the plain ``int``, which nothing observes since each passes ``flags=None``. A DATA - round trip is **still** lossy for an unrelated reason found while measuring: payload - length callbacks are mis-parenthesised, so an unpadded ``DATA``, ``HEADERS`` - or ``PUSH_PROMISE`` frame parses with its whole payload dropped (:issue:`668`, - below) (:issue:`652`, :issue:`650`). + TCP's ``connection``. The seed is guarded because :class:`.FrameType.Flags` declares + no members and, from Python 3.11, a memberless ``enum.Flag`` subclass refuses + ``Flags(0)``, so the frame schemas that inherit it + (:class:`~pcapkit.protocols.schema.application.httpv2.UnassignedFrame`, + :class:`~pcapkit.protocols.schema.application.httpv2.PriorityFrame`, + :class:`~pcapkit.protocols.schema.application.httpv2.RSTStreamFrame`, + :class:`~pcapkit.protocols.schema.application.httpv2.GoawayFrame`, + :class:`~pcapkit.protocols.schema.application.httpv2.WindowUpdateFrame`) keep the + plain ``int``, which nothing observes since each passes ``flags=None``. A DATA round + trip is **still** lossy for an unrelated reason found while measuring: payload length + callbacks are mis-parenthesised, so an unpadded ``DATA``, ``HEADERS`` or + ``PUSH_PROMISE`` frame parses with its whole payload dropped (:issue:`668`, below) + (:issue:`652`, :issue:`650`). * MPTCP error messages carried a doubled separator, rendering as ``TCP: : [OptNo 30] 1: invalid flags combination``. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty @@ -1843,32 +1914,33 @@ Fixed ``len(packet) == 0`` against a ``captured_len`` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those block types, exactly ``PCAPNG.PACKET_TYPES``, are the whole blast radius. - The octets were read correctly and thrown away: ``PCAPNG.unpack`` extracted them - from the block schema, and ``ProtocolBase.__init__`` overwrote the result with + The octets were read correctly and thrown away: :meth:`.PCAPNG.unpack` extracted them + from the block schema, and :meth:`.ProtocolBase.__init__` overwrote the result with ``self.packet.payload``. The inherited ``packet`` splits a protocol at ``self.length`` octets of header and takes the rest as payload, which holds only for - a header followed by its payload; ``PCAPNG.length`` is the *Block Total Length*, so - the split consumed the whole block as header (``len(frame.packet.header) == 348`` on - a 348-octet block), and a block's trailer (option list, repeated total length) means - no ``length`` could make it right. The fix is at the computation rather than the - injection: ``PCAPNG.packet`` is overridden to take the payload from the block - schema's ``__payload__`` field and the header from the octets ahead of it, summed - out of the schema buffers so the packet types' payload offsets (28, 12, 28) are - not hard-coded. Teaching ``ProtocolBase.__init__`` to skip a protocol that had set - ``packet`` was rejected: it heals ``frame.info.packet`` while ``frame.packet`` still - reports the whole block as header. The override is a plain ``property``, not the - ``cached_property`` it replaces: the inherited one caches because it *reads the - stream*, where this one only walks buffers already filled, and caching it would let - a second ``unpack`` on one instance return the *first* call's octets. This moves the - parse output of every PCAP-NG capture on the success path, so it ships labelled - breaking: ``frame.info.packet`` goes from ``b''`` to hundreds of octets, - ``frame.packet.header`` from the whole block to the pre-payload prefix. It was also - a wire-format defect: ``PCAPIO`` writes ``value.packet`` after each record header, - so a dump declared hundreds of octets per record and delivered none, + a header followed by its payload; + :attr:`PCAPNG.length ` is the *Block + Total Length*, so the split consumed the whole block as header + (``len(frame.packet.header) == 348`` on a 348-octet block), and a block's trailer + (option list, repeated total length) means no ``length`` could make it right. The fix + is at the computation rather than the injection: :attr:`.PCAPNG.packet` is overridden + to take the payload from the block schema's ``__payload__`` field and the header from + the octets ahead of it, summed out of the schema buffers so the packet types' payload + offsets (28, 12, 28) are not hard-coded. Teaching :meth:`.ProtocolBase.__init__` to + skip a protocol that had set ``packet`` was rejected: it heals ``frame.info.packet`` + while ``frame.packet`` still reports the whole block as header. The override is a + plain ``property``, not the ``cached_property`` it replaces: the inherited one caches + because it *reads the stream*, where this one only walks buffers already filled, and + caching it would let a second ``unpack`` on one instance return the *first* call's + octets. This moves the parse output of every PCAP-NG capture on the success path, so + it ships labelled breaking: ``frame.info.packet`` goes from ``b''`` to hundreds of + octets, ``frame.packet.header`` from the whole block to the pre-payload prefix. It was + also a wire-format defect: :class:`.PCAPIO` writes ``value.packet`` after each record + header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by ``incl_len``. It round-trips now. The uncaught ``ValueError`` on an EOF-truncated PCAP-NG (:issue:`678`) is untouched, and - :issue:`685` (below) later removed the stale ``examples/captures/pcapng.txt`` from - the index (:issue:`646`). + :issue:`685` (below) later removed the stale ``examples/captures/pcapng.txt`` from the + index (:issue:`646`). * an unpadded HTTP/2 ``DATA``, ``HEADERS`` or ``PUSH_PROMISE`` frame parsed with its whole payload silently discarded. The payload length callbacks for those frames in ``pcapkit/protocols/schema/application/httpv2.py`` put the @@ -1882,14 +1954,15 @@ Fixed keep their header block fragment. Padded frames are unaffected, since ``(A - B) if T else 0`` and ``A - (B if T else 0)`` are both ``A - B`` for truthy ``T``; the padded arm has no off-by-one (:issue:`668` asked that be checked), - because ``Schema.unpack`` decrements ``packet['__length__']`` by each field's width, - so ``pad_len``'s own octet is already out by the time the payload field runs. The - same lambda also governs **packing**: an unpadded ``DATA`` frame carrying twelve + because :meth:`.Schema.unpack` decrements ``packet['__length__']`` by each field's + width, so ``pad_len``'s own octet is already out by the time the payload field runs. + The same lambda also governs **packing**: an unpadded ``DATA`` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a ``make`` -> parse -> ``make`` cycle of a non-empty unpadded payload now closes byte-for-byte. The only other - conditional of this shape, ``TSOption.remainder`` in - ``pcapkit/protocols/schema/internet/ipv4.py``, is correct: its ``else 0`` is + conditional of this shape, + :attr:`TSOption.remainder ` + in ``pcapkit/protocols/schema/internet/ipv4.py``, is correct: its ``else 0`` is intentional because ``ts_data`` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros :rfc:`9113#section-6.1` tells a sender to use, so failing to subtract it yields a different byte string. Ships @@ -1897,118 +1970,130 @@ Fixed every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct (:issue:`668`). * the schema half of every option-like registrar overwrote silently while the parser - half of the same call warned. ``EnumSchema.register`` was a bare + half of the same call warned. :meth:`.EnumSchema.register` was a bare ``cls.__enum__[code] = schema``, and is the schema half of the public option registrars, so one ``register_ipv4_option`` call replacing a built-in named the parser - it displaced and said nothing about the schema. ``EnumSchema.__init_subclass__`` + it displaced and said nothing about the schema. :meth:`.EnumSchema.__init_subclass__` reaches the same registry without calling ``register``, so a subclass declared with a ``code=`` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's branches are folded into one loop. Presence is a - faithful test only because ``_EnumRegistry.__missing__`` returns a miss without + faithful test only because :meth:`._EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate - registration warn. The code-keyed parser registrars (on ``ProtocolBase``, - ``Link``, ``Internet``, ``Frame``, ``PCAPNG``, ``Transport`` and ``SCTP``) now name - the displaced entry and its replacement, where they reported only that something was - overwritten. Their firing condition is harmonised onto the narrower guard :pr:`681` - gave ``register_protocol`` (:issue:`718`, :pr:`726`) rather than presence alone: - that key is derived from the value it stores, whereas these take a caller-supplied - ``code``. Where a table is pre-seeded with an unresolved ``ModuleDescriptor``, the - guard's ``is`` comparison measures the resolved incoming class against the - descriptor, so re-registering the same class under its own pre-seeded code + registration warn. The code-keyed parser registrars (on :class:`.ProtocolBase`, + :class:`.Link`, :class:`.Internet`, :class:`~pcapkit.protocols.misc.pcap.frame.Frame`, + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`, + :class:`~pcapkit.protocols.transport.transport.Transport` and + :class:`~pcapkit.protocols.transport.sctp.SCTP`) now name the displaced entry and its + replacement, where they reported only that something was overwritten. Their firing + condition is harmonised onto the narrower guard :pr:`681` gave ``register_protocol`` + (:issue:`718`, :pr:`726`) rather than presence alone: that key is derived from the + value it stores, whereas these take a caller-supplied ``code``. Where a table is + pre-seeded with an unresolved :class:`.ModuleDescriptor`, the guard's ``is`` + comparison measures the resolved incoming class against the descriptor, so + re-registering the same class under its own pre-seeded code (``Internet.register(TransType.TCP, TCP)``) still warns, a false positive the - harmonised guard does not close. ``ContextRegistry.register`` already raises on a - duplicate, and the reassembly and ESP registrars append to lists with no key, so - none of them takes a guard. ``import pcapkit`` emits no ``RegistryWarning``: - none of its registry writes (``R1CounterParameter``'s second code, from - :issue:`690`, among them) lands on a key already present (:issue:`692`). -* every EOF-truncated PCAP-NG file raised a bare ``ValueError`` out of ``Extractor``, - losing the whole extraction rather than the one truncated block. Block Total Length - is cross-checked against its trailing copy and never against the file, so a last - block declaring more than the file holds left ``PCAPNG.read`` seeking *past* the - end, and the next block read measured a **negative** remainder that + harmonised guard does not close. :meth:`.ContextRegistry.register` already raises on a + duplicate, and the reassembly and ESP registrars append to lists with no key, so none + of them takes a guard. ``import pcapkit`` emits no :exc:`.RegistryWarning`: none of + its registry writes + (:class:`~pcapkit.protocols.schema.internet.hip.R1CounterParameter`'s second code, + from :issue:`690`, among them) lands on a key already present (:issue:`692`). +* every EOF-truncated PCAP-NG file raised a bare ``ValueError`` out of + :class:`.Extractor`, losing the whole extraction rather than the one truncated block. + Block Total Length is cross-checked against its trailing copy and never against the + file, so a last block declaring more than the file holds left :meth:`.PCAPNG.read` + seeking *past* the end, and the next block read measured a **negative** remainder that ``io.RawIOBase.read`` refused. The seek is now clamped to the octets - ``_read_fileng`` returned, with a ``ProtocolWarning`` naming the overrun, and a tail - under the twelve octets a block needs is the quiet ``StreamEOFError`` that - ``Extractor.record_frames`` already catches (deliberately not a clamp, which would - fabricate a frame out of zero padding). Another route to the same shape is closed - by a new ``nonnegative()`` helper, composed into ``bounded_option`` and + ``_read_fileng`` returned, with a :exc:`.ProtocolWarning` naming the overrun, and a + tail under the twelve octets a block needs is the quiet :exc:`.StreamEOFError` that + :meth:`.Extractor.record_frames` already catches (deliberately not a clamp, which + would fabricate a frame out of zero padding). Another route to the same shape is + closed by a new ``nonnegative()`` helper, composed into ``bounded_option`` and ``bounded_area`` and applied to the unwrapped ``length - N`` spans and the - ``__option_padding__`` fields: ``_TextField.__call__`` builds ``f'{length}s'`` + ``__option_padding__`` fields: :meth:`._TextField.__call__` builds ``f'{length}s'`` unconditionally, so a negative became ``'-8s'`` and ``struct.calcsize`` raised an exception caught nowhere a frame loop looks. Both of :issue:`678`'s failure families are gone; others remain: ``ValueError: N is not a valid BlockType`` (filed as :issue:`701`) and ``MemoryError`` from :pr:`593`'s 32-bit band. Bare exceptions - in ``SystemdJournalExportBlock.post_process``, reachable from valid input, are fixed - with it: the body's NUL alignment padding was read as a field *name* + in :meth:`.SystemdJournalExportBlock.post_process`, reachable from valid input, are + fixed with it: the body's NUL alignment padding was read as a field *name* (``struct.error``; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (``OverflowError``; now clamped and reported); and a non-UTF-8 name, key or value raised ``UnicodeDecodeError`` (now ``errors='replace'`` and reported). The silent loss of every field after a binary one is filed as :issue:`704`. The unclamped non-packet option areas keep the per-block framing assumption :pr:`676`'s note describes. Also here, the last - registrar with no overwrite guard: ``Option.register`` now reports a displaced - option schema as a ``RegistryWarning`` naming every namespace the ``ns='opt'`` + registrar with no overwrite guard: :meth:`.Option.register` now reports a displaced + option schema as a :exc:`.RegistryWarning` naming every namespace the ``ns='opt'`` fan-out displaced something in, tested by membership (subscripting the per-namespace - ``defaultdict`` would insert ``UnknownOption`` for a code nobody registered). The - condition is identity-based like the parser registrars and :pr:`681`'s guard - for ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the call itself - creates is exempt. Flooring the decryption-secrets payloads that read ``__length__`` - *whole* was tried and reverted, since ``Schema.pack`` leaves that key at ``-1`` for - "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any - truncated PCAP-NG now yields the frames before the cut where it raised, the last of - which may carry zero-padded octets, and some short-file depths move - from ``ProtocolError: unknown byteorder magic`` to ``StreamEOFError``. The :pr:`683` - drift :issue:`685` tracks is unchanged (:issue:`678`). + ``defaultdict`` would insert + :class:`~pcapkit.protocols.schema.misc.pcapng.UnknownOption` for a code nobody + registered). The condition is identity-based like the parser registrars and + :pr:`681`'s guard for ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the + call itself creates is exempt. Flooring the decryption-secrets payloads that read + ``__length__`` *whole* was tried and reverted, since :meth:`.Schema.pack` leaves that + key at ``-1`` for "unknown". Ships labelled breaking: well-formed captures are + byte-identical, but any truncated PCAP-NG now yields the frames before the cut where + it raised, the last of which may carry zero-padded octets, and some short-file depths + move from ``ProtocolError: unknown byteorder magic`` to :exc:`.StreamEOFError`. The + :pr:`683` drift :issue:`685` tracks is unchanged (:issue:`678`). * HIP's ``R1_COUNTER``/``R1_Counter`` parameter packed 12 octets where :rfc:`7401#section-5.2.3` requires 16, and ``LOCATOR_SET`` declared its ``Length`` in 4-octet units where :rfc:`7401#section-5.2.1`'s ``Length`` is a byte count -- independent defects, fixed together because the evidence for either needed the other - out of the way. ``R1CounterParameter``'s ``counter`` was a ``UInt32Field`` where the - RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" - in prose; both versions whose *packing* reaches the class, ``R1_Counter`` (128) and - ``R1_COUNTER`` (129), packed a 12-octet record at ``4 (mod 8)``. It is a - ``UInt64Field`` now and the record is 16 octets (:issue:`672`). - ``LocatorSetParameter``'s ``Length`` was ``sum(Locator.len)``, in the 4-octet units - :rfc:`8046#section-4` gives ``Locator Length``; a set of *n* plain IPv6 locators - declared ``4n`` octets against ``24n`` present, and ``Schema.unpack`` handed the - nested ``ListField`` only the declared octets, so ``n = 2`` and ``n = 5`` parsed one - truncated locator and left the rest unconsumed. ``HIP._make_param_locator_set`` now - sums ``8 + Locator.len * 4`` per locator (:issue:`679`). Another defect in the same + out of the way. :class:`~pcapkit.protocols.schema.internet.hip.R1CounterParameter`'s + ``counter`` was a :class:`.UInt32Field` where the RFC states the width twice, "8 + bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose + *packing* reaches the class, ``R1_Counter`` (128) and ``R1_COUNTER`` (129), packed a + 12-octet record at ``4 (mod 8)``. It is a :class:`.UInt64Field` now and the record is + 16 octets (:issue:`672`). + :class:`~pcapkit.protocols.schema.internet.hip.LocatorSetParameter`'s ``Length`` was + ``sum(Locator.len)``, in the 4-octet units :rfc:`8046#section-4` gives + ``Locator Length``; a set of *n* plain IPv6 locators declared ``4n`` octets against + ``24n`` present, and :meth:`.Schema.unpack` handed the nested :class:`.ListField` only + the declared octets, so ``n = 2`` and ``n = 5`` parsed one truncated locator and left + the rest unconsumed. :meth:`.HIP._make_param_locator_set` now sums + ``8 + Locator.len * 4`` per locator (:issue:`679`). Another defect in the same parameter's padding, which :issue:`651` left alone because it cancelled this one, is - fixed alongside: the nested ``Locator`` schemas share the parameter's packet context - and pack their own ``len`` over it, so ``padding``, evaluated after the list, saw 4 - rather than the real length. A new ``locator_set_len_callback`` snapshots ``Length`` - under a private key before any locator packs, and ``locator_set_padding_len`` reads - that snapshot. The ``counter`` defaulted to ``0`` and the width defect could not be - told apart from it, so the fixture now overrides both ``R1_COUNTER`` codes with - ``counter=0xaabbccdd``; new tests are ``test_hip_r1_counter_width_unit.py`` and - ``test_hip_locator_set_length_unit.py``. The ``hip-parameter/R1_Counter`` - ``EXPECTED_FAILURES`` entry (code 128 registered no schema of its own, since - ``R1CounterParameter`` declared only ``code=129``) was deleted once that - registry defect was fixed as :issue:`690`. ``HIP_COPIES`` dropped to one - as :issue:`689`, once :issue:`672` and :issue:`679` left it routing around nothing. -* ``httpv1.HTTP`` raised a bare exception for any payload that is not an HTTP/1 - message, at these sites: the header/body split in ``read``, the header's CRLF split - and the start-line split in ``_read_http_header``, and ``item[1]`` on a field line - with no colon. ``HTTP._guess_version`` falls through on ``ProtocolError`` alone, so - the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for each - of those inputs. Each site now raises ``ProtocolError``, which each method's + fixed alongside: the nested :class:`~pcapkit.protocols.schema.internet.hip.Locator` + schemas share the parameter's packet context and pack their own ``len`` over it, so + ``padding``, evaluated after the list, saw 4 rather than the real length. A new + ``locator_set_len_callback`` snapshots ``Length`` under a private key before any + locator packs, and ``locator_set_padding_len`` reads that snapshot. The ``counter`` + defaulted to ``0`` and the width defect could not be told apart from it, so the + fixture now overrides both ``R1_COUNTER`` codes with ``counter=0xaabbccdd``; new tests + are ``test_hip_r1_counter_width_unit.py`` and ``test_hip_locator_set_length_unit.py``. + The ``hip-parameter/R1_Counter`` ``EXPECTED_FAILURES`` entry (code 128 registered no + schema of its own, since + :class:`~pcapkit.protocols.schema.internet.hip.R1CounterParameter` declared only + ``code=129``) was deleted once that registry defect was fixed as :issue:`690`. + ``HIP_COPIES`` dropped to one as :issue:`689`, once :issue:`672` and :issue:`679` left + it routing around nothing. +* :class:`httpv1.HTTP ` raised a bare + exception for any payload that is not an HTTP/1 message, at these sites: the + header/body split in ``read``, the header's CRLF split and the start-line split in + ``_read_http_header``, and ``item[1]`` on a field line with no colon. + :meth:`.HTTP._guess_version` falls through on :exc:`.ProtocolError` alone, so the + HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for each of + those inputs. Each site now raises :exc:`.ProtocolError`, which each method's ``Raises:`` section documented, and a test pins the invariant (nothing escapes - ``httpv1.HTTP`` that is not a ``ProtocolError``), so any further bare-raising site - would fail there. An obs-fold continuation line [:rfc:`9112#section-5.2`] is legal - HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own - (with no colon it raised that ``IndexError``; with one it parsed into a spurious - extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to - ``httpv2.HTTP``, where it usually fails inside the schema with a bare - ``struct.error`` (not a ``ValueError``, ``ProtocolError`` or pcapkit ``StructError``, - so uncatchable); ``read`` converts it and ``_guess_version`` suppresses it on the - last arm only, since an arm that is not last hands the payload onward when it - swallows. Separately, ``PayloadField.protocol`` looked a ``str`` up in ``__proto__`` - as given while the registry is keyed on the upper-cased class name, so a name not - already upper-case resolved to ``None`` and yielded ``Raw``; it now folds case and - warns with ``RegistryWarning`` for a genuinely unregistered name, and ``__init__`` - assigns through that property instead of writing ``_protocol`` past it (:issue:`787`, - :pr:`789`). + :class:`httpv1.HTTP ` that is not a + :exc:`.ProtocolError`), so any further bare-raising site would fail there. An obs-fold + continuation line [:rfc:`9112#section-5.2`] is legal HTTP/1 and is now unfolded, the + RFC's remedy, rather than read as a field of its own (with no colon it raised that + ``IndexError``; with one it parsed into a spurious extra field). Reaching the HTTP/2 + arm exposed a payload under nine octets to + :class:`httpv2.HTTP `, where it usually + fails inside the schema with a bare ``struct.error`` (not a ``ValueError``, + :exc:`.ProtocolError` or pcapkit :exc:`.StructError`, so uncatchable); ``read`` + converts it and ``_guess_version`` suppresses it on the last arm only, since an arm + that is not last hands the payload onward when it swallows. Separately, + :attr:`.PayloadField.protocol` looked a ``str`` up in ``__proto__`` as given while the + registry is keyed on the upper-cased class name, so a name not already upper-case + resolved to ``None`` and yielded :class:`~pcapkit.protocols.misc.raw.Raw`; it now + folds case and warns with :exc:`.RegistryWarning` for a genuinely unregistered name, + and ``__init__`` assigns through that property instead of writing ``_protocol`` past + it (:issue:`787`, :pr:`789`). * a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: ``b'MESSAGE=hello'`` plus three NULs @@ -2019,33 +2104,33 @@ Fixed length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 octets, so at most the trailing three octets of a terminator-less line are stripped, - with a ``SchemaWarning``, and anything past that is left as data. The gate reads the - *raw* line, since a real trailing newline, or any other raw last octet including + with a :exc:`.SchemaWarning`, and anything past that is left as data. The gate reads + the *raw* line, since a real trailing newline, or any other raw last octet including ASCII whitespace, proves the padding is zero (:issue:`794`). ``examples/generators/pcapng.py``'s ``_journal_entry`` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names :issue:`678`'s ``not line.strip(b'\x00')`` guard and the mechanism (:issue:`791`). -* ``httpv2.HTTP.read`` tested only the 24-bit declared length off the wire +* :meth:`.httpv2.HTTP.read` tested only the 24-bit declared length off the wire (``schema.length < 9``), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of ``_read_http_settings``'s own ``(declared - 9) % 6`` check. ``HTTP.unpack`` now rejects a non-empty buffer under nine octets before the schema - layer runs, preserving ``StreamEOFError`` for a genuinely exhausted implicit-length - stream, and ``read``'s guard also requires the declared length not to exceed the - buffer. A buffer that clears nine octets can still carry a frame type whose fixed - fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded + layer runs, preserving :exc:`.StreamEOFError` for a genuinely exhausted + implicit-length stream, and ``read``'s guard also requires the declared length not to + exceed the buffer. A buffer that clears nine octets can still carry a frame type whose + fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare - ``struct.error``; that root cause is in the generic ``Schema.unpack``/ - ``FieldBase.length`` machinery the schema modules share, out of scope and filed + ``struct.error``; that root cause is in the generic :meth:`.Schema.unpack`/ + :attr:`.FieldBase.length` machinery the schema modules share, out of scope and filed as :issue:`805`, which is why ``_guess_version``'s last arm keeps suppressing - ``struct.error`` alongside ``ProtocolError`` (:pr:`802`). -* ``SystemdJournalExportBlock.post_process`` skipped a binary field's one trailing + ``struct.error`` alongside :exc:`.ProtocolError` (:pr:`802`). +* :meth:`.SystemdJournalExportBlock.post_process` skipped a binary field's one trailing newline with a bare ``entry_data.read()``, which reads to **EOF**: the next ``readline()`` returned ``b''`` and ended the entry, discarding every field behind the first binary one. The skip is bound to ``entry_data.read(1)``; a missing or - non-``b'\n'`` octet ends the entry with a ``SchemaWarning`` instead of + non-``b'\n'`` octet ends the entry with a :exc:`.SchemaWarning` instead of resynchronising at the wrong offset. ``test_journal_fields_following_a_binary_field_are_not_discarded`` fails against the unfixed code with ``MissingKeyError: 'SECOND'`` (:issue:`704`). @@ -2058,18 +2143,18 @@ Fixed separator on the block's last octet was indistinguishable from EOF and swallowed, so a rebuild wrote ``length`` one octet short (:issue:`723`). * **a breaking change to** ``.get()``-backed enum fields: TCP/UDP/SCTP's - ``PortEnumField`` and PCAP-NG's ``OptionEnumField`` each minted a member through + :class:`.PortEnumField` and PCAP-NG's ``OptionEnumField`` each minted a member through ``aenum.extend_enum`` for every value no row or documented span covers (111 calls on ``http.pcap``, all ephemeral ports). Each now peeks at what ``.get()`` would consult and call it only once the value resolves; a genuine miss gets - ``EnumField._unregistered_member``, a real member of the same registry built through - its storage base's ``__new__``, so ``isinstance`` holds and the lookup table stops - growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code - was minted under ``opt``, ``OptionType.get`` merged ``opt`` over the requested - namespace and most other namespaces answered ``opt_unknown`` for it; each now gets its - own ``_unknown``. **Breaking** because :issue:`575` asked for byte-identical - output across the sample captures, as :pr:`420` and :pr:`427` did, and this - deliberately misses that bar: some sample captures change, entirely from the new + :meth:`.EnumField._unregistered_member`, a real member of the same registry built + through its storage base's ``__new__``, so ``isinstance`` holds and the lookup table + stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once + a code was minted under ``opt``, :meth:`.OptionType.get` merged ``opt`` over the + requested namespace and most other namespaces answered ``opt_unknown`` for it; each + now gets its own ``_unknown``. **Breaking** because :issue:`575` asked for + byte-identical output across the sample captures, as :pr:`420` and :pr:`427` did, and + this deliberately misses that bar: some sample captures change, entirely from the new ```` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, ``_unregistered_member`` installs a ``__reduce_ex__`` rebuilding an equivalent @@ -2088,98 +2173,118 @@ Fixed per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a ``type <= 9`` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged (:issue:`800`). -* ``TCP.__proto__`` bound ``httpv1.HTTP`` directly for ports 80 and 8080, so a segment - on either port was HTTP/1 by assertion of the port alone; both versions share those - ports, so the payload has to decide. Repointed to the generic - ``pcapkit.protocols.application.http.HTTP`` proxy, whose identification became +* :attr:`.TCP.__proto__` bound + :class:`httpv1.HTTP ` directly for ports 80 + and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both + versions share those ports, so the payload has to decide. Repointed to the generic + :class:`pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once :issue:`800` and :pr:`814` landed, which is why this waited. ``udp.py`` already bound the proxy for both ports; this removes the asymmetry its - docstring and ``docs/source/contributing/pep.rst`` documented as an open request. - Over the sample captures every HTTP/1.1 frame keeps its chain, and the genuine - HTTP/2 frames in ``options-transport.pcap`` change ``Ethernet:IPv4:TCP:Raw`` to - ``Ethernet:IPv4:TCP:HTTP/2``. Not labelled breaking, but not free: - ``TCP:80/8080`` traffic that is neither valid HTTP/1 nor preface-carrying is now - exposed to the fall-through arm, where the direct ``httpv1`` binding left it ``Raw`` - regardless; a caller depending on that is the one who would notice (:issue:`682`). + docstring and ``docs/source/contributing/pep.rst`` documented as an open request. Over + the sample captures every HTTP/1.1 frame keeps its chain, and the genuine HTTP/2 + frames in ``options-transport.pcap`` change ``Ethernet:IPv4:TCP:Raw`` to + ``Ethernet:IPv4:TCP:HTTP/2``. Not labelled breaking, but not free: ``TCP:80/8080`` + traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the + fall-through arm, where the direct ``httpv1`` binding left it + :class:`~pcapkit.protocols.misc.raw.Raw` regardless; a caller depending on that is the + one who would notice (:issue:`682`). * **a breaking change to** more bespoke registries: step 2 of :issue:`860`, - ahead of ``AppType`` (:pr:`874`, above), bringing ``StatusCode``, ``ReturnCode``, - ``ResponseKind``, ``GroupingInformation``, ``OptionType``, ``FEATCode``, ``Command`` - and ``Method`` onto ``EnumRegistry`` and applying :issue:`775`'s mint/unmint ruling - to their ``get()`` as well as ``_missing_``. ``StatusCode``, ``ReturnCode``, - ``ResponseKind``, ``GroupingInformation`` and ``OptionType`` convert their unambiguous - placeholder branches (``Unassigned``, ``Unknown``, ``opt_unknown``) to - ``_unregistered_member``; those with a custom ``__new__`` (``StatusCode``, - ``ReturnCode``, ``OptionType``) get an override reconstructing the attributes the - base's generic helper would leave unset, and ``StatusCode``/``ReturnCode``'s - hand-written ``get()``, still on the retired ``default == -1`` convention, is - replaced by the base's (no caller relied on the old form). ``OptionType`` keeps its - own ``get()`` for its multi-namespace dispatch, but a later review found it still - minted on both its int/namespace and ``str`` paths, and the live pcapng parse path - (``PCAPNG._make_pcapng_options``) calls it with wire bytes, so parsing an undeclared - option code still registered a permanent member; both paths now build an unregistered - member. ``FEATCode``, ``Command`` and ``Method`` mint the literal wire value as its - own name rather than any placeholder; per the owner's ruling that ``get()`` lacks - the information to create new members (``Command`` needs - ``feat``/``desc``/``type``/``conf`` and ``Method`` needs ``safe``/``idempotent``, - which a bare wire string lacks), both ``_missing_`` and each class's own ``get()`` - (an independent mint site of its own) now build an unregistered member. ``FEATCode``'s - crawler now declares every real FEAT-code value from the live IANA table instead of - minting some of them as a side effect of evaluating ``Command``'s rows at import time, - the import-time-mutation shape :pr:`861` removed from ``FilterType``; pinned - count-agnostically so a future IANA update cannot fail a correct regeneration. - Untouched, per rulings: ``CommandType`` stays ``IntFlag`` (real ``A|P`` composites in - the generated data), and ``TransportProtocol``'s ``auto()`` renumbering and - ``AppType`` are :pr:`874`'s. ``pcapkit/protocols/schema/misc/pcapng.py``'s - ``OptionEnumField.post_process`` docstring is corrected: it still bypasses - ``OptionType.get()`` directly, now for ``pickle`` round-trip safety (:issue:`860` - already stops the minting that used to be the reason), since neither - of ``OptionType``'s paths gets both pickling and correct rendering right at once - (:pr:`869`). -* ``register`` on ``ProtocolBase``, ``Frame``, ``PCAPNG``, ``SCTP``, ``Link``, - ``Internet`` and the concrete ``Transport`` subclasses (``TCP``, ``UDP``) now raises - ``RegistryError`` for a non-class argument, where it raised a bare ``TypeError`` - ("issubclass() arg 1 must be a class") from inside ``abc``. Each guard gains an - explicit ``isinstance(protocol, type)`` test ahead of its ``issubclass``, after the - ``ModuleDescriptor`` is unwrapped, so a descriptor naming a non-class attribute is - rejected too. Every such ``Raises:`` clause already promised ``RegistryError`` if - ``protocol`` "is not a ... subclass", which a non-class satisfied in spirit but not + ahead of :class:`~pcapkit.const.reg.apptype.apptype.AppType` (:pr:`874`, above), + bringing :class:`~pcapkit.const.http.status_code.StatusCode`, + :class:`~pcapkit.const.ftp.return_code.ReturnCode`, ``ResponseKind``, + ``GroupingInformation``, :class:`~pcapkit.const.pcapng.option_type.OptionType`, + :class:`.FEATCode`, :class:`~pcapkit.const.ftp.command.Command` and + :class:`~pcapkit.const.http.method.Method` onto :class:`.EnumRegistry` and applying + :issue:`775`'s mint/unmint ruling to their ``get()`` as well as ``_missing_``. + :class:`~pcapkit.const.http.status_code.StatusCode`, + :class:`~pcapkit.const.ftp.return_code.ReturnCode`, ``ResponseKind``, + ``GroupingInformation`` and :class:`~pcapkit.const.pcapng.option_type.OptionType` + convert their unambiguous placeholder branches (``Unassigned``, ``Unknown``, + ``opt_unknown``) to ``_unregistered_member``; those with a custom ``__new__`` + (:class:`~pcapkit.const.http.status_code.StatusCode`, + :class:`~pcapkit.const.ftp.return_code.ReturnCode`, + :class:`~pcapkit.const.pcapng.option_type.OptionType`) get an override reconstructing + the attributes the base's generic helper would leave unset, and + :class:`~pcapkit.const.http.status_code.StatusCode`/:class:`~pcapkit.const.ftp.return_code.ReturnCode`'s + hand-written ``get()``, still on the retired ``default == -1`` convention, is replaced + by the base's (no caller relied on the old form). + :class:`~pcapkit.const.pcapng.option_type.OptionType` keeps its own ``get()`` for its + multi-namespace dispatch, but a later review found it still minted on both its + int/namespace and ``str`` paths, and the live pcapng parse path + (:meth:`.PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an + undeclared option code still registered a permanent member; both paths now build an + unregistered member. :class:`.FEATCode`, :class:`~pcapkit.const.ftp.command.Command` + and :class:`~pcapkit.const.http.method.Method` mint the literal wire value as its own + name rather than any placeholder; per the owner's ruling that ``get()`` lacks the + information to create new members (:class:`~pcapkit.const.ftp.command.Command` needs + ``feat``/``desc``/``type``/``conf`` and :class:`~pcapkit.const.http.method.Method` + needs ``safe``/``idempotent``, which a bare wire string lacks), both ``_missing_`` and + each class's own ``get()`` (an independent mint site of its own) now build an + unregistered member. :class:`.FEATCode`'s crawler now declares every real FEAT-code + value from the live IANA table instead of minting some of them as a side effect of + evaluating :class:`~pcapkit.const.ftp.command.Command`'s rows at import time, the + import-time-mutation shape :pr:`861` removed from + :class:`~pcapkit.const.pcapng.filter_type.FilterType`; pinned count-agnostically so a + future IANA update cannot fail a correct regeneration. Untouched, per rulings: + ``CommandType`` stays ``IntFlag`` (real ``A|P`` composites in the generated data), and + :class:`.TransportProtocol`'s ``auto()`` renumbering and + :class:`~pcapkit.const.reg.apptype.apptype.AppType` are :pr:`874`'s. + ``pcapkit/protocols/schema/misc/pcapng.py``'s ``OptionEnumField.post_process`` + docstring is corrected: it still bypasses :meth:`.OptionType.get()` directly, now for + ``pickle`` round-trip safety (:issue:`860` already stops the minting that used to be + the reason), since neither of :class:`~pcapkit.const.pcapng.option_type.OptionType`'s + paths gets both pickling and correct rendering right at once (:pr:`869`). +* ``register`` on :class:`.ProtocolBase`, + :class:`~pcapkit.protocols.misc.pcap.frame.Frame`, + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`, + :class:`~pcapkit.protocols.transport.sctp.SCTP`, :class:`.Link`, :class:`.Internet` + and the concrete :class:`~pcapkit.protocols.transport.transport.Transport` subclasses + (:class:`~pcapkit.protocols.transport.tcp.TCP`, + :class:`~pcapkit.protocols.transport.udp.UDP`) now raises :exc:`.RegistryError` for a + non-class argument, where it raised a bare ``TypeError`` ("issubclass() arg 1 must be + a class") from inside ``abc``. Each guard gains an explicit + ``isinstance(protocol, type)`` test ahead of its ``issubclass``, after the + :class:`.ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute + is rejected too. Every such ``Raises:`` clause already promised :exc:`.RegistryError` + if ``protocol`` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target - class changes, and a wrong class still raises ``RegistryError`` as before. - ``Transport.register`` on ``Transport`` itself still raises ``UnsupportedCall`` - first; only its subclasses reach the guard. Not breaking for exception handling: - ``RegistryError`` subclasses ``TypeError`` through ``BaseError``, so - ``except TypeError`` still catches it, and the type and message text are the only - things a handler can key on that change. Observable side effects do come - with it, because ``BaseError`` is loud by default: the non-class path now emits - pcapkit's one ``CRITICAL`` log record, and outside devmode it installs + class changes, and a wrong class still raises :exc:`.RegistryError` as before. + :meth:`.Transport.register` on + :class:`~pcapkit.protocols.transport.transport.Transport` itself still raises + :exc:`.UnsupportedCall` first; only its subclasses reach the guard. Not breaking for + exception handling: :exc:`.RegistryError` subclasses ``TypeError`` through + :exc:`.BaseError`, so ``except TypeError`` still catches it, and the type and message + text are the only things a handler can key on that change. Observable side effects do + come with it, because :exc:`.BaseError` is loud by default: the non-class path now + emits pcapkit's one ``CRITICAL`` log record, and outside devmode it installs ``sys.excepthook`` and ``threading.excepthook``, neither of which the bare - ``TypeError`` from ``abc`` did. That is the wrong-class path's existing behaviour, - so the fix makes the two consistent rather than introducing something new. - Follows the same fix for the foundation registrars (:issue:`1021`) - (:issue:`1026`). -* ``PCAPNG``'s ``ns_dnsname``, ``ns_dnsIP4addr`` and ``ns_dnsIP6addr`` scope guards, - on both the read and the make side, now say the option "must be in Name Resolution - Block", which is the block they check; they used to name the systemd(1) Journal - Export Block, with a literal ``:manpage:`` role in the text. ``register_record``'s - ``RegistryWarning`` likewise now says "name resolution record already registered". - The make-side ``ns_dnsIP4addr``/``ns_dnsIP6addr`` messages also drop the lowercase - ``ns_dnsip4addr``/``ns_dnsip6addr`` spelling for the ``OptionType`` member name. - Other guards named the wrong option too: ``_make_option_if_rxspeed``'s scope and - duplicate guards now say ``[if_rxspeed]`` rather than ``[if_txspeed]``, and - ``_read_option_epb_queue``'s length guard says ``[epb_queue]`` rather than - ``[epb_packetid]``. Only message text changes; the exception and warning types are - unchanged (:issue:`1038`). -* ``httpv1.HTTP`` refused every HTTP/1 message with no header fields, which - [:rfc:`9112#section-2.1`] allows (``*( field-line CRLF )``): - ``b'GET / HTTP/1.0\r\n\r\n'`` raised ``ProtocolError: HTTP: invalid format``, - because ``_read_http_header`` treated a header with no CRLF as malformed. So + ``TypeError`` from ``abc`` did. That is the wrong-class path's existing behaviour, so + the fix makes the two consistent rather than introducing something new. Follows the + same fix for the foundation registrars (:issue:`1021`) (:issue:`1026`). +* :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`'s ``ns_dnsname``, ``ns_dnsIP4addr`` and + ``ns_dnsIP6addr`` scope guards, on both the read and the make side, now say the option + "must be in Name Resolution Block", which is the block they check; they used to name + the systemd(1) Journal Export Block, with a literal ``:manpage:`` role in the text. + ``register_record``'s :exc:`.RegistryWarning` likewise now says "name resolution + record already registered". The make-side ``ns_dnsIP4addr``/``ns_dnsIP6addr`` messages + also drop the lowercase ``ns_dnsip4addr``/``ns_dnsip6addr`` spelling for the + :class:`~pcapkit.const.pcapng.option_type.OptionType` member name. Other guards named + the wrong option too: ``_make_option_if_rxspeed``'s scope and duplicate guards now say + ``[if_rxspeed]`` rather than ``[if_txspeed]``, and ``_read_option_epb_queue``'s length + guard says ``[epb_queue]`` rather than ``[epb_packetid]``. Only message text changes; + the exception and warning types are unchanged (:issue:`1038`). +* :class:`httpv1.HTTP ` refused every HTTP/1 + message with no header fields, which [:rfc:`9112#section-2.1`] allows + (``*( field-line CRLF )``): ``b'GET / HTTP/1.0\r\n\r\n'`` raised + ``ProtocolError: HTTP: invalid format``, because ``_read_http_header`` treated a + header with no CRLF as malformed. So ``HTTPv1(method='GET', uri='/', http_version='1.1')`` failed to read back the bytes - its own ``make`` built, and ``HTTP._guess_version`` answered + its own ``make`` built, and :meth:`.HTTP._guess_version` answered ``unknown HTTP version``, since ``_test_start_line`` mirrored the same check. A - CRLF-less header is now a start line with zero fields, in the parser and the - predicate alike. That check was also what refused the HTTP/2 connection preface, - whose header is exactly ``PRI * HTTP/2.0``; it is now refused by name instead, to - the same message (:issue:`1041`). + CRLF-less header is now a start line with zero fields, in the parser and the predicate + alike. That check was also what refused the HTTP/2 connection preface, whose header is + exactly ``PRI * HTTP/2.0``; it is now refused by name instead, to the same message + (:issue:`1041`). * ``httpv1.HTTP.from_data`` could not rebuild an HTTP/1 message whose body is empty: ``HTTP.from_data(info)`` for ``b'GET / HTTP/1.1\r\nHost: a\r\n\r\n'`` raised ``TypeError: can't concat NoneType to bytes``. ``read`` records an empty @@ -2192,51 +2297,52 @@ Fixed ``(6, 0x6a, 0xbcdef0)``; it now reads ``(6, 0xab, 0xcdef0)``. The parser reads these fields through the schema, so only a direct call was affected (:issue:`1094`). * the Mobility Header Message ID option wrote a zero fraction for every timestamp, and - ``MesgIDOption.post_process`` ignored the fraction on read. ``_make_opt_mesg_id`` now - writes ``round(frac * 2**32)``, clamped to 32 bits, and the reader adds + :meth:`.MesgIDOption.post_process` ignored the fraction on read. ``_make_opt_mesg_id`` + now writes ``round(frac * 2**32)``, clamped to 32 bits, and the reader adds ``fraction / 2**32`` seconds, so ``timestamp`` keeps sub-second precision (:issue:`1093`). * HIP parameter defects. ``HOST_ID`` packed DI-Type and DI-Length as a 4-octet - ``BitField`` where :rfc:`7401` section 5.2.9 has one 16-bit word, so every built + :class:`.BitField` where :rfc:`7401` section 5.2.9 has one 16-bit word, so every built record carried two surplus octets and read back the wrong algorithm (:issue:`1118`); with it, ``make_samples.py`` builds every HIP parameter. ``HIP_CIPHER`` warned at six Cipher IDs, which section 5.2.8 allows; it now warns only above six and keeps the first six, as a recipient must. The maker is unchanged (:issue:`1092`). And the HIPv2-only ``DH_GROUP_LIST``, ``HIP_CIPHER``, ``HIT_SUITE_LIST`` and ``TRANSPORT_FORMAT_LIST`` were built and parsed in a HIPv1 packet; at version 1 the - reader and the maker now raise ``ProtocolError``, as ``R1_Counter`` and + reader and the maker now raise :exc:`.ProtocolError`, as ``R1_Counter`` and ``HIP_TRANSFORM`` already do on a version mismatch (:issue:`1133`). -* the PCAP ``Header`` could not rebuild itself. ``_make_magic``'s explicit-endian branch - ignored ``nanosecond`` (:issue:`1096`), and ``_make_data`` returned a - ``magic_number`` that ``make()`` does not take -- one of the keys the - :issue:`617` entry above reports -- so ``from_data`` dropped it with an - ``UnknownFieldWarning`` and rebuilt every header with the host-default ``d4c3b2a1`` - magic. ``_make_data`` now returns ``byteorder`` and ``nanosecond``, and every byte - order and timestamp resolution rebuilds unchanged. ``Header._read_protos`` also read - four octets whatever ``size`` asked for (:issue:`1125`). And ``_make_magic`` honoured - ``lilendian`` and ``bigendian`` only as a pair, so ``Header(bigendian=True)`` wrote - ``d4c3b2a1`` on a little-endian host; either flag alone now picks the byte order and - overrides ``byteorder``, and equal flags still raise ``EndianError`` +* the PCAP :class:`~pcapkit.protocols.misc.pcap.header.Header` could not rebuild itself. + ``_make_magic``'s explicit-endian branch ignored ``nanosecond`` (:issue:`1096`), and + ``_make_data`` returned a ``magic_number`` that ``make()`` does not take -- one of the + keys the :issue:`617` entry above reports -- so ``from_data`` dropped it with an + :exc:`.UnknownFieldWarning` and rebuilt every header with the host-default + ``d4c3b2a1`` magic. ``_make_data`` now returns ``byteorder`` and ``nanosecond``, and + every byte order and timestamp resolution rebuilds unchanged. ``Header._read_protos`` + also read four octets whatever ``size`` asked for (:issue:`1125`). And ``_make_magic`` + honoured ``lilendian`` and ``bigendian`` only as a pair, so ``Header(bigendian=True)`` + wrote ``d4c3b2a1`` on a little-endian host; either flag alone now picks the byte order + and overrides ``byteorder``, and equal flags still raise :exc:`.EndianError` (:issue:`1144`). * TCP's NS flag was written by ``make()`` but never parsed: it had been commented out - of ``Flags``, the reader and ``_make_data``, with no reason recorded. Per the owner's - ruling it is exposed rather than removed, so ``info.flags.ns`` exists and - ``from_data`` keeps the bit (:issue:`1098`). -* ``from_data`` round trips. ``Frame._make_data`` returned ``ts_src`` and - ``L2TPv2._make_data`` ``prio``, the other keys the :issue:`617` entry reports, so - ``from_data`` discarded both with an ``UnknownFieldWarning`` and an L2TPv2 priority - bit was lost; they are now ``ts_sec`` and ``priority`` (:issue:`1097`). And - ``ProtocolBase.from_data`` took no keywords, so ``Frame.from_data`` and + of :class:`~pcapkit.protocols.data.transport.tcp.Flags`, the reader and + ``_make_data``, with no reason recorded. Per the owner's ruling it is exposed rather + than removed, so ``info.flags.ns`` exists and ``from_data`` keeps the bit + (:issue:`1098`). +* ``from_data`` round trips. :meth:`.Frame._make_data` returned ``ts_src`` and + :meth:`.L2TPv2._make_data` ``prio``, the other keys the :issue:`617` entry reports, so + ``from_data`` discarded both with an :exc:`.UnknownFieldWarning` and an L2TPv2 + priority bit was lost; they are now ``ts_sec`` and ``priority`` (:issue:`1097`). And + :meth:`.ProtocolBase.from_data` took no keywords, so ``Frame.from_data`` and ``PCAPNG.from_data``, which need ``num``/``header`` and ``num``/``sct``/``ctx``, could not be called at all. Extra keywords now reach the construction, overriding ``_make_data``'s, under the construction-keyword check, so a typo raises - ``UnsupportedCall`` (:issue:`1126`). ``HTTP.from_data`` always raised + :exc:`.UnsupportedCall` (:issue:`1126`). ``HTTP.from_data`` always raised ``invalid HTTP version: 0``, looking up a ``version`` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's ``_make_data`` also - returned the payload ``length``, which ``ProtocolBase`` takes as the parse length, so - even a direct call failed as shorter than the 9-octet frame header; ``make`` now - computes it, and every HTTP frame in the sample captures rebuilds byte for byte - (:issue:`1154`). + returned the payload ``length``, which :class:`.ProtocolBase` takes as the parse + length, so even a direct call failed as shorter than the 9-octet frame header; + ``make`` now computes it, and every HTTP frame in the sample captures rebuilds byte + for byte (:issue:`1154`). * Quick-Start options did not round-trip. The IPv4, HOPOPT and IPv6-Opts readers took the function from ``schema.func``, which only a parsed schema carries, so a built option raised ``AttributeError``; HOPOPT and IPv6-Opts also sized the 8-octet option @@ -2244,14 +2350,18 @@ Fixed TCP's Quick-Start Response reader read one octet past the option, and its maker produced a negative rate code below 40 kbps; such a rate now rounds down to zero, which :rfc:`4782` permits (:issue:`1119`). -* ``Ethernet``, ``ARP`` and ``PCAPNG`` packed MAC addresses as ASCII hex text. +* :class:`~pcapkit.protocols.link.ethernet.Ethernet`, + :class:`~pcapkit.protocols.link.arp.ARP` and + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` packed MAC addresses as ASCII hex text. ``Ethernet._make_mac_addr`` returned the separator-stripped hex string and - ``BytesField(length=6)`` truncated it, so ``Ethernet()`` put ``303030303030`` on the - wire for an all-zero address (:issue:`1123`). ``ARP._make_addr_resolve``, + ``BytesField(length=6)`` truncated it, so + :class:`Ethernet() ` put ``303030303030`` on + the wire for an all-zero address (:issue:`1123`). ``ARP._make_addr_resolve``, ``PCAPNG._make_mac_addr`` and ``_make_eui_addr`` did the same; all now decode the hex to octets, and ``from_data`` rebuilds each byte for byte (:issue:`1134`). * TCP's User Timeout maker wrote ``length=3`` where :rfc:`5482` and the reader require - 4, so every option it built failed to parse with ``ProtocolError`` (:issue:`1120`). + 4, so every option it built failed to parse with :exc:`.ProtocolError` + (:issue:`1120`). * **a breaking change to** the HTTP/2 frame ``length``: it counted the 9-octet frame header, where :rfc:`9113` section 4.1 counts the payload alone, so ``make()`` wrote ``00000e`` for a 5-octet PRIORITY frame and ``read`` rejected RFC-conformant frames @@ -2260,98 +2370,111 @@ Fixed old convention no longer parse; the PRIORITY, RST_STREAM, SETTINGS (with and without ACK), PING, WINDOW_UPDATE, DATA, PUSH_PROMISE and GOAWAY frames that Scapy's ``contrib.http2`` packs now parse and rebuild byte for byte (:issue:`1121`). -* **a breaking change to** ``ARP.make()``: ``hlen`` and ``plen`` are derived from the - packed ``sha`` and ``spa`` when omitted, and an address that disagrees with its length - field raises ``ProtocolError`` where it used to be truncated silently -- +* **a breaking change to** :meth:`.ARP.make()`: ``hlen`` and ``plen`` are derived from + the packed ``sha`` and ``spa`` when omitted, and an address that disagrees with its + length field raises :exc:`.ProtocolError` where it used to be truncated silently -- ``ARP(ptype=IPv6, spa='::1')`` wrote ``plen=4`` and four octets of the address. A non-IP ``str`` protocol address packs as hex octets rather than ASCII text, as hardware addresses do since :issue:`1134`, and non-hex input raises - ``ProtocolError`` (:issue:`1139`). -* HTTP/2 ``SETTINGS`` built each setting pair as a ``SchemaField`` with no length, so a - well-formed frame warned ``packet length < 0`` twice per pair. The item now has its - fixed 6-octet size, as TCP SACK and SCTP gap-ack items already do (:issue:`1140`). -* ``HTTP._guess_version`` took almost any binary for HTTP/2 on its fall-through: nine - zero octets read as a DATA frame. The HTTP/2 trial now runs only on a header with a - type :rfc:`9113` defines, no undefined flag, the reserved bit clear and a stream that - type is sent on. Over 5,000 fuzzed zero-heavy buffers, HTTP/2 labels fell from 566 to - 101, every one a legal DATA header. Explicit ``version=2`` parsing is unchanged + :exc:`.ProtocolError` (:issue:`1139`). +* HTTP/2 ``SETTINGS`` built each setting pair as a :class:`.SchemaField` with no length, + so a well-formed frame warned ``packet length < 0`` twice per pair. The item now has + its fixed 6-octet size, as TCP SACK and SCTP gap-ack items already do (:issue:`1140`). +* :meth:`.HTTP._guess_version` took almost any binary for HTTP/2 on its fall-through: + nine zero octets read as a DATA frame. The HTTP/2 trial now runs only on a header with + a type :rfc:`9113` defines, no undefined flag, the reserved bit clear and a stream + that type is sent on. Over 5,000 fuzzed zero-heavy buffers, HTTP/2 labels fell from + 566 to 101, every one a legal DATA header. Explicit ``version=2`` parsing is unchanged (:issue:`1141`). -* **a breaking change to** ``TCP.make()``'s options: ``_make_tcp_options`` dropped the - ``NOP`` and ``EOOL`` options it was given and padded each other option to 4 octets +* **a breaking change to** :meth:`.TCP.make()`'s options: ``_make_tcp_options`` dropped + the ``NOP`` and ``EOOL`` options it was given and padded each other option to 4 octets separately, so ``from_data`` rebuilt none of the segments in ``http.pcap`` with their original header octets. Options are now emitted as given and padded once at the end -- zero octets after a trailing ``EOOL``, otherwise ``NOP`` options and an ``EOOL`` -- so a built header can be shorter than before; every TCP segment of the measured sample captures rebuilds its header octets, and ``options-*.pcap`` regenerate unchanged (:issue:`1151`). -* **a breaking change to** ``from_data`` on a capture truncated by snaplen: ``IPv4``, - ``IPv6`` and ``UDP`` rebuilt their length field from the payload at hand, writing - the recorded length, so ``big_endian.pcap`` frame 3 came back as ``45000052`` where - ``450004a2`` was on the wire. ``_make_data`` now passes the parsed length through to - ``make()``'s new ``total_length`` (IPv4, UDP) or ``payload_length`` (IPv6) keyword, - which computes the length as before when omitted, so ``IPv4(payload=...)`` is - unchanged (:issue:`1155`). +* **a breaking change to** ``from_data`` on a capture truncated by snaplen: + :class:`~pcapkit.protocols.internet.ipv4.IPv4`, + :class:`~pcapkit.protocols.internet.ipv6.IPv6` and + :class:`~pcapkit.protocols.transport.udp.UDP` rebuilt their length field from the + payload at hand, writing the recorded length, so ``big_endian.pcap`` frame 3 came back + as ``45000052`` where ``450004a2`` was on the wire. ``_make_data`` now passes the + parsed length through to ``make()``'s new ``total_length`` (IPv4, UDP) or + ``payload_length`` (IPv6) keyword, which computes the length as before when omitted, + so ``IPv4(payload=...)`` is unchanged (:issue:`1155`). * **a breaking change to** the layer chain of a capture cut inside a header: a next layer parsed from too few captured octets was zero-filled, and ``from_data`` rebuilt - the fabricated octets. ``IPv4.read`` handed UDP the declared payload length when + the fabricated octets. :meth:`.IPv4.read` handed UDP the declared payload length when nothing past the IPv4 header was captured, so ``test.pcapng`` frame 3 rebuilt with 8 invented UDP octets (:issue:`1167`). Every ``_import_next_layer`` now goes through - one guard: no captured octets gives ``NoPayload``, and fewer than the next layer's - ``__length_hint__`` gives ``Raw`` unless the parsed header fits anyway, as a 12-octet - L2TPv2 ZLB does. A header-only IPv6 packet is ``IPv6`` rather than ``IPv6:UDP:Raw``, - and frame 3 is ``Ethernet:Raw``, rebuilt byte for byte (:issue:`1170`). The guard - needed honest hints, so ``IPv6_Route`` (4), ``MH`` (6), ``HOPOPT`` and - ``IPv6_Opts`` (2) now hint their 8-octet minimum header (:issue:`1173`), and so does - the ``IPv6_Ext`` fallback, which hinted 2 (:issue:`1178`). -* **a breaking change to** ``L2TPv2``, against :rfc:`2661` section 3.1. ``Type`` was - inverted; it is now ``Data = 0`` and ``Control = 1``, regenerated in - ``pcapkit.const`` from the fixed ``pcapkit.vendor`` table, so the same bytes parse - to the other ``flags.type``, and a default ``make()`` writes T=0 where it wrote T=1. - A control message now gets ``L`` and a Length field automatically, and the offset - pad, which ``from_data`` zero-filled, is kept as ``info.padding`` -- a new key in - ``info`` and in dumped output (:issue:`1175`). ``from_data`` also cleared ``L`` and - dropped the Length field: ``_make_data`` returned it as ``length``, which - ``ProtocolBase`` takes as the parse length. ``make()``'s ``length`` keyword is now - ``total_length``, written as given; a call still passing ``length`` loses ``L`` - silently (:issue:`1172`). ``make()`` also gains ``length_flag`` and ``padding``, - and raises ``ProtocolError`` for a ``total_length`` with ``length_flag=False``. + one guard: no captured octets gives :class:`~pcapkit.protocols.misc.null.NoPayload`, + and fewer than the next layer's ``__length_hint__`` gives + :class:`~pcapkit.protocols.misc.raw.Raw` unless the parsed header fits anyway, as a + 12-octet L2TPv2 ZLB does. A header-only IPv6 packet is ``IPv6`` rather than + ``IPv6:UDP:Raw``, and frame 3 is ``Ethernet:Raw``, rebuilt byte for byte + (:issue:`1170`). The guard needed honest hints, so + :class:`~pcapkit.protocols.internet.ipv6_route.IPv6_Route` (4), + :class:`~pcapkit.protocols.internet.mh.MH` (6), + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` and + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` (2) now hint their 8-octet + minimum header (:issue:`1173`), and so does the + :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` fallback, which hinted 2 + (:issue:`1178`). +* **a breaking change to** :class:`.L2TPv2`, against :rfc:`2661` section 3.1. + :class:`~pcapkit.const.l2tp.type.Type` was inverted; it is now ``Data = 0`` and + ``Control = 1``, regenerated in :mod:`pcapkit.const` from the fixed + :mod:`pcapkit.vendor` table, so the same bytes parse to the other ``flags.type``, and + a default ``make()`` writes T=0 where it wrote T=1. A control message now gets ``L`` + and a Length field automatically, and the offset pad, which ``from_data`` zero-filled, + is kept as ``info.padding`` -- a new key in ``info`` and in dumped output + (:issue:`1175`). ``from_data`` also cleared ``L`` and dropped the Length field: + ``_make_data`` returned it as ``length``, which :class:`.ProtocolBase` takes as the + parse length. ``make()``'s ``length`` keyword is now ``total_length``, written as + given; a call still passing ``length`` loses ``L`` silently (:issue:`1172`). + ``make()`` also gains ``length_flag`` and ``padding``, and raises + :exc:`.ProtocolError` for a ``total_length`` with ``length_flag=False``. * **a breaking change to** ``IPv6.from_data``: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden ``__exthdr__`` key, absent from ``to_dict`` and dumped output, and re-emitted ahead of the upper layer. Sample frames that rebuilt wrong now rebuild exactly, though some that had rebuilt with wrong bytes raised ``AttributeError`` instead (:issue:`1177`), from older defects fixed with it: - ``HIP.alias`` read ``_info`` before ``read`` had set it, which also broke a + :attr:`.HIP.alias` read ``_info`` before ``read`` had set it, which also broke a standalone HIP parse and ``HIP.from_data`` (:issue:`1184`), and the MPL option's maker called ``bit_length()`` on the ``IPv6Address`` seed of an S=0 option. MPL in - ``HOPOPT`` and ``IPv6_Opts`` now takes ``S`` from the parsed option or a new - ``seed_type`` keyword, and a missing or oversized seed raises ``ProtocolError`` - (:issue:`1185`). With both fixed, every IPv6 frame of the + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` and + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` now takes ``S`` from the + parsed option or a new ``seed_type`` keyword, and a missing or oversized seed raises + :exc:`.ProtocolError` (:issue:`1185`). With both fixed, every IPv6 frame of the ``options-*.pcap`` captures rebuilds byte for byte. -* **a breaking change to** the SMF_DPD option's maker in ``HOPOPT`` and - ``IPv6_Opts``: it overwrote the parsed DPD mode, so H-DPD rebuilt as I-DPD, and - inferred the TaggerID type from the Tagger ID's length, so a 4-octet ``DEFAULT`` - Tagger ID rebuilt as IPv4. Both now come from the parsed option, or from ``mode`` - and a new ``tid_type`` keyword. A Tagger ID that does not fit its type raises - ``ProtocolError``, so a 17-octet one does where it raised ``FieldValueError``, and - ``tid=b''`` builds a NULL Tagger ID where it raised ``ValueError`` (:issue:`1190`). - An H-DPD option with an empty Hash Assist Value raises ``ProtocolError`` where it - raised a bare ``IndexError`` (:issue:`1193`). -* a ``ForwardMatchField`` that read short at the end of the stream rewound by its +* **a breaking change to** the SMF_DPD option's maker in + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` and + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts`: it overwrote the parsed DPD + mode, so H-DPD rebuilt as I-DPD, and inferred the TaggerID type from the Tagger ID's + length, so a 4-octet ``DEFAULT`` Tagger ID rebuilt as IPv4. Both now come from the + parsed option, or from ``mode`` and a new ``tid_type`` keyword. A Tagger ID that does + not fit its type raises :exc:`.ProtocolError`, so a 17-octet one does where it raised + :exc:`.FieldValueError`, and ``tid=b''`` builds a NULL Tagger ID where it raised + ``ValueError`` (:issue:`1190`). An H-DPD option with an empty Hash Assist Value raises + :exc:`.ProtocolError` where it raised a bare ``IndexError`` (:issue:`1193`). +* a :class:`.ForwardMatchField` that read short at the end of the stream rewound by its declared width, not by the octets it read, so the fields after it re-read the - preceding octet. An SMF_DPD option in the last two octets of a ``HOPOPT`` or - ``IPv6_Opts`` header was re-read from the padding and raised a bare - ``TypeError``; it now raises ``FieldValueError``, as CALIPSO and MPL do. A full - read rewinds as before (:issue:`1195`). -* ``HIP()`` with no parameters wrote Header Length 0, below :rfc:`7401` section - 5.1's minimum of 4 for the two HITs, so it failed to read back with - ``ProtocolError``; it now writes 4. ``read`` names a Header Length below 4 in a new - ``invalid header length`` message. Lengths 0 to 3 already raised ``ProtocolError``, - so only the message changes (:issue:`1189`). -* the RPL option, type ``0x23``, raised ``AttributeError`` in ``HOPOPT`` and - ``IPv6_Opts`` (:issue:`1196`, :pr:`1198`). The test that an option of any type in - the last two octets of the header fails in-library no longer skips it; test-only - (:issue:`1771`, :pr:`1776`). + preceding octet. An SMF_DPD option in the last two octets of a + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` or + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` header was re-read from the + padding and raised a bare ``TypeError``; it now raises :exc:`.FieldValueError`, as + CALIPSO and MPL do. A full read rewinds as before (:issue:`1195`). +* :class:`HIP() ` with no parameters wrote Header + Length 0, below :rfc:`7401` section 5.1's minimum of 4 for the two HITs, so it failed + to read back with :exc:`.ProtocolError`; it now writes 4. ``read`` names a Header + Length below 4 in a new ``invalid header length`` message. Lengths 0 to 3 already + raised :exc:`.ProtocolError`, so only the message changes (:issue:`1189`). +* the RPL option, type ``0x23``, raised ``AttributeError`` in + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` and + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` (:issue:`1196`, :pr:`1198`). + The test that an option of any type in the last two octets of the header fails + in-library no longer skips it; test-only (:issue:`1771`, :pr:`1776`). * **a breaking change to** the RPL option: it keeps its reserved flag bits on rebuild and reads :rfc:`6553` sub-TLVs (:issue:`1199`, :pr:`1201`). * ``make()`` rejected an enum member name given as ``str`` unless a namespace was @@ -2383,22 +2506,23 @@ Fixed * IPv4 option areas, route slots, timestamps and reserved bits (:pr:`1344`); the TS option's slots beyond the pointer (:issue:`1349`, :pr:`1390`); TS and SEC options byte for byte (:pr:`1399`). - * IPv6 core headers (:pr:`1307`); ``HOPOPT`` and ``IPv6_Opts`` option areas - (:pr:`1310`); a CALIPSO option's trailing pad (:issue:`1403`, :pr:`1407`), and - its ``make()`` writing a corrupt length on overflow (:issue:`1408`, :pr:`1409`); an - extension header parsed with ``extension=True`` (:issue:`1446`, :pr:`1449`); - RPL source routes (:pr:`1463`); and a cut extension-header chain (:pr:`1476`). + * IPv6 core headers (:pr:`1307`); :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` + and :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` option areas + (:pr:`1310`); a CALIPSO option's trailing pad (:issue:`1403`, :pr:`1407`), and its + ``make()`` writing a corrupt length on overflow (:issue:`1408`, :pr:`1409`); an + extension header parsed with ``extension=True`` (:issue:`1446`, :pr:`1449`); RPL + source routes (:pr:`1463`); and a cut extension-header chain (:pr:`1476`). * octets past IPv4 Total Length and IPv6 Payload Length (:issue:`1209`, :pr:`1431`), and past the IPX Length field (:issue:`1433`, :pr:`1435`). IPX ``from_data`` raised ``struct.error``, and AH wrote its reserved field back as zero (:pr:`1338`). * HIP: Header Length, an empty parameter list, the PUZZLE lifetime and reserved bits - (:pr:`1342`); Locator flags (:issue:`1365`, :pr:`1367`); ``BitField`` + (:pr:`1342`); Locator flags (:issue:`1365`, :pr:`1367`); :class:`.BitField` namespaces' reserved bits (:issue:`1417`, :pr:`1419`); and the two fixed header bits (:issue:`1459`, :pr:`1462`). * MH: zero-length PadN, reserved fields and BU/BA flag bits (:issue:`1324`, - :pr:`1345`); the reserved bits of the ``BitField`` flag namespaces - (:issue:`1418`, :pr:`1421`); and the reserved octets ``PaddingField`` sites + :pr:`1345`); the reserved bits of the :class:`.BitField` flag namespaces + (:issue:`1418`, :pr:`1421`); and the reserved octets :class:`.PaddingField` sites capture (:pr:`1442`). * PCAP-NG: constructed and rebuilt blocks (:pr:`1339`), lengths and padding (:pr:`1373`), blocks and options as parsed (:pr:`1378`), further follow-ups @@ -2407,20 +2531,20 @@ Fixed captured padding octets (:pr:`1441`). * a cut PCAP header or PCAP-NG block (:issue:`1470`, :pr:`1475`). -* ``Schema.pack`` keeps option padding, ``SchemaField`` spans and iterated keys - (:pr:`1374`); a ``SchemaField``'s unread octets survive ``from_data`` rather than - rebuilding zero-filled (:issue:`1380`, :pr:`1397`); a header cut inside a - fixed-width field rebuilds as captured (:pr:`1465`); and each non-final ``Schema`` - subclass is finalised on its own state (:issue:`1498`, :pr:`1499`). +* :meth:`.Schema.pack` keeps option padding, :class:`.SchemaField` spans and iterated + keys (:pr:`1374`); a :class:`.SchemaField`'s unread octets survive ``from_data`` + rather than rebuilding zero-filled (:issue:`1380`, :pr:`1397`); a header cut inside a + fixed-width field rebuilds as captured (:pr:`1465`); and each non-final + :class:`.Schema` subclass is finalised on its own state (:issue:`1498`, :pr:`1499`). * ``from_data`` rebuilds the payload from ``info.to_dict()`` (:issue:`1447`, :pr:`1452`). * building a PCAP frame with ``snaplen`` 0 wrote a record with no packet data (:issue:`1385`, :pr:`1410`); an out-of-range ``if_tzone`` is kept as captured and every ``fromtimestamp`` failure is caught (:issue:`1340`, :pr:`1429`). * registration: the parser registrars warn only when the entry really changes - (:issue:`1362`, :pr:`1400`); re-registering a stored ``ModuleDescriptor`` keeps it - (:issue:`1364`, :pr:`1415`); restoring a registration puts back the shipped - descriptor, including through ``PCAPNG.register`` (:pr:`1533`, :pr:`1573`); and + (:issue:`1362`, :pr:`1400`); re-registering a stored :class:`.ModuleDescriptor` keeps + it (:issue:`1364`, :pr:`1415`); restoring a registration puts back the shipped + descriptor, including through :meth:`.PCAPNG.register` (:pr:`1533`, :pr:`1573`); and unregistering a custom protocol removes its name from ``__proto__`` (:issue:`1505`, :pr:`1530`). * fragments: an IPv6 extension header behind a first fragment is dissected as without @@ -2432,21 +2556,21 @@ Fixed (:issue:`1434`, :pr:`1440`), a jumbogram :rfc:`2675` calls an error is rejected (:issue:`1687`, :pr:`1713`), and so is a header whose Version is not 6 (:issue:`1680`, :pr:`1723`). -* the warnings of a next layer replaced by ``Raw`` are dropped (:issue:`1580`, - :pr:`1588`). +* the warnings of a next layer replaced by :class:`~pcapkit.protocols.misc.raw.Raw` are + dropped (:issue:`1580`, :pr:`1588`). * IPv6-Route's data-type readers name the routing type in their - ``ProtocolError``, where each message read ``[TypeNo ]`` + :exc:`.ProtocolError`, where each message read ``[TypeNo ]`` (:issue:`442`), and ``HTTP(..., version=1)`` parses from the frame's data rather than the drained file, where it raised ``ValueError`` (:issue:`447`, :pr:`451`). -* ``Schema.pack`` accepts a ``tuple`` for a ``ListField``, as the data models +* :meth:`.Schema.pack` accepts a ``tuple`` for a :class:`.ListField`, as the data models declare, so HIP and MH options that hold one round-trip (:issue:`476`, :pr:`480`). * a Mobility Header CGA extension reports its length with its 4-octet header, where every parsed one read 2 octets short (:issue:`512`, :issue:`517`, :pr:`528`). * TCP's SACK option enforces the ``8n + 2`` length :rfc:`2018` gives it, as its docstring promised (:issue:`519`, :pr:`535`). -* ``IPv4.from_data()`` rebuilds a parsed datagram: ``Schema.pack`` checks a payload - against ``ProtocolBase``, and option padding ends in an ``EOOL`` option rather +* ``IPv4.from_data()`` rebuilds a parsed datagram: :meth:`.Schema.pack` checks a payload + against :class:`.ProtocolBase`, and option padding ends in an ``EOOL`` option rather than a bare wire code (:issue:`506`, :pr:`536`). * IPv4's SEC option sets the termination bit its own reader requires, sizes the bitmap from the bit count, and SID is 16 bits wide (:issue:`534`, :issue:`537`, @@ -2454,20 +2578,22 @@ Fixed * the MPTCP option header reads ``length`` from bit 8, per :rfc:`8684#section-3`, rather than bit 1 (:issue:`553`, :pr:`558`). * Mobility Header enumerations defined inline from their RFCs - (``FastBindingAcknowledgmentStatus``, ``IPv6AddressPrefixCode``, - ``LocalizedRoutingStatus``, ``LMAAddressCode``) raise ``EnumValueError`` for an - unassigned value rather than minting ``Unassigned_N`` (:issue:`880`, :pr:`889`), and - each inherits ``get`` and its ``default`` from ``EnumLookup`` rather than - overriding it (:issue:`935`, :pr:`940`). -* ``pcapkit.protocols.protocol.__all__`` lists ``Protocol`` beside ``ProtocolBase``, - so ``from pcapkit.protocols.protocol import *`` binds the module's public class + (:class:`.FastBindingAcknowledgmentStatus`, :class:`.IPv6AddressPrefixCode`, + :class:`.LocalizedRoutingStatus`, :class:`.LMAAddressCode`) raise + :exc:`.EnumValueError` for an unassigned value rather than minting ``Unassigned_N`` + (:issue:`880`, :pr:`889`), and each inherits ``get`` and its ``default`` from + :class:`.EnumLookup` rather than overriding it (:issue:`935`, :pr:`940`). +* ``pcapkit.protocols.protocol.__all__`` lists + :class:`~pcapkit.protocols.protocol.Protocol` beside :class:`.ProtocolBase`, so + ``from pcapkit.protocols.protocol import *`` binds the module's public class (:issue:`1770`, :pr:`1778`). * the data and schema packages re-export what their submodules list in ``__all__``, - under the prefixes their siblings use: ``pcapkit.protocols.data.link`` exports the - L2TP data model as ``L2TP`` and ``L2TP_Flags`` (:issue:`1797`, :pr:`1806`), and the - Mobility Header models, the HOPOPT and IPv6-Opts ``SMFDPDOption`` and - ``QuickStartOption``, PCAP-NG's ``PacketBlock`` and ``PACK_*`` options and HTTP/2's - ``Settings`` reach their packages and the aggregators (:issue:`1807`, :pr:`1815`). + under the prefixes their siblings use: :mod:`pcapkit.protocols.data.link` exports the + L2TP data model as :class:`~pcapkit.protocols.data.link.l2tp.L2TP` and ``L2TP_Flags`` + (:issue:`1797`, :pr:`1806`), and the Mobility Header models, the HOPOPT and IPv6-Opts + ``SMFDPDOption`` and ``QuickStartOption``, PCAP-NG's ``PacketBlock`` and ``PACK_*`` + options and HTTP/2's ``Settings`` reach their packages and the aggregators + (:issue:`1807`, :pr:`1815`). * the MPL option's ``seed`` in the HOPOPT and IPv6-Opts schemas is annotated ``int | IPv6Address``, since an S=0 option holds the source address (:issue:`1840`, :pr:`1866`). @@ -2482,22 +2608,22 @@ Added ``ENCAP_TYPE_TO_LINKTYPE`` and ``FILTER_NAME_TO_LINKTYPE`` in place inside ``pcapkit/toolkit/pyshark.py``, so the tables :pr:`850` hand-built stop being hand-maintained. It lives under ``util/`` rather than ``pcapkit/vendor/`` - because ``Vendor``'s base class is hardwired to an HTTP fetch and a generated + because :class:`.Vendor`'s base class is hardwired to an HTTP fetch and a generated enum-class file, where this generator's source of truth is the local ``tshark``/``editcap`` binaries. It guards against its worst failure mode: a naive ``LinkType(dlt)`` cannot detect an unmapped DLT, because ``_missing_`` mints a - placeholder rather than raising, so the generator snapshots every known ``LinkType`` - value first and calls ``LinkType(dlt)`` only once a value is confirmed a member. The - real-``tshark`` sweep is gated on a new ``HAS_WIRESHARK`` flag, registered in - ``tests/_dependency_gates.py``'s ``NON_DISTRIBUTION_FLAGS`` next to ``HAS_PROC_FD``, - for things pip cannot install. The sweep arithmetic is version-pinned - (``editcap -T`` accepts 226 encapsulations on Wireshark 4.6.9, 224 on 4.2.2, CI's - Ubuntu noble), so count assertions run only under exactly the measured version, with - a separate ``VersionIndependentInvariantTests`` class for what holds regardless. The - only change to ``pcapkit/toolkit/pyshark.py`` itself is the ``IPMB_LINUX`` entry - becoming ``I2C_LINUX``, the canonical name :pr:`848` gave value 209 -- the same - member either way (:pr:`853`). -* ``tcp_segment`` and its result type are public in ``pcapkit.toolkit.pcap`` + placeholder rather than raising, so the generator snapshots every known + :class:`~pcapkit.const.reg.linktype.LinkType` value first and calls ``LinkType(dlt)`` + only once a value is confirmed a member. The real-``tshark`` sweep is gated on a new + ``HAS_WIRESHARK`` flag, registered in ``tests/_dependency_gates.py``'s + ``NON_DISTRIBUTION_FLAGS`` next to ``HAS_PROC_FD``, for things pip cannot install. The + sweep arithmetic is version-pinned (``editcap -T`` accepts 226 encapsulations on + Wireshark 4.6.9, 224 on 4.2.2, CI's Ubuntu noble), so count assertions run only under + exactly the measured version, with a separate ``VersionIndependentInvariantTests`` + class for what holds regardless. The only change to ``pcapkit/toolkit/pyshark.py`` + itself is the ``IPMB_LINUX`` entry becoming ``I2C_LINUX``, the canonical name + :pr:`848` gave value 209 -- the same member either way (:pr:`853`). +* ``tcp_segment`` and its result type are public in :mod:`pcapkit.toolkit.pcap` (:issue:`1570`, :pr:`1579`). Fixed @@ -2528,29 +2654,31 @@ Fixed whose ``__init__`` unpacks the header with ``struct.unpack``, so every field came out garbage without raising. Fixed with ``binascii.unhexlify(packet.packet)`` ahead of the decoder call, in preference to decoding eagerly at ``layers>0`` inside the lazy - generator, which would lose ``_decode()``'s per-frame ``AttributeWarning`` fallback. - Link-layer dispatch resolves from the header's ``ll_type`` and is unaffected. Landed - with :issue:`743`: standalone, this turns a non-crashing extraction into a crashing - one, since ``Extractor.run()`` catches only ``EOFError``, ``StopIteration`` and - ``KeyboardInterrupt`` and the ``AddressValueError``/``struct.error`` :issue:`743` - fixes escape until both land (:issue:`746`). + generator, which would lose ``_decode()``'s per-frame :exc:`.AttributeWarning` + fallback. Link-layer dispatch resolves from the header's ``ll_type`` and is + unaffected. Landed with :issue:`743`: standalone, this turns a non-crashing extraction + into a crashing one, since :meth:`.Extractor.run()` catches only ``EOFError``, + ``StopIteration`` and ``KeyboardInterrupt`` and the + ``AddressValueError``/``struct.error`` :issue:`743` fixes escape until both land + (:issue:`746`). * **a breaking change to** pyshark link-type resolution in ``tcp_traceflow``: an - unrecognised encapsulation or filter name now raises ``MissingKeyError`` instead of - substituting a plausible DLT. ``pcapkit/toolkit/pyshark.py`` resolved a frame's link - type from ``packet.layers[0].layer_name``, a Wireshark display-filter name, with a - blanket ``Enum_LinkType.get(name.upper())`` fallback. One filter name serves several + unrecognised encapsulation or filter name now raises :exc:`.MissingKeyError` instead + of substituting a plausible DLT. ``pcapkit/toolkit/pyshark.py`` resolved a frame's + link type from ``packet.layers[0].layer_name``, a Wireshark display-filter name, with + a blanket ``Enum_LinkType.get(name.upper())`` fallback. One filter name serves several encapsulations, so that fallback returned *wrong DLTs silently* (``null`` for a ``DLT_LOOP`` capture, ``101`` (``RAW``) for ``rawip6``). Re-keyed onto ``frame.encap_type``, which is distinct per encapsulation, and the fallback is gone. Both tables were measured rather than transcribed (Wireshark's source is not on the build host): each entry round-tripped through ``editcap -F pcap -T `` and read back with ``tshark``. ``ENCAP_TYPE_TO_LINKTYPE`` grows to cover every measured - encapsulation whose DLT has a ``LinkType`` member, and ``FILTER_NAME_TO_LINKTYPE`` - grows from :pr:`838`'s hand-built entries to every root filter name, pseudo-protocol - roots aside, that resolved to a single ``LinkType`` member across the encapsulations - this host's Wireshark can write. ``pcapkit/toolkit/pyshark.py`` reaches full coverage - under new tests that fail against the unfixed tree; live pyshark parsing stays - unexercised, since ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). + encapsulation whose DLT has a :class:`~pcapkit.const.reg.linktype.LinkType` member, + and ``FILTER_NAME_TO_LINKTYPE`` grows from :pr:`838`'s hand-built entries to every + root filter name, pseudo-protocol roots aside, that resolved to a single + :class:`~pcapkit.const.reg.linktype.LinkType` member across the encapsulations this + host's Wireshark can write. ``pcapkit/toolkit/pyshark.py`` reaches full coverage under + new tests that fail against the unfixed tree; live pyshark parsing stays unexercised, + since ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). * ``pypcapfile``'s ``tcp_reassembly`` set ``last`` one past the segment. ``TCP_Packet.last`` is inclusive, and the other adapters set ``seqnum + len(payload) - 1``, as this one now does too (:issue:`1100`). @@ -2564,8 +2692,8 @@ Fixed the NS bit (:issue:`1618`, :pr:`1621`); and the scapy adapters stop at the IP datagram's declared length (:issue:`1584`, :pr:`1589`). * timestamps: dpkt and pypcapfile hand over the correctly rounded timestamp - (:issue:`1668`, :pr:`1690`), and a copied ``DecimalTimestamp`` keeps ``orig_len`` - under ``_pydecimal`` (:issue:`1647`, :pr:`1653`). + (:issue:`1668`, :pr:`1690`), and a copied :class:`.DecimalTimestamp` keeps + ``orig_len`` under ``_pydecimal`` (:issue:`1647`, :pr:`1653`). * the third-party engines are best effort: the code each adapter carried only to match the default engine is reverted (:pr:`1718`, :pr:`1721`, :pr:`1724`, :pr:`1727`, closing :issue:`1708` through :pr:`1743`). What each engine leaves out, @@ -2579,17 +2707,17 @@ pcapkit.utilities Added ~~~~~ -* ``pcapkit.utilities.logging`` as a real interface: ``get_logger()`` for per-module +* :mod:`pcapkit.utilities.logging` as a real interface: ``get_logger()`` for per-module children, ``configure()`` to set level, handler, stream, format or propagation at runtime, ``reset()`` to return to library-neutral, and ``ensure_output()``. Each module that logs now does so under its own ``__name__``, so a consumer can silence - ``pcapkit.foundation.registry`` while keeping ``pcapkit.foundation.extraction`` + :mod:`pcapkit.foundation.registry` while keeping :mod:`pcapkit.foundation.extraction` (:pr:`384`). -* ``pcapkit.utilities.config``, one module for every tunable limit, each read once +* :mod:`pcapkit.utilities.config`, one module for every tunable limit, each read once from a ``PCAPKIT_*`` environment variable: the frame-layer and extension-header limits, the pypcapfile engine's TCP-over-IPv6 warning limit, the dumpkit fallback, the charset-detection cache, the read chunk and the zero-padding guards. A - bad value warns with ``ConfigWarning`` and falls back to the default; ``-1`` lifts + bad value warns with :exc:`.ConfigWarning` and falls back to the default; ``-1`` lifts a limit. The homes these limits had in the prereleases keep no re-export (:issue:`1726`, :pr:`1743`). @@ -2613,29 +2741,29 @@ Changed ``warnings.filterwarnings('ignore', category=BaseWarning)``. ``quiet=True`` now means no record at any level and no longer sets ``sys.tracebacklimit``, and the ``pcapkit.utilities.warnings.DEVMODE`` re-export is gone; its home is - ``pcapkit.utilities.logging``. + :mod:`pcapkit.utilities.logging`. Fixed ~~~~~ * further gaps the :issue:`514` keyword audit turned up, none covered by a test: - ``StreamEOFError``'s docstring did not say that ``@prepare`` always raises it with - ``quiet=True`` (the end-of-stream convention ``StructError`` follows via its own + :exc:`.StreamEOFError`'s docstring did not say that ``@prepare`` always raises it with + ``quiet=True`` (the end-of-stream convention :exc:`.StructError` follows via its own ``eof=True``), so nothing pinned that silence against regression; and ``register_extractor_engine``'s real keyword, ``name``, was not itself under test, only its corrected docstring, so a rename could put the two out of step again (:pr:`577`). -* ``hold_warnings`` is in ``pcapkit.utilities.warnings.__all__`` (:issue:`1602`, +* :class:`.hold_warnings` is in ``pcapkit.utilities.warnings.__all__`` (:issue:`1602`, :pr:`1603`). * the default threading excepthook is the fallback before Python 3.10, which has no ``threading.__excepthook__`` (:issue:`1691`, :pr:`1701`). * ``stacklevel()`` returns a level relative to its caller, as ``warnings.warn`` and - ``logging`` count it, so a warning or a ``BaseError`` log record names the + ``logging`` count it, so a warning or a :exc:`.BaseError` log record names the caller's frame at any stack depth rather than a frame inside ``logging`` or pcapkit (:pr:`397`). -* re-executing ``pcapkit.utilities.logging`` under ``PCAPKIT_DEVMODE`` keeps one +* re-executing :mod:`pcapkit.utilities.logging` under ``PCAPKIT_DEVMODE`` keeps one stderr handler, where each ``importlib.reload`` added another (:pr:`391`). -* a loud ``BaseError`` no longer sets the process-global ``sys.tracebacklimit``, +* a loud :exc:`.BaseError` no longer sets the process-global ``sys.tracebacklimit``, which truncated every later traceback, pcapkit's or not; lazily installed ``sys.excepthook`` and ``threading.excepthook`` keep its output terse instead (:pr:`983`). @@ -2674,10 +2802,10 @@ Fixed new guard at the crawler-emission layer requires a hard-coded ``_missing_`` body to end in a return and no emitted line to re-enter ``_missing_`` for the same value -- deliberately narrower than "every emitted line must return", which - ``Vendor.process``'s shape legitimately breaks. The guard sees only some crawlers; - the rest are covered behaviourally. Closes :issue:`866` (:pr:`867`). + :meth:`.Vendor.process`'s shape legitimately breaks. The guard sees only some + crawlers; the rest are covered behaviourally. Closes :issue:`866` (:pr:`867`). * ``python -m pcapkit.vendor `` could not fail: ``run()`` swallowed - every crawler exception as a filterable ``VendorRuntimeWarning`` and + every crawler exception as a filterable :exc:`.VendorRuntimeWarning` and ``main()`` always returned ``0``, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about @@ -2689,7 +2817,7 @@ Fixed filterable warning. **Per-target snapshot and restore**: a new ``_snapshot_and_restore`` context manager backs up a target's const file with ``tempfile.mkstemp`` before it runs (resolving the destination through - ``Vendor._dest_path``, now a ``classmethod`` in ``pcapkit/vendor/default.py`` + :meth:`.Vendor._dest_path`, now a ``classmethod`` in ``pcapkit/vendor/default.py`` so it can be called before ``Vendor.__init__`` fetches, renders or writes), restores the backup via ``os.replace`` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a @@ -2721,43 +2849,45 @@ Fixed from the ``except``. Left out of scope: an untested ``os.close(fd)`` leak (harmless, since ``copy2`` reopens the path) and the semantically identical unindented form of the same ``else:``. Closes :issue:`875` (:pr:`876`). -* ``TLSKeyLabel`` tracked the Mozilla NSS key-log wiki, but - ``draft-ietf-opsawg-pcapng-06`` section 4.7 now points at ``[I-D.ietf-tls-keylogfile]`` - (draft-05), published as :rfc:`9850`, "The SSLKEYLOGFILE Format for TLS", whose section 4.2 - creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, ``TLSKeyLabel`` - was missing ``ECH_SECRET`` and ``ECH_CONFIG`` (:rfc:`9850` sections 2.3 and 4.2), so - ``TLSKeyLabel('ECH_SECRET')`` raised ``ValueError`` on a legitimate Decryption - Secrets Block; both are added, ``ECH_SECRET`` keeping the file's ``# nosec B105`` - bandit convention for ``*_SECRET`` members. ``RSA`` has no row in :rfc:`9850`'s - registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and - is kept, since removing a public member is breaking, with a docstring comment - marking it the historical exception. ``pcapkit/vendor/pcapng/secrets_type.py``'s - stale ``# NSS Key Log Format`` comment is repointed at RFC 9850, and - ``TLSKeyLabel``'s docstring cites it. A new test pins the member set against RFC - 9850 plus the ``RSA`` exception. Closes :issue:`882` (:pr:`883`). +* :class:`~pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` tracked the Mozilla NSS + key-log wiki, but ``draft-ietf-opsawg-pcapng-06`` section 4.7 now points at + ``[I-D.ietf-tls-keylogfile]`` (draft-05), published as :rfc:`9850`, "The SSLKEYLOGFILE + Format for TLS", whose section 4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" + registry. Against it, :class:`~pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` was + missing ``ECH_SECRET`` and ``ECH_CONFIG`` (:rfc:`9850` sections 2.3 and 4.2), so + ``TLSKeyLabel('ECH_SECRET')`` raised ``ValueError`` on a legitimate Decryption Secrets + Block; both are added, ``ECH_SECRET`` keeping the file's ``# nosec B105`` bandit + convention for ``*_SECRET`` members. ``RSA`` has no row in :rfc:`9850`'s registry (it + is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since + removing a public member is breaking, with a docstring comment marking it the + historical exception. ``pcapkit/vendor/pcapng/secrets_type.py``'s stale + ``# NSS Key Log Format`` comment is repointed at RFC 9850, and + :class:`~pcapkit.const.pcapng.tls_key_label.TLSKeyLabel`'s docstring cites it. A new + test pins the member set against RFC 9850 plus the ``RSA`` exception. Closes + :issue:`882` (:pr:`883`). * both spellings of an IANA citation render as ``:rfc:`` roles in the generated - ``pcapkit.const`` docstrings (:issue:`1664`, :pr:`1673`). -* ``MAX_RETRY`` and ``CI_MODE`` stay in ``pcapkit.vendor.default``, read from + :mod:`pcapkit.const` docstrings (:issue:`1664`, :pr:`1673`). +* ``MAX_RETRY`` and ``CI_MODE`` stay in :mod:`pcapkit.vendor.default`, read from ``PCAPKIT_VENDOR_RETRY`` and ``PCAPKIT_CI_MODE``; a bad value warns with - ``ConfigWarning`` and falls back, where 1.4.x crashed on a non-integer and read + :exc:`.ConfigWarning` and falls back, where 1.4.x crashed on a non-integer and read ``0`` as ``1`` (:pr:`1745`). * the IPX socket crawler no longer scrapes Wikipedia, whose page dropped the socket table, so a regeneration stops failing on it (:issue:`507`, :pr:`511`). * the crawlers send a descriptive ``User-Agent`` built from the package metadata, which Wikipedia requires, and the PCAP-NG crawlers read draft ``-03``, the one revision that reproduces their constant files (:issue:`518`, :pr:`524`). -* ``pcapkit.vendor`` and ``pcapkit.vendor.mh`` list ``MH_HandoverInitiateStatus`` +* :mod:`pcapkit.vendor` and :mod:`pcapkit.vendor.mh` list ``MH_HandoverInitiateStatus`` in ``__all__``, where each repeated ``MH_HandoverACKStatus`` in its place, so a bare ``pcapkit-vendor`` run reaches its crawler (:pr:`531`). * with no terminal on stdin, a failed fetch re-raises rather than waiting for an operator to save the page, a wait that printed without end under a closed or piped stdin (:issue:`522`, :pr:`542`). -* each crawler derives its ``const/`` path from the ``pcapkit.vendor`` root, so a +* each crawler derives its ``const/`` path from the :mod:`pcapkit.vendor` root, so a crawler at any depth, such as ``vendor/reg/apptype/tcp.py``, writes to the right file (:pr:`741`). -* ``pcapkit.vendor.ipx`` names its ``sys.path`` temporary ``_path`` rather than +* :mod:`pcapkit.vendor.ipx` names its ``sys.path`` temporary ``_path`` rather than ``path``, which read as a public name of the package, and the public-API test - covers ``pcapkit.vendor``, which it used to skip (:issue:`1772`, :pr:`1779`). + covers :mod:`pcapkit.vendor`, which it used to skip (:issue:`1772`, :pr:`1779`). * the generated const comments render as written: a URL reference followed by another bracket no longer autolinks into it, a leading enumerator such as ``(S)`` is escaped, and a hyphenated word is no longer split across lines (:issue:`1846`, @@ -2782,8 +2912,8 @@ Added option whose declared length asks for more data than the capture holds, pinning that it still parses with the short read zero-padded rather than rejected. The test :issue:`431` left covers only an option area with no data behind it; a candidate fix - for :issue:`554` turned the untested half into an unwrapped ``FieldValueError`` while - the rest of the suite stayed green (:pr:`571`, :issue:`572`). + for :issue:`554` turned the untested half into an unwrapped :exc:`.FieldValueError` + while the rest of the suite stayed green (:pr:`571`, :issue:`572`). * ``CITATION.cff``, citation metadata in Citation File Format 1.2.0, which GitHub renders as the "Cite this repository" button and which citation managers and dependency inventories read directly. It is the machine-readable half of the @@ -2871,19 +3001,19 @@ Changed (``esp._resolve``, ``esp._CRYPTO``, ``ngap._convert``, ``ngap._revert``, ``ngap._PYCRATE``, ``ngap._PDU_LOCK``, ``pypcapfile._NamedStream``) were dropped, while ``autoattribute`` directives were added for class-private state that *is* - contract, among them ``Extractor._flag_f``, the ``PyPCAP`` and ``PCAP_CT`` engines' - ``_backend`` (the only third-party engines with one), ``TraceFlow``'s - internal fields, ``FieldBase``/``Field`` internals, the built-in ``PCAP`` engine's - ``_gbhdr``, ``_vinfo``, ``_nnsec`` and ``_dlink`` (shared with ``PyPCAP``, - ``PCAP_CT`` and ``PyPCAPFile``) and ``PCAPNG``'s ``_ctx``/``_ctx_list``. The - runtime definitions of ``_missing_`` (under ``pcapkit.const``, and inline in - ``pcapkit.protocols``) gained one write-up in place of a directive per class, a new + contract, among them :attr:`.Extractor._flag_f`, the :class:`.PyPCAP` and + :class:`.PCAP_CT` engines' ``_backend`` (the only third-party engines with one), + :class:`.TraceFlow`'s internal fields, :class:`.FieldBase`/:class:`.Field` internals, + the built-in :class:`.PCAP` engine's ``_gbhdr``, ``_vinfo``, ``_nnsec`` and ``_dlink`` + (shared with :class:`.PyPCAP`, :class:`.PCAP_CT` and :class:`.PyPCAPFile`) and + :class:`~pcapkit.foundation.engines.pcapng.PCAPNG`'s ``_ctx``/``_ctx_list``. The + runtime definitions of ``_missing_`` (under :mod:`pcapkit.const`, and inline in + :mod:`pcapkit.protocols`) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in ``docs/source/pcapkit/const/index.rst`` cross-referenced from ``registry.rst`` (``conf.py`` already excludes ``_missing_`` - from every ``autoclass``). ``CONTRIBUTING.md`` gained the rule as a named section. - The new Sphinx warning, a bare ``Type`` newly rendered by - ``TraceFlow._foutio``'s directive, is :issue:`709`. No line under ``pcapkit/`` - changed (:issue:`684`). + from every ``autoclass``). ``CONTRIBUTING.md`` gained the rule as a named section. The + new Sphinx warning, a bare ``Type`` newly rendered by :attr:`.TraceFlow._foutio`'s + directive, is :issue:`709`. No line under ``pcapkit/`` changed (:issue:`684`). * ``examples/captures/out.json``, ``out.plist``, ``out.txt`` and ``pcapng.txt`` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. ``pcapng.txt`` is exactly that: it still recorded @@ -2921,14 +3051,15 @@ Changed the new ``(flag, job)`` pairs, each expected (:issue:`779`). * ``engine='pyshark'`` had no CI job driving it through a real ``tshark``-parsed capture. ``pypcap-parity`` installs ``pyshark`` but its ``apt-get install`` named no - ``tshark`` package, so ``PyShark.unsupported_reason()`` declined the engine, the + ``tshark`` package, so :meth:`.PyShark.unsupported_reason()` declined the engine, the extractor fell back to the built-in parser, and the only real-extraction assertion, ``assertGreater(extractor.length, 0)``, passed against the fallback's output: a vacuous pass invisible in every skip count. ``pypcap-parity`` now installs ``tshark`` with the debconf pre-seed ``engine-tests`` carries, and the test proves which engine ran (no - ``EngineWarning``, ``extractor._exnam == 'pyshark'``, a ``PyShark`` instance), as the - PyPCAP parity test does. Only ``tests/integration/test_engine_runtime.py`` and - ``test_engine_parity.py`` change; the length assertion is kept (:pr:`846`). + :exc:`.EngineWarning`, ``extractor._exnam == 'pyshark'``, a :class:`.PyShark` + instance), as the PyPCAP parity test does. Only + ``tests/integration/test_engine_runtime.py`` and ``test_engine_parity.py`` change; the + length assertion is kept (:pr:`846`). * ``engine-tests`` ran a Python-version leg per interpreter regardless of which engines that interpreter could install, so ``pyproject.toml``'s ``PyPCAPFile = ["pypcapfile; python_version < '3.12'"]`` marker resolved to @@ -2961,14 +3092,14 @@ Changed change; the ``Compat`` names are excluded from the replacement. Only ``.github/workflows/unit-tests.yml`` changes; no library or test code (:pr:`856`). * the prose the post-:issue:`719` defect wave added or touched, checked against the - code; no code changes. ``MesgIDOption.seconds`` in the Mobility Header schema said - seconds since 1970 where it holds an :rfc:`1305` NTP count from 1900; - ``HTTP._guess_version`` comments predated the HTTP/2 gate; and the - ``_parse_next_layer`` note miscounted the captured octets. ``Extractor._open_output`` - now says only the PCAP engine passes header fields, ``Extractor.run`` and - the PCAP-NG engine's ``run`` list the ``FormatError`` for PCAP output from a PCAP-NG - input, and the extraction and engine pages document the PCAP-output path (:pr:`1179`, - :pr:`1180`, :pr:`1186`, :issue:`1103`). + code; no code changes. :attr:`.MesgIDOption.seconds` in the Mobility Header schema + said seconds since 1970 where it holds an :rfc:`1305` NTP count from 1900; + :meth:`.HTTP._guess_version` comments predated the HTTP/2 gate; and the + ``_parse_next_layer`` note miscounted the captured octets. + :meth:`.Extractor._open_output` now says only the PCAP engine passes header fields, + :meth:`.Extractor.run` and the PCAP-NG engine's ``run`` list the :exc:`.FormatError` + for PCAP output from a PCAP-NG input, and the extraction and engine pages document the + PCAP-output path (:pr:`1179`, :pr:`1180`, :pr:`1186`, :issue:`1103`). * **a breaking change to** the ``all`` extra: it carries the core addons only -- ``cli``, ``crypto`` and ``NGAP``. The third-party engines 1.4.x bundled in it (``dpkt``, ``scapy`` and ``pyshark``) and the vendor crawler's ``requests`` and @@ -3001,7 +3132,7 @@ Fixed * places where a documentation page contradicted the code (:pr:`413`), ambiguous cross-references and autodoc signature failures (:pr:`416`), and - ``Extractor``'s documented exception plus phantom or stale ``Args:`` labels + :class:`.Extractor`'s documented exception plus phantom or stale ``Args:`` labels (:pr:`501`). * both halves of what the ``Changelog drift`` gate told an author, in ``util/changelog_md.py``. ``ResidualMarkupError`` named the entry file but numbered @@ -3016,7 +3147,7 @@ Fixed Sphinx's own (:issue:`592`). Regenerating ``CHANGELOG.md`` is byte-identical over every committed entry: the fix changes what the gate *says*, not what it emits. * ``examples/generators/options.py``'s ``TCP_BASE`` spelled header fields with - names ``TCP.make`` does not declare, so every option fixture in + names :meth:`.TCP.make` does not declare, so every option fixture in ``examples/captures/options-tcp.pcap`` was built from values the generator had not asked for. ``make`` ends in ``**kwargs`` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. ``'seq': 1`` was @@ -3029,7 +3160,7 @@ Fixed ``options-tcp.pcap`` changes only in the low octet of each frame's sequence number, with no case status or ``EXPECTED_FAILURES`` entry moving. The module's other base mappings were audited: the rest pass nothing their ``make`` does - not declare, and HIP's ``extension`` is declared by ``HIP.read``, this library's + not declare, and HIP's ``extension`` is declared by :meth:`.HIP.read`, this library's documented way of forwarding a read-path keyword, not another instance (:issue:`602`). * ``util/bump_version.py`` left ``CITATION.cff`` naming the previous release. Nothing @@ -3163,10 +3294,12 @@ Fixed trusts ``committed_captures()`` for the tracked set. A hardcoded literal that happens to match the tracked names still passes; the test catches a set that is wrong when it runs (:issue:`708`). -* ``pcapkit.all.__all__`` listed ``OSPF``, ``RARP`` and ``DRARP`` on its Link Layer - line after :issue:`719` moved them (:issue:`1124`), and omitted names its packages - export, among them ``ESP``, ``HTTPv1``, ``HTTPv2``, ``PCAPNG`` and ``EnumRegistry``, - so ``from pcapkit.all import *`` did not bind them. ``pcapkit.utilities`` stays +* ``pcapkit.all.__all__`` listed :class:`~pcapkit.protocols.application.ospf.OSPF`, + :class:`.RARP` and :class:`.DRARP` on its Link Layer line after :issue:`719` moved + them (:issue:`1124`), and omitted names its packages export, among them + :class:`~pcapkit.protocols.internet.esp.ESP`, ``HTTPv1``, ``HTTPv2``, + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` and :class:`.EnumRegistry`, so + ``from pcapkit.all import *`` did not bind them. :mod:`pcapkit.utilities` stays unlisted deliberately, so generic helpers such as ``warn`` do not arrive through ``import *`` (:issue:`1136`). * ``examples/generators/options.py`` wrote every HTTP/2 sample frame on stream 0, which @@ -3192,15 +3325,15 @@ Fixed attributes several packages left out are listed (:issue:`533`, :pr:`544`). * the documentation build's "more than one target found" warnings, fixed where they arise: an implicit reference to ``dict``, ``type`` or ``set`` no longer - matches a registry member such as ``TCP.dict``, an enum member's docstring holding - a URL no longer gains a spurious type field, and the hand-written ``TypeVar`` + matches a registry member such as :attr:`.TCP.dict`, an enum member's docstring + holding a URL no longer gains a spurious type field, and the hand-written ``TypeVar`` directives take the bare name under their ``.. module::`` (:issue:`1775`, :pr:`1790`). * the Mermaid diagrams render at their natural size, shrunk only to the page width, where each was fitted into a fixed box, and their ``click`` targets are relative to the page, so they resolve under the site's ``/PyPCAPKit/`` prefix (:issue:`1103`, :pr:`1819`). -* the API reference renders what the code declares. ``Extractor._vfunc`` keeps its +* the API reference renders what the code declares. :attr:`.Extractor._vfunc` keeps its annotation (:issue:`1839`, :pr:`1867`), and the HTTP/2 frame schemas' and the engine classes' annotations resolve (:issue:`1857`, :pr:`1870`). Under the pinned sphinx-autodoc-typehints 3.15.0 (:issue:`1856`, :pr:`1877`), ``conf.py`` renders an diff --git a/tests/project/test_changelog_md.py b/tests/project/test_changelog_md.py index cb566a468..3885c261c 100644 --- a/tests/project/test_changelog_md.py +++ b/tests/project/test_changelog_md.py @@ -420,19 +420,104 @@ def test_markdown_metacharacters_are_escaped_in_text_and_target(self) -> None: def test_an_unknown_role_is_still_fatal(self) -> None: # Shows the guard was not loosened; it is not evidence of new behaviour. The - # ``:mod:`` half passed before the change too -- only the ``:issue:`` half + # ``:ref:`` half passed before the change too -- only the ``:issue:`` half # failed without it, by being left unconverted. index = self.make_tree( - entry=ENTRY + '\nSee :issue:`29` and :mod:`pcapkit.const` for more.\n') + entry=ENTRY + '\nSee :issue:`29` and :ref:`changelog` for more.\n') with self.assertRaises(changelog_md.ResidualMarkupError) as error: changelog_md.render(index) self.assertIn('role', str(error.exception)) - self.assertIn(':mod:', str(error.exception)) + self.assertIn(':ref:', str(error.exception)) self.assertNotIn(':issue:', str(error.exception)) +class PythonRoleTests(ChangelogTreeMixin, unittest.TestCase): + """Rule 2's Python-domain half: ``:class:``, ``:meth:`` and the rest become code spans. + + #1941: the 1.5.0 entry links its names to the API, so each Python-domain role + has to reach the Markdown as the name Sphinx displays -- not as raw role text, + which the guard refuses, and not as the link target when the two differ. + + """ + + def body(self, line: str) -> str: + return self.convert(ENTRY + f'\n{line}\n') + + def test_each_role_becomes_a_code_span_of_its_name(self) -> None: + cases = { + ':class:`pcapkit.foundation.extraction.Extractor`': + '`pcapkit.foundation.extraction.Extractor`', + ':exc:`pcapkit.utilities.exceptions.ProtocolError`': + '`pcapkit.utilities.exceptions.ProtocolError`', + ':attr:`pcapkit.foundation.extraction.Extractor.engine`': + '`pcapkit.foundation.extraction.Extractor.engine`', + ':data:`pcapkit.interface.core.PCAPKit`': '`pcapkit.interface.core.PCAPKit`', + ':mod:`pcapkit.const`': '`pcapkit.const`', + ':meth:`pcapkit.foundation.extraction.Extractor.run`': + '`pcapkit.foundation.extraction.Extractor.run()`', + ':func:`pcapkit.interface.extract`': '`pcapkit.interface.extract()`', + } + # One assertion over every role rather than a ``subTest`` per role: under + # this repository's pytest a failing subtest leaves its parent ``PASSED``. + markdown = self.body('\n\n'.join(f'See {role} here.' for role in cases)) + missing = [role for role, expected in cases.items() + if f'See {expected} here.' not in markdown] + + self.assertEqual(missing, []) + self.assertEqual(changelog_md.residual(markdown), []) + + def test_a_tilde_keeps_only_the_last_component(self) -> None: + markdown = self.body(':class:`~pcapkit.foundation.extraction.Extractor` and ' + ':meth:`~pcapkit.foundation.extraction.Extractor.run`.') + + self.assertIn('`Extractor` and `run()`.', markdown) + self.assertNotIn('pcapkit.foundation', markdown) + + def test_a_leading_dot_is_dropped_from_the_name(self) -> None: + markdown = self.body(':exc:`.ProtocolError`, :meth:`.Extractor.run` and ' + ':class:`~.Extractor`.') + + self.assertIn('`ProtocolError`, `Extractor.run()` and `Extractor`.', markdown) + + def test_an_explicit_title_is_shown_as_written(self) -> None: + markdown = self.body(':meth:`AppType.get `' + ' and :class:`TCP() `.') + + self.assertIn('`AppType.get` and `TCP()`.', markdown) + self.assertNotIn(' None: + markdown = self.body(':meth:`.Extractor.run()` and :func:`!print`.') + + self.assertIn('`Extractor.run()` and `print()`.', markdown) + self.assertNotIn('()()', markdown) + + def test_a_suppressed_link_keeps_the_rest_as_written(self) -> None: + # Sphinx does no tilde or dot processing behind a ``!``. + markdown = self.body(':class:`!~pcapkit.Extractor`.') + + self.assertIn('`~pcapkit.Extractor`.', markdown) + + def test_the_domain_prefixed_spelling_is_converted_too(self) -> None: + markdown = self.body(':py:class:`~pcapkit.foundation.extraction.Extractor`.') + + self.assertIn('`Extractor`.', markdown) + self.assertNotIn(':py:', markdown) + + def test_a_role_entry_renders_without_residual_markup(self) -> None: + index = self.make_tree( + entry=ENTRY + '\n* raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`\n' + ' from :meth:`.Extractor.run`.\n') + + rendered = changelog_md.render(index) + + self.assertIn('- raises `ProtocolError` from `Extractor.run()`.', rendered) + self.assertNotIn(':exc:', rendered) + self.assertNotIn(':meth:', rendered) + + class ResidualMarkupTests(ChangelogTreeMixin, unittest.TestCase): """The guard against six regexes quietly copying through what they cannot convert.""" @@ -445,7 +530,7 @@ def test_an_unconvertible_directive_is_fatal(self) -> None: self.assertIn('directive', str(error.exception)) def test_an_unconverted_role_is_fatal(self) -> None: - index = self.make_tree(entry=ENTRY + '\nSee :mod:`pcapkit.const` for more.\n') + index = self.make_tree(entry=ENTRY + '\nSee :ref:`changelog` for more.\n') with self.assertRaises(changelog_md.ResidualMarkupError) as error: changelog_md.render(index) @@ -629,17 +714,17 @@ def cited(self, complaints: list[str]) -> list[int]: return numbers def test_a_cited_line_holds_the_construct_it_was_cited_for(self) -> None: - entry = padded_entry('and then :data:`sys.modules` at the very end.') + entry = padded_entry('and then :term:`sys.modules` at the very end.') number, = self.cited(self.complaints(entry)) - self.assertIn(':data:`sys.modules`', entry.split('\n')[number - 1], + self.assertIn(':term:`sys.modules`', entry.split('\n')[number - 1], f'line {number} of the entry does not hold the cited construct') def test_a_cited_line_can_lie_past_the_end_of_the_converted_body(self) -> None: # The bound that made the old numbers provably wrong: they were counted in # the body, so they could never reach a construct written below it. - entry = padded_entry('and then :data:`sys.modules` at the very end.') + entry = padded_entry('and then :term:`sys.modules` at the very end.') body = changelog_md.convert(entry) number, = self.cited(self.complaints(entry)) @@ -653,20 +738,20 @@ def test_a_cited_line_can_lie_past_the_end_of_the_converted_body(self) -> None: def test_constructs_joined_onto_one_line_keep_their_own_lines(self) -> None: entry = padded_entry( - 'first :data:`sys.modules`,', - 'then :func:`importlib.import_module`,', - 'and last :class:`dict`.', + 'first :term:`sys.modules`,', + 'then :envvar:`importlib.import_module`,', + 'and last :ref:`dict`.', ) joined = [line for line in changelog_md.convert(entry).split('\n') - if ':data:' in line] + if ':term:' in line] self.assertEqual(len(joined), 1, 'the three roles were meant to be joined') - self.assertIn(':class:', joined[0], 'the three roles were meant to be joined') + self.assertIn(':ref:', joined[0], 'the three roles were meant to be joined') numbers = self.cited(self.complaints(entry)) self.assertEqual(len(set(numbers)), 3, f'three constructs on three source lines cited {numbers}') - for number, role in zip(numbers, (':data:', ':func:', ':class:')): + for number, role in zip(numbers, (':term:', ':envvar:', ':ref:')): self.assertIn(role, entry.split('\n')[number - 1], f'line {number} does not hold {role}') @@ -720,11 +805,11 @@ def test_residual_without_a_source_map_says_which_frame_it_counted(self) -> None # ``residual`` is callable on a bare string, and then there is nothing to map # through. It has to admit that rather than pass a body line number off as a # line of an entry -- which is precisely what #588 was. - markdown = changelog_md.convert(ENTRY + '\nSee :mod:`pcapkit.const` for more.\n') + markdown = changelog_md.convert(ENTRY + '\nSee :ref:`changelog` for more.\n') unmapped = changelog_md.residual(markdown) mapped = changelog_md.residual( - *changelog_md.convert_traced(ENTRY + '\nSee :mod:`pcapkit.const` for more.\n')) + *changelog_md.convert_traced(ENTRY + '\nSee :ref:`changelog` for more.\n')) self.assertEqual(len(unmapped), 1) self.assertTrue(unmapped[0].startswith('converted line '), unmapped[0]) diff --git a/util/changelog_md.py b/util/changelog_md.py index 9bc8885dc..7d1e44e1d 100644 --- a/util/changelog_md.py +++ b/util/changelog_md.py @@ -48,7 +48,9 @@ Sphinx accepts too -- become Markdown links to the RFC on the IETF datatracker, and every role in the ``extlinks`` table of ``docs/source/conf.py`` (``:issue:``, ``:pr:``, ``:wikipedia:`` ...), - mirrored in ``EXTLINKS``, becomes a link to that role's URL; + mirrored in ``EXTLINKS``, becomes a link to that role's URL, and the + Python-domain roles in ``PY_ROLES`` (``:class:``, ``:meth:`` ...) become + code spans of the name Sphinx displays -- ``:class:`~a.b.C``` is ```C```; 3. ``double backtick`` literals become single-backtick code spans; 4. ``*`` bullets become ``-`` bullets; 5. ``[n]_`` / ``.. [n]`` footnotes become GitHub's ``[^n]`` / ``[^n]:``; @@ -74,7 +76,7 @@ believed to be broader than it is, is worse than no guard: * a ``double backtick`` literal rule 3 did not reach; - * an interpreted-text role, ``:mod:`x```, rule 2 does not know; + * an interpreted-text role, ``:ref:`x```, rule 2 does not know; * a directive or comment, any line opening ``..`` -- which also catches the ``.. _name: url`` target that a bare ``name_`` reference needs, so that form is covered through its definition rather than through the reference; @@ -210,6 +212,24 @@ #: ``:issue:`the parsing bug <591>``` is the text *the parsing bug* linking to 591. _EXPLICIT_TITLE = re.compile(r'^(.+?)\s*(?$', re.DOTALL) +#: The Python-domain cross-reference roles rule 2 renders, each mapped to whether +#: Sphinx's role is built with ``fix_parens`` -- ``sphinx.domains.python`` +#: (Sphinx 9.1.0) sets it on ``:func:`` and ``:meth:`` only, which is what puts +#: ``()`` after the displayed name of a callable. +PY_ROLES = { + 'attr': False, + 'class': False, + 'data': False, + 'exc': False, + 'func': True, + 'meth': True, + 'mod': False, +} # type: dict[str, bool] + +#: Every :data:`PY_ROLES` role, bare or with its ``py:`` domain prefix. +_PY_ROLE = re.compile( + r':(?:py:)?(' + '|'.join(sorted(PY_ROLES, key=len, reverse=True)) + r'):`([^`\n]+)`') + #: Where the full rendered history lives. Must match ``[project.urls].changelog`` #: in ``pyproject.toml`` -- ``MANIFEST.in`` prunes ``docs/``, so in a source #: distribution this is the only route from the shipped entry to the rest of the @@ -239,7 +259,7 @@ #: Stands in for one character of a masked code span. A *marker* rather than #: nothing, because the difference between a leftover role and prose about a role #: is precisely whether a code span follows the ``:name:`` or encloses it: -#: ``:mod:`pcapkit.const``` is a role that escaped rule 2, and ```:mod:``` is a +#: ``:ref:`changelog``` is a role that escaped rule 2, and ```:mod:``` is a #: sentence mentioning one. #: #: A span is masked character for character rather than collapsed to one marker, @@ -467,6 +487,48 @@ def extlink(name: str, text: str, return f'[{title}]({url})' +def py_role(name: str, text: str) -> str: + """Render one Python-domain role as a code span of the name Sphinx displays. + + A code span rather than a link: the page that documents an object is known only + to a built ``objects.inv``, and this script runs where there is no + documentation build -- a source distribution prunes ``docs/`` altogether. + + The displayed name follows Sphinx's ``XRefRole`` and ``PyXRefRole`` + (Sphinx 9.1.0), so an entry names the same thing in the Markdown as in the + rendered documentation: + + * an explicit ``title `` displays *title*, as written; + * a leading ``!`` suppresses the link, and the rest displays as written; + * otherwise a leading ``.`` is dropped, and a leading ``~`` keeps only the + last dotted component -- ``~pcapkit.foundation.extraction.Extractor`` is + ``Extractor``; + * ``:func:`` and ``:meth:`` gain ``()`` unless the name already ends in it, + since ``add_function_parentheses`` is left at its default in + ``docs/source/conf.py``. + + Args: + name: The role name, a key of :data:`PY_ROLES`. + text: The role content, between the backticks. + + Returns: + A Markdown code span. + + """ + match = _EXPLICIT_TITLE.match(text) + if match: + title = match.group(1) + else: + title = text[1:] if text.startswith('!') else text + if PY_ROLES[name] and not title.endswith('()'): + title += '()' + if not text.startswith('!'): + title = title.lstrip('.') + if title.startswith('~'): + title = title[1:].rpartition('.')[2] + return f'`{title}`' + + def convert(rst: str) -> str: """Apply the conversion rules to one per-version entry. @@ -528,9 +590,11 @@ def convert_traced(rst: str) -> tuple[str, list[tuple[int, int]]]: continue # 2. the roles these entries use: ``:rfc:`` in both spellings Sphinx - # accepts, and every role in :data:`EXTLINKS`. + # accepts, every role in :data:`EXTLINKS`, and the Python-domain + # roles in :data:`PY_ROLES`. line = _RFC_ROLE.sub(lambda match: rfc_link(match[1], match[2] or ''), line) line = _EXTLINK_ROLE.sub(lambda match: extlink(match[1], match[2]), line) + line = _PY_ROLE.sub(lambda match: py_role(match[1], match[2]), line) # 3. literals. line = re.sub(r'``([^`]+)``', r'`\1`', line) # 4. bullets, at any indent.