diff --git a/CHANGELOG.md b/CHANGELOG.md index b25003ec2..029649aeb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ ## 1.5.0 -- unreleased -The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing 685 issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#1791](https://github.com/JarryShaw/PyPCAPKit/pull/1791). +The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#1828](https://github.com/JarryShaw/PyPCAPKit/pull/1828). Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and `1.5.0b4` to `1.5.0b12` (2026-09-20 to 2026-10-13), dated in UTC and all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` reshipped it -- an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels ([#497](https://github.com/JarryShaw/PyPCAPKit/pull/497), [#498](https://github.com/JarryShaw/PyPCAPKit/pull/498)). @@ -8,10 +8,10 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Changed -- **a breaking change to** `pcapkit.const.reg.apptype`. `AppType` becomes a memberless base over one registry per transport protocol -- `TCP`, `UDP`, `SCTP` and `DCCP` -- because `aenum` refuses to subclass an enumeration that has members. `isinstance(TCP.http, AppType)` still holds and `AppType.get(port, proto=...)` proxies to the registry, so nothing under `pcapkit.protocols` changes. **The 1,004 rows IANA assigns no port, and the 704 it assigns no transport protocol, stop being members**; they become list-table docstrings on the registry that owns them. `__members_proto__` is now a per-registry `MultiDict` keyed on port, so all three services IANA registers on port 80 survive. `get()` answers with the canonical service, curated from `/etc/services` because IANA names no precedence, and `get_all()` reaches the aliases ([#754](https://github.com/JarryShaw/PyPCAPKit/pull/754)). -- the six `%`-formatted `raise ValueError(...)` calls in the generated `pcapkit.const.reg.apptype.apptype`, left by [#759](https://github.com/JarryShaw/PyPCAPKit/issues/759), are f-strings ([#792](https://github.com/JarryShaw/PyPCAPKit/issues/792)). The remainder followed: three `_missing_` guards (`pcapkit.vendor.tcp.flags`, `.ftp.command`, `.http.method`), `AppType`'s `__new__`/`__repr__`/`__str__`, and the `extend_enum` fallbacks in `get()` and in the span-handling tail. The `consider-using-f-string` disable drops from `tcp.flags` and `reg.apptype.apptype` and stays on `ftp.command` and `http.method`, whose `__repr__` still used `%`. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows ([#798](https://github.com/JarryShaw/PyPCAPKit/issues/798)). -- the three `__repr__` methods [#798](https://github.com/JarryShaw/PyPCAPKit/issues/798) left on `%` (`pcapkit/const/ftp/command.py:40,119`, `pcapkit/const/http/method.py:42`) are f-strings, so the disable drops from both files and both `pcapkit/vendor/` templates. The other bespoke templates (`{const,vendor}/ftp/return_code.py`, `{const,vendor}/http/status_code.py`, `vendor/http/frame.py`) and `vendor/default.py` still carry it, so [#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)'s claim that nothing in `pcapkit/{const,vendor}/{ftp,http}/` needs it holds for this pair only; `repr()` output is unchanged ([#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)). -- `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)'s (above) 53 preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a `'..._0x%s' % hex(value)[2:].upper().zfill(4)` expression, and the exact `exempt_hits == 53` bound stays. Test-only. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). +- **a breaking change to** `pcapkit.const.reg.apptype`. `AppType` becomes a memberless base over one registry per transport protocol -- `TCP`, `UDP`, `SCTP` and `DCCP` -- because `aenum` refuses to subclass an enumeration that has members. `isinstance(TCP.http, AppType)` still holds and `AppType.get(port, proto=...)` proxies to the registry, so nothing under `pcapkit.protocols` changes. **The rows IANA assigns no port, and those it assigns no transport protocol, stop being members**; they become list-table docstrings on the registry that owns them. `__members_proto__` is now a per-registry `MultiDict` keyed on port, so every service IANA registers on port 80 survives. `get()` answers with the canonical service, curated from `/etc/services` because IANA names no precedence, and `get_all()` reaches the aliases ([#754](https://github.com/JarryShaw/PyPCAPKit/pull/754)). +- the `%`-formatted `raise ValueError(...)` calls in the generated `pcapkit.const.reg.apptype.apptype`, left by [#759](https://github.com/JarryShaw/PyPCAPKit/issues/759), are f-strings ([#792](https://github.com/JarryShaw/PyPCAPKit/issues/792)). The remainder followed: three `_missing_` guards (`pcapkit.vendor.tcp.flags`, `.ftp.command`, `.http.method`), `AppType`'s `__new__`/`__repr__`/`__str__`, and the `extend_enum` fallbacks in `get()` and in the span-handling tail. The `consider-using-f-string` disable drops from `tcp.flags` and `reg.apptype.apptype` and stays on `ftp.command` and `http.method`, whose `__repr__` still used `%`. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows ([#798](https://github.com/JarryShaw/PyPCAPKit/issues/798)). +- the `__repr__` methods [#798](https://github.com/JarryShaw/PyPCAPKit/issues/798) left on `%` (in `pcapkit/const/ftp/command.py` and `pcapkit/const/http/method.py`) are f-strings, so the disable drops from both files and both `pcapkit/vendor/` templates. The other bespoke templates (`{const,vendor}/ftp/return_code.py`, `{const,vendor}/http/status_code.py`, `vendor/http/frame.py`) and `vendor/default.py` still carry it, so [#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)'s claim that nothing in `pcapkit/{const,vendor}/{ftp,http}/` needs it holds for this pair only; `repr()` output is unchanged ([#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)). +- `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)'s (above) preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a `'..._0x%s' % hex(value)[2:].upper().zfill(4)` expression, and the exact `exempt_hits == 53` bound stays. Test-only. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). - **a breaking change to** `AppType.proto`: it is the single transport of the registry the member lives in, not every transport IANA assigned the service, and `register_apptype` takes its transports as varargs ([#806](https://github.com/JarryShaw/PyPCAPKit/issues/806), [#815](https://github.com/JarryShaw/PyPCAPKit/pull/815)). Aliases show in `repr()` and `str()`, and can be registered per transport ([#807](https://github.com/JarryShaw/PyPCAPKit/issues/807), [#837](https://github.com/JarryShaw/PyPCAPKit/pull/837)). - **a breaking change to** `TransportProtocol`: its base moves from `aenum.IntFlag` to `aenum.IntEnum`, so `|` between two members gives a plain `int` rather than a composite member ([#808](https://github.com/JarryShaw/PyPCAPKit/issues/808), [#836](https://github.com/JarryShaw/PyPCAPKit/pull/836)). - **a breaking change to** `TLSKeyLabel`: the hand-written class in `pcapkit.protocols.misc.pcapng` becomes the generated `pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` (`PCAPNG_TLSKeyLabel` in `pcapkit.const.pcapng`), crawled from the IANA registry [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) creates ([#886](https://github.com/JarryShaw/PyPCAPKit/issues/886), [#890](https://github.com/JarryShaw/PyPCAPKit/pull/890)), and the old path is removed ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). @@ -21,34 +21,36 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Fixed -- constant lookups that rejected a value the registry defines. `RouterAlert(0)` is the only value [RFC 2113](https://datatracker.ietf.org/doc/html/rfc2113) defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPX `Socket(0)` is that protocol's own default, so `bytes(IPX(...))` crashed on its own defaults; and two FTP `_missing_` overrides were plain methods rather than classmethods, so every unregistered value raised `TypeError` instead of extending the enumeration ([#492](https://github.com/JarryShaw/PyPCAPKit/issues/492), [#503](https://github.com/JarryShaw/PyPCAPKit/pull/503)). +- constant lookups that rejected a value the registry defines. `RouterAlert(0)` is the only value [RFC 2113](https://datatracker.ietf.org/doc/html/rfc2113) defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPX `Socket(0)` is that protocol's own default, so `bytes(IPX(...))` crashed on its own defaults; and the FTP `_missing_` overrides were plain methods rather than classmethods, so every unregistered value raised `TypeError` instead of extending the enumeration ([#492](https://github.com/JarryShaw/PyPCAPKit/issues/492), [#503](https://github.com/JarryShaw/PyPCAPKit/pull/503)). - the string-keyed `get()` in `pcapkit.const.ftp.command` and `pcapkit.const.http.method` tested membership with the raw key but registered `key.upper()`, so the *first* lowercase or mixed-case token raised `TypeError: 'RETR' already in use`. Reachable from wire data: `pcapkit.protocols.application.ftp` matches with `re.I` and passes the match verbatim, and [RFC 959 Section 5](https://datatracker.ietf.org/doc/html/rfc959#section-5) makes FTP commands case-insensitive, so `retr file.txt` was a valid request the library could not parse. `get()` and `_missing_` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate `GET` would have lacked the real one's `safe` and `idempotent` attributes ([#582](https://github.com/JarryShaw/PyPCAPKit/issues/582), [#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). `http.method` no longer folds case: [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1) makes the method token case-sensitive, so `Method.get` matches it exactly ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), below). -- `get()`'s documented `default` was ignored on the integer path across the generated `pcapkit.const` tree, because `_missing_` cannot see the caller's `default` -- `Hardware.get(99999, 0)` raised `ValueError` instead of returning `0`. The integer path now consults `default` before letting the error escape. `-1`, the placeholder the generated signature already carried, still separates "no default" from "a default"; [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859) replaced it with an identity sentinel once the shared base [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below) put `get()` behind one call site. The sweep covers 110 of the 118 integer registries, not the three [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584) named; `pcapng` `OptionType` and `reg` `AppType` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix ([#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)). -- `_missing_` in the four Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). All four now end in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and most const modules ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s two new ones included) already did, so `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. Minting through `extend_enum` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a member per bit pattern. The range guard is untouched ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). -- six of the 123 constant registries rejected an invalid value differently from the built-in `enum`, and three did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. All six now raise a bare `ValueError`, as `enum.IntEnum` does and 113 of the 117 modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those 113 with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in all 113 generated `get()` bodies (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register; the [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) and [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) entries below record how all three came to return an unregistered member instead. The three flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed; `Flags` accepts `0 <= value <= 0xFFFF`. A new `tests/const/test_const_enum_builtin_parity.py` asserts that every one of the 123 registries raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how three of the six escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). -- **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) (below) made it an unregistered member. `Transport._make_port` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk.** Separately, the crawler rendered one `_missing_` branch per registry row, so the two spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the 762 naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). -- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` was still an `aenum.IntFlag` and a member's `proto` carried the whole set IANA assigned the service, so `tcp | udp` was an ordinary value to pass back in; the `AppType.proto` and `TransportProtocol` entries above record how [#806](https://github.com/JarryShaw/PyPCAPKit/issues/806) and [#808](https://github.com/JarryShaw/PyPCAPKit/issues/808) later retired both. Over every multi-transport member this misrouted two: port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). -- `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the two shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. All 105 generated registries are updated; the non-minting `get()` miss applies to all 105, but the `_missing_` fix changes behaviour only for the **21** with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; four reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a two-entry `FILTER_NAME_TO_LINKTYPE` (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. The other minting call sites in `_missing_`, `get` and the hand-written `mh.py` and `ngap.py` were deferred; the 106 `register`/`register_alias` sites are not deferred but the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). -- **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: three of its five archived range branches were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 values that** `_missing_` **resolves (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). +- `get()`'s documented `default` was ignored on the integer path across the generated `pcapkit.const` tree, because `_missing_` cannot see the caller's `default` -- `Hardware.get(99999, 0)` raised `ValueError` instead of returning `0`. The integer path now consults `default` before letting the error escape. `-1`, the placeholder the generated signature already carried, still separates "no default" from "a default"; [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859) replaced it with an identity sentinel once the shared base [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below) put `get()` behind one call site. The sweep covers the integer registries generated from the shared template, not only those [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584) named; `pcapng` `OptionType` and `reg` `AppType` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix ([#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)). +- `_missing_` in the Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). Each now ends in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and most const modules ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s new ones included) already did, so `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. Minting through `extend_enum` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a member per bit pattern. The range guard is untouched ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). +- constant registries that rejected an invalid value differently from the built-in `enum`, or did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. These now raise a bare `ValueError`, as `enum.IntEnum` does and the other modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in every generated `get()` body (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register; the [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) and [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) entries below record how each came to return an unregistered member instead. The flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed; `Flags` accepts `0 <= value <= 0xFFFF`. A new `tests/const/test_const_enum_builtin_parity.py` asserts that every registry raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how the `IntFlag` ones escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). +- **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) (below) made it an unregistered member. `Transport._make_port` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk.** Separately, the crawler rendered one `_missing_` branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the branches naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). +- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` was still an `aenum.IntFlag` and a member's `proto` carried the whole set IANA assigned the service, so `tcp | udp` was an ordinary value to pass back in; the `AppType.proto` and `TransportProtocol` entries above record how [#806](https://github.com/JarryShaw/PyPCAPKit/issues/806) and [#808](https://github.com/JarryShaw/PyPCAPKit/issues/808) later retired both. Over every multi-transport member this misrouted port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). +- `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. Every generated registry is updated; the non-minting `get()` miss applies to all of them, but the `_missing_` fix changes behaviour only for **those** with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; the reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a `FILTER_NAME_TO_LINKTYPE` map (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. The other minting call sites in `_missing_`, `get` and the hand-written `mh.py` and `ngap.py` were deferred; the `register`/`register_alias` sites are not deferred but the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). +- **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: archived range branches that were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 values that** `_missing_` **resolves (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). - `LinkType(209).name` returned `'IPMB_LINUX'`, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current `I2C_LINUX = 209` and `aenum` gives a value to the member defined first. `pcapkit/vendor/reg/linktype.py` sinks a row whose note mentions "legacy" after every current row, so the current name is defined first; in the registry crawled for this release that affects value 209 alone. `IPMB_LINUX` stays a reachable alias (`LinkType['IPMB_LINUX']` still resolves to 209). Not breaking: only the canonical name for the shared value moves ([#848](https://github.com/JarryShaw/PyPCAPKit/pull/848)). The rule sinks only a row whose value another row also holds, counting values inside ranges, so a current row that merely mentions a legacy name stays first ([#852](https://github.com/JarryShaw/PyPCAPKit/issues/852), [#854](https://github.com/JarryShaw/PyPCAPKit/pull/854)). -- **a breaking change to** 82 more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): a final, concrete assigned name mints, and a notation for readers does not) across the 89 `_missing_` files still calling `extend_enum`. 80 registries convert wholly (164 branches), each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Two are mixed: `EtherType`'s `DEC Unassigned` (3 branches) and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." (1) convert, while 46 vendor-company names across 50 range blocks (Xyplex, Motorola, Walker Richer & Quinn and 43 more) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. 7 of the 89 are left alone: `CGAType`'s `Tag_` mint is not an IANA-style range, and the other 6 files sit on the 9 classes [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) moves onto `EnumRegistry` (`AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method`, `OptionType`; [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) and [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. The worked examples in `docs/source/contributing/conventions/mint-criterion.rst` keep `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). +- **a breaking change to** more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): a final, concrete assigned name mints, and a notation for readers does not) across the `_missing_` files still calling `extend_enum`. Most convert wholly, each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Some are mixed: `EtherType`'s `DEC Unassigned` and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." branches convert, while vendor-company names (Xyplex, Motorola, Walker Richer & Quinn and others) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. 7 of the 89 are left alone: `CGAType`'s `Tag_` mint is not an IANA-style range, and the other 6 files sit on the 9 classes [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) moves onto `EnumRegistry` (`AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method`, `OptionType`; [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) and [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. The worked examples in `docs/source/contributing/conventions/mint-criterion.rst` keep `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). - **a breaking change to** `EtherType`'s `_missing_` ordering: values `0x0101`-`0x01FF` now resolve to their Old Xerox Experimental name instead of an `IEEE802_3_Length_Field_0x0101`-style one. The generated method tested `0x0000 <= value <= 0x05DC` (IEEE802.3 Length Field) first, and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike [#841](https://github.com/JarryShaw/PyPCAPKit/issues/841)'s IPX fix, these bounds come from the live IANA CSV, so the fix is general: `pcapkit/vendor/reg/ethertype.py`'s `process()` collects range rows and a new `EtherType._insert_range` places each ahead of the first already-placed range that fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only containment among the 56 ranges. Closes [#862](https://github.com/JarryShaw/PyPCAPKit/issues/862) ([#865](https://github.com/JarryShaw/PyPCAPKit/pull/865)). -- **a breaking change to** `Method`: `__new__` called `str.__new__(cls)` with no argument, so all 40 registered members' `str` payload was empty regardless of value -- `str(Method.GET) == ''` and `Method.GET == 'GET'` was `False`; `.value` was always correct. [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) (above) had fixed the *unregistered* path only, leaving `str(Method('frob')) == 'frob'` beside `str(Method.GET) == ''`. Fixed to `str.__new__(cls, value)`, mirroring `Command.__new__`, in the generator (`pcapkit/vendor/http/method.py`) and the generated module. No sibling hand-rolled `__new__` has the defect (`OptionType`/`AppType` deliberately store a display string). Consequences, all corrections: `Method.GET == 'GET'` is `True`; `bool(Method.GET)` flips `False` to `True`; `json.dumps(Method.GET)` emits `"GET"` not `""`; `sorted()` is lexicographic rather than input-order-stable; and `{m: ... for m in Method}` gains 40 entries where it had 1, because all 40 members previously hashed as `''` and compared equal. Nothing under `pcapkit/protocols` relied on the old behaviour. Closes [#870](https://github.com/JarryShaw/PyPCAPKit/issues/870) ([#871](https://github.com/JarryShaw/PyPCAPKit/pull/871)). -- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 2 of 2 (the 8 non-`AppType` bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its four transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s 766 range branches and `get()`'s second mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that only IANA-registered values are legitimate and that `get()` lacks the information to create new ones. All 766+1 branches convert, including 8 naming a real service assigned to a whole port span (7 distinct names: `x11` twice, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). -- closes the remaining scope of [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): `EtherType` (52 branches) and `Socket` (1), the last registries whose `_missing_` minted a permanent member, now return a non-registering `cls._unregistered_member(value, name)`. The issue's AST census of `extend_enum` in `_missing_`/`get` goes from 54 sites to the single `CGAType` carve-out (its mint is not an IANA-style range), from the original 1,169. `EtherType(0x0888)` used to grow `__members__` from 160 to 161 and return the identical object on a repeat; it now returns `` with `__members__` unchanged, absent from `_value2member_map_`, equal but not identical across lookups -- likewise `Socket(0x0010)`. The regenerated `EtherType` was not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape `test_const_enum_no_mint.py`'s `is_manufactured()` sweep forbids; rather than weaken the predicate, a narrow exemption covers exactly these two files, since the collision the check guards against cannot occur once `_unregistered_member` never registers ([#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)). +- **a breaking change to** `Method`: `__new__` called `str.__new__(cls)` with no argument, so every registered member's `str` payload was empty regardless of value -- `str(Method.GET) == ''` and `Method.GET == 'GET'` was `False`; `.value` was always correct. [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) (above) had fixed the *unregistered* path only, leaving `str(Method('frob')) == 'frob'` beside `str(Method.GET) == ''`. Fixed to `str.__new__(cls, value)`, mirroring `Command.__new__`, in the generator (`pcapkit/vendor/http/method.py`) and the generated module. No sibling hand-rolled `__new__` has the defect (`OptionType`/`AppType` deliberately store a display string). Consequences, all corrections: `Method.GET == 'GET'` is `True`; `bool(Method.GET)` flips `False` to `True`; `json.dumps(Method.GET)` emits `"GET"` not `""`; `sorted()` is lexicographic rather than input-order-stable; and `{m: ... for m in Method}` gains an entry per member where it had one, because every member previously hashed as `''` and compared equal. Nothing under `pcapkit/protocols` relied on the old behaviour. Closes [#870](https://github.com/JarryShaw/PyPCAPKit/issues/870) ([#871](https://github.com/JarryShaw/PyPCAPKit/pull/871)). +- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), the `AppType` half (the other bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s range branches and `get()`'s second mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that only IANA-registered values are legitimate and that `get()` lacks the information to create new ones. Every branch converts, including those naming a real service assigned to a whole port span (`x11`, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). +- closes the remaining scope of [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): `EtherType` and `Socket`, the last registries whose `_missing_` minted a permanent member, now return a non-registering `cls._unregistered_member(value, name)`. The issue's AST census of `extend_enum` in `_missing_`/`get` is down to the single `CGAType` carve-out (its mint is not an IANA-style range). `EtherType(0x0888)` used to add a member to `__members__` and return the identical object on a repeat; it now returns `` with `__members__` unchanged, absent from `_value2member_map_`, equal but not identical across lookups -- likewise `Socket(0x0010)`. The regenerated `EtherType` was not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape `test_const_enum_no_mint.py`'s `is_manufactured()` sweep forbids; rather than weaken the predicate, a narrow exemption covers exactly these files, since the collision the check guards against cannot occur once `_unregistered_member` never registers ([#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)). - `OptionType.register()` and `Command.register()` built members that `get()` could not find again, and `OptionType.register_alias()` raised `not a registered OptionType` for every argument. Both registries now round-trip through `get()`, in the generated files and the vendor templates ([#1311](https://github.com/JarryShaw/PyPCAPKit/pull/1311), [#1330](https://github.com/JarryShaw/PyPCAPKit/issues/1330), [#1369](https://github.com/JarryShaw/PyPCAPKit/pull/1369)). - the Quick-Start function registries rejected 9 to 15, though [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) gives the Function field four bits; the whole 4-bit range now resolves ([#1350](https://github.com/JarryShaw/PyPCAPKit/issues/1350), [#1353](https://github.com/JarryShaw/PyPCAPKit/pull/1353)). - every three-digit FTP reply and HTTP status code resolves, including those no registry row names ([#1443](https://github.com/JarryShaw/PyPCAPKit/issues/1443), [#1444](https://github.com/JarryShaw/PyPCAPKit/pull/1444)). - the `AppType` registry's dict union is defined only on interpreters whose `dict` has one ([#1694](https://github.com/JarryShaw/PyPCAPKit/issues/1694), [#1703](https://github.com/JarryShaw/PyPCAPKit/pull/1703)). - NGAP's `ProcedureCode` and `ProtocolIE` are generated into `pcapkit.const.ngap` from `pycrate`'s compiled NGAP specification, with no network fetch. An unseen value gets a non-registering member, where every unknown key used to mint a member at the shared value `-1` and so aliased every other ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#899](https://github.com/JarryShaw/PyPCAPKit/pull/899)). - `FEATCode.get` matches case-insensitively, as [RFC 5797 Section 2](https://datatracker.ietf.org/doc/html/rfc5797#section-2) defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold ([#903](https://github.com/JarryShaw/PyPCAPKit/issues/903), [#913](https://github.com/JarryShaw/PyPCAPKit/pull/913)). +- `register` refuses what `get` refuses. `TCP.register(70000, 'x')` and `Hardware.register(70000, 'x')` minted a member that `get` then returned, because no registry overrode the `_validate_value` hook `register` calls. Each generated registry, `AppType` included, now overrides it with the check its `_missing_` makes, raising `EnumValueError`. `get(value, default)` on a value the hook refuses still returns `default`, but now also logs the `EnumValueError` at `CRITICAL`, as any loud pcapkit error is ([#1792](https://github.com/JarryShaw/PyPCAPKit/issues/1792), [#1818](https://github.com/JarryShaw/PyPCAPKit/pull/1818), [#1816](https://github.com/JarryShaw/PyPCAPKit/issues/1816), [#1823](https://github.com/JarryShaw/PyPCAPKit/pull/1823)). +- `Command.get` refuses a key that is not a `str` with `EnumValueError`, as `Method.get` does. `get(b'\x01')` minted a member keyed by `bytes`, and `get(-1)` raised `AttributeError` whatever its `default` ([#1824](https://github.com/JarryShaw/PyPCAPKit/issues/1824), [#1826](https://github.com/JarryShaw/PyPCAPKit/pull/1826)). ### pcapkit.corekit #### Added - `tests/corekit/test_fields_numbers_width_repair.py`, covering each octet boundary in its own method, since the defect is a pattern and a single case would pass against a fix that special-cased the reported width ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). -- `FieldBaseShortReadPaddingSideTests`, covering both byte orders at 2, 4 and 8 octets and every truncation point, including the two figures [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) reports, plus guard rails for a full read and for a read against an empty buffer, the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) behaviour the option and list loops depend on ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- `FieldBaseShortReadPaddingSideTests`, covering both byte orders at 2, 4 and 8 octets and every truncation point, including the figures [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) reports, plus guard rails for a full read and for a read against an empty buffer, the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) behaviour the option and list loops depend on ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - `EnumLookup`, the base carrying `get` and `get_all` that `EnumRegistry` and the non-registry enumerations share ([#877](https://github.com/JarryShaw/PyPCAPKit/issues/877), [#906](https://github.com/JarryShaw/PyPCAPKit/pull/906), [#921](https://github.com/JarryShaw/PyPCAPKit/pull/921), [#932](https://github.com/JarryShaw/PyPCAPKit/pull/932)). Its `get` raises pcapkit exceptions in the shape of the stdlib `Enum`: a name miss raises `EnumKeyError`, a `KeyError` ([#923](https://github.com/JarryShaw/PyPCAPKit/issues/923), [#927](https://github.com/JarryShaw/PyPCAPKit/pull/927)). - `MultiInfo` and `OrderedMultiInfo`, immutable `Info` subclasses that are also a `MultiDict` and an `OrderedMultiDict`, so a repeated key keeps every value. The data models' option lists use them ([#1484](https://github.com/JarryShaw/PyPCAPKit/issues/1484), [#1523](https://github.com/JarryShaw/PyPCAPKit/pull/1523)). - `pcapkit.corekit.fields.byteorder`, holding `packet_byteorder` and `byteorder_callback`, the byte-order helpers the PCAP and PCAP-NG schemas share ([#1528](https://github.com/JarryShaw/PyPCAPKit/pull/1528), [#1711](https://github.com/JarryShaw/PyPCAPKit/issues/1711), [#1715](https://github.com/JarryShaw/PyPCAPKit/pull/1715)). @@ -68,19 +70,19 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - stdlib exceptions leaking out where the library's own were promised: a malformed IP field value raised a bare `ValueError` instead of `FieldValueError` ([#465](https://github.com/JarryShaw/PyPCAPKit/issues/465)); a `bool` address was silently packed as `0.0.0.1` or `0.0.0.0`, `bool` being an `int` subclass ([#491](https://github.com/JarryShaw/PyPCAPKit/issues/491), [#500](https://github.com/JarryShaw/PyPCAPKit/pull/500)); and `@prepare` discarded extra arguments silently and treated a *declared* zero length as end of stream, which is now distinguished from a genuinely exhausted one and raises `StreamEOFError` ([#454](https://github.com/JarryShaw/PyPCAPKit/issues/454), [#458](https://github.com/JarryShaw/PyPCAPKit/issues/458)). - `FieldBase.unpack` zero-padded up to a field's declared `length` with `rjust()` however little data `buffer` held; a ~40-octet PCAP-NG Decryption Secrets Block with a bogus inner length forced a multi-gigabyte allocation, since `length` is often wire-derived. A declared length past 262144 octets -- libpcap's `MAXIMUM_SNAPLEN` and this package's default `snaplen` -- that the buffer cannot back now raises `FieldValueError`; the option and list loops' tolerance for a short read past a truncated area ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) is far under that ceiling and untouched ([#554](https://github.com/JarryShaw/PyPCAPKit/issues/554)). - that ceiling bounds one field, and a packet holds many, so the *sum* of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring `secrets_length` 262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing. `FieldBase.unpack` now keeps a per-context ledger of octets supplied against octets synthesised, and raises `FieldValueError` once the total of shortfalls *past 65,536 octets* passes `262144 + 16 * supplied`. The same 200 blocks retain 0.2 MiB (54.6x). A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded unconditionally and charged to nothing, so no snapshot-truncated capture is subject to the budget; without that band the same legitimate frame parsed differently depending on what had been parsed before. **What this does not close**: a length declared by a 16-bit field can still be repeated without limit, and at this layer a crafted file is indistinguishable from a legitimate snapshot-truncated capture of offload-sized segments; separating them needs the frame's own `incl_len`/`orig_len`. Nothing resets the ledger, so it bounds everything a context has parsed rather than one file ([#573](https://github.com/JarryShaw/PyPCAPKit/issues/573)). -- which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised `ProtocolError` as documented, but a process that had popped `pcapkit.corekit.fields.misc` from `sys.modules` (as the `#439` ABC-cache regression tests do in every `setUp`/`tearDown`) raised `FieldValueError` from a different layer, before the documented check ([#525](https://github.com/JarryShaw/PyPCAPKit/issues/525)). `ListField.unpack` resolved `SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so `isinstance` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** `FieldValueError` **now gets** `ProtocolError`, deterministically, matching the docstring. +- which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised `ProtocolError` as documented, but a process that had popped `pcapkit.corekit.fields.misc` from `sys.modules` (as the [#439](https://github.com/JarryShaw/PyPCAPKit/issues/439) ABC-cache regression tests do in every `setUp`/`tearDown`) raised `FieldValueError` from a different layer, before the documented check ([#525](https://github.com/JarryShaw/PyPCAPKit/issues/525)). `ListField.unpack` resolved `SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so `isinstance` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** `FieldValueError` **now gets** `ProtocolError`, deterministically, matching the docstring. - a `NumberField` whose `length` was a callable could not pack or parse at any width `struct` has a native integer code for. `length` is `-1` until the callable resolves, `-1` has no native code, and the template builder raised `_need_process` for it and never cleared it -- a latch. Resolving the real width rebuilt the template but left the latch set, so `pre_process` handed `bytes` to a template that had become `>Q` (`struct.error: required argument is not an integer`), and parsing failed in mirror by calling `int.from_bytes` on an integer `struct.unpack` had produced. The flag is now recomputed from the width in force, which also keeps a callable resolving to a width with no native code (3 octets) byte-packed. **All four native widths were affected, not only the 8 reported**, on `NumberField` and `EnumField` alike. This made every extended 8-octet MPTCP DSS form unbuildable, those widths being chosen at runtime from the DSS flags; [#585](https://github.com/JarryShaw/PyPCAPKit/pull/585) worked around it in the TCP schema alone ([#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)). -- `NumberField.pre_process` sized a value with floor division dressed as a ceiling. When a field is packed while `length` is still the `-1` placeholder, the width came from `math.ceil(value.bit_length() // 8)`; the `//` had already floored, so `math.ceil` did nothing and the width was one octet short: `256` at one octet, `65536` at two, `16777216` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so `1` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s suite, whose values all had byte-aligned bit lengths, missed it; [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack `128`. The same typo at the two ILNP nonce option builders is a separate change ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). +- `NumberField.pre_process` sized a value with floor division dressed as a ceiling. When a field is packed while `length` is still the `-1` placeholder, the width came from `math.ceil(value.bit_length() // 8)`; the `//` had already floored, so `math.ceil` did nothing and the width was one octet short: `256` at one octet, `65536` at two, `16777216` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so `1` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s suite, whose values all had byte-aligned bit lengths, missed it; [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack `128`. The same typo in the ILNP nonce option builders is a separate change ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). - `FieldBase.unpack` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) -- the shortfall was zero-filled with `rjust()`, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now `ljust()`, correct for **both** byte orders: one octet of a four-octet little-endian `120` read as `2013265920`, and three octets of a four-octet big-endian `0x01020304` as `0x10203`. A full read is untouched ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but two defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Three more contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). -- the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to all fifteen variable-width option and record payloads in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A second, `bounded_area`, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The five non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). +- `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but separate defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Further contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). +- the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A second, `bounded_area`, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). - **a breaking change to** `FieldBase.length`: it called `struct.calcsize` on a template built from a negative resolved length (e.g. `'-5s'`, from a `length=` callback whose running counter had been overdrawn), raising a bare `struct.error` that ordinary caller code does not catch. The chokepoint every affected schema shares now raises `ProtocolError`, covering `application/{ftp,httpv1,httpv2,ngap}.py`, `internet/{hip,ipv6_route,mh}.py`, `link/ethernet.py`, `misc/pcapng.py` and `transport/sctp.py` with no other file changed; a non-negative length is unaffected. This closes the follow-on the `httpv2` length-guard entry above filed as out of scope ([#805](https://github.com/JarryShaw/PyPCAPKit/issues/805)). -- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it converts the first six registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and four `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those six: the other 105 const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships two contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). The six also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the five `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other 105 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). -- **a breaking change to** `EnumRegistry.register()`, extended: the 105 const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The 10 modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype`) are left alone, so **111 of the 127 classes** inherit `EnumRegistry`, up from 6 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the maintainer's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). +- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it converts the first six registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and four `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those: the other const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). These also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other registries ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). +- **a breaking change to** `EnumRegistry.register()`, extended: the const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype` among them) are left alone, so **every other class** inherits `EnumRegistry` ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the maintainer's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). - `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). -- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but both `cls(default)` call sites reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, both sites now resolve `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). +- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but each `cls(default)` call site reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, each site now resolves `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). - `ProtoChain.__add__` returned the left operand's cached `protocols` and `aliases`: once `a.protocols` had been read, `a + ProtoChain(IPv4)` reported `chain == 'Ethernet:IPv4'` but `protocols == (Ethernet,)`. `copy.copy(self)` carried the cache across; the result is now built fresh, as `from_list` builds one, and `+=` follows, since it falls back to `__add__` ([#1099](https://github.com/JarryShaw/PyPCAPKit/issues/1099)). -- a `SchemaField` with no declared length handed its `-1` placeholder to the nested schema as that schema's length, so every item of such a list warned `packet length < 0`: 12 `SchemaWarning` for a two-locator HIP `LOCATOR_SET`, 6 for one MH CGA Parameters item. `SchemaField.unpack` now passes what is left in the buffer as a declared length -- not `None`, which would turn a truncated item's `FieldValueError` into `prepare()`'s quiet `StreamEOFError`. Fixed-length items are unchanged; the HTTP/2 `SETTINGS` pair, whose size is fixed, is given it in its own schema instead ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140), below) ([#1148](https://github.com/JarryShaw/PyPCAPKit/issues/1148)). +- a `SchemaField` with no declared length handed its `-1` placeholder to the nested schema as that schema's length, so every item of such a list warned `packet length < 0`: a HIP `LOCATOR_SET` and an MH CGA Parameters item among them. `SchemaField.unpack` now passes what is left in the buffer as a declared length -- not `None`, which would turn a truncated item's `FieldValueError` into `prepare()`'s quiet `StreamEOFError`. Fixed-length items are unchanged; the HTTP/2 `SETTINGS` pair, whose size is fixed, is given it in its own schema instead ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140), below) ([#1148](https://github.com/JarryShaw/PyPCAPKit/issues/1148)). - a signed `NumberField` decoded as unsigned, a zero-length `PayloadField` misread, and an empty `Raw` did not rebuild ([#1331](https://github.com/JarryShaw/PyPCAPKit/pull/1331)). - `from_dict(to_dict())` lost nested values, and `len()` of an `Info` counted its hidden keys ([#1334](https://github.com/JarryShaw/PyPCAPKit/pull/1334)). - `StringField` packs back exactly the octets it unpacked, where a guessed charset, a `U+FFFD` substitution or a character-count length used to change them; the decoded text is unchanged ([#1293](https://github.com/JarryShaw/PyPCAPKit/issues/1293), [#1336](https://github.com/JarryShaw/PyPCAPKit/pull/1336)). A charset that rejects `'replace'` falls back to UTF-8 ([#1706](https://github.com/JarryShaw/PyPCAPKit/issues/1706), [#1728](https://github.com/JarryShaw/PyPCAPKit/pull/1728)), and Python 3.8 unquotes through `pcapkit.utilities.compat.unquote` ([#1693](https://github.com/JarryShaw/PyPCAPKit/issues/1693), [#1707](https://github.com/JarryShaw/PyPCAPKit/pull/1707)). @@ -93,7 +95,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `ListField` unpacks a schema item from the per-item copy its callback and `length_callback` configured, not from the unconfigured original ([#433](https://github.com/JarryShaw/PyPCAPKit/issues/433), [#453](https://github.com/JarryShaw/PyPCAPKit/pull/453)). - an `*IntField` given a `signed=` that contradicts its fixed sign, such as `UInt8Field(signed=True)`, raises `FieldValueError`; the argument used to be discarded silently ([#545](https://github.com/JarryShaw/PyPCAPKit/issues/545), [#549](https://github.com/JarryShaw/PyPCAPKit/pull/549)). - negative lengths raise `ProtocolError` rather than a bare stdlib error: `FieldBase.length` tells a malformed template from a negative resolved length ([#825](https://github.com/JarryShaw/PyPCAPKit/issues/825), [#827](https://github.com/JarryShaw/PyPCAPKit/pull/827)), and `NumberField` checks a negative resolved `length` and a negative `bit_length` ([#828](https://github.com/JarryShaw/PyPCAPKit/issues/828), [#829](https://github.com/JarryShaw/PyPCAPKit/pull/829), [#831](https://github.com/JarryShaw/PyPCAPKit/issues/831), [#834](https://github.com/JarryShaw/PyPCAPKit/pull/834)). -- `ModuleDescriptor.klass` raises `ProtocolError` for a class name that resolves to nothing, or one never given, rather than a bare `AttributeError`; the two registry modules share one `NULL` sentinel compared by identity, where each had its own string ([#832](https://github.com/JarryShaw/PyPCAPKit/issues/832), [#833](https://github.com/JarryShaw/PyPCAPKit/issues/833), [#835](https://github.com/JarryShaw/PyPCAPKit/pull/835)). +- `ModuleDescriptor.klass` raises `ProtocolError` for a class name that resolves to nothing, or one never given, rather than a bare `AttributeError`; the registry modules share one `NULL` sentinel compared by identity, where each had its own string ([#832](https://github.com/JarryShaw/PyPCAPKit/issues/832), [#833](https://github.com/JarryShaw/PyPCAPKit/issues/833), [#835](https://github.com/JarryShaw/PyPCAPKit/pull/835)). - `PortEnumField` given a `proto` with no `.name` -- a string such as `'tcp'`, or a composite such as `TransportProtocol.tcp | TransportProtocol.udp`, which is a bare `int` -- raised `AttributeError` on a port the registry does not hold. The miss takes its transport from the registry's `__transport__`, as `AppType.get` does ([#1774](https://github.com/JarryShaw/PyPCAPKit/issues/1774), [#1781](https://github.com/JarryShaw/PyPCAPKit/pull/1781)). ### pcapkit.dumpkit @@ -101,9 +103,9 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Fixed - a flag value with no declared bits no longer dumps as `Type::None [0]`. **This changes user-visible output in every textual dump format.** `make_dumper`'s `object_hook` renders each enumeration member as `Type::name [value]` and interpolated the name unguarded, but a `Flag` value composed entirely of undeclared bits has `name is None`, so the literal `None` landed in the name half: `Flags(0)` rendered as `Flags::None [0]` in `json`, `tree`, `text`, `txt`, `plist` and `xml`, out of both `Extractor` and `TraceFlow`. A nameless member now renders with the value's decimal spelling, so `Flags(0)` gives `Flags::0 [0]` and `Flags(8)` gives `Flags::8 [8]`. That is the spelling the enumeration libraries use for an undeclared residue (`Flags(2057).name` is `ACK|9`), and it cannot be mistaken for a member name, since an identifier may not begin with a digit, whereas `NONE` is a real declared name elsewhere. The guard is on `name is None` and not on the value, because the defect was never about zero (`Flags(1)` and `Flags(9)` are equally nameless). Surfaced by [#634](https://github.com/JarryShaw/PyPCAPKit/pull/634), which made a flagless TCP segment reach this path ([#648](https://github.com/JarryShaw/PyPCAPKit/issues/648)). -- `tests/dumpkit/test_nameless_enum_rendering_unit.py` (added by [#670](https://github.com/JarryShaw/PyPCAPKit/pull/670)) carried two probes assuming every flag-enum registry is 16 bits wide: `test_scalar_return_renders_a_nameless_member_as_its_value` used one fixed tuple ending in `65536`, and `test_no_flag_registry_renders_the_literal_none` swept every registry but only against `registry(0)`. Widths differ: `ftp.command.CommandType` 3 bits, `reg.apptype.TransportProtocol` 4 (computed at runtime), the three `mh` ack/initiate flags 8, and `mh.binding_update_flag.BindingUpdateFlag` and `tcp.flags.Flags` 16. `tcp.flags.Flags(65536)` correctly raises (`0 <= value <= 0xFFFF`), so the sweep was failing on correct behaviour. The probes now derive each registry's bound from its declared members, and a new test pins that one past the widest in-field value is refused. Test-only ([#702](https://github.com/JarryShaw/PyPCAPKit/issues/702)). +- `tests/dumpkit/test_nameless_enum_rendering_unit.py` (added by [#670](https://github.com/JarryShaw/PyPCAPKit/pull/670)) carried probes assuming every flag-enum registry is 16 bits wide: `test_scalar_return_renders_a_nameless_member_as_its_value` used one fixed tuple ending in `65536`, and `test_no_flag_registry_renders_the_literal_none` swept every registry but only against `registry(0)`. Widths differ: `ftp.command.CommandType` 3 bits, `reg.apptype.TransportProtocol` 4 (computed at runtime), the `mh` ack/initiate flags 8, and `mh.binding_update_flag.BindingUpdateFlag` and `tcp.flags.Flags` 16. `tcp.flags.Flags(65536)` correctly raises (`0 <= value <= 0xFFFF`), so the sweep was failing on correct behaviour. The probes now derive each registry's bound from its declared members, and a new test pins that one past the widest in-field value is refused. Test-only ([#702](https://github.com/JarryShaw/PyPCAPKit/issues/702)). - the plist writer emitted mapping keys unescaped, so a key carrying `&`, `<` or `>` produced a report that is not well-formed XML. `dictdumper`'s `_append_dict` interpolates the key into `'{item}'` and calls `_encode_value` on the value only, so both branches of `pcapkit.dumpkit.common` that build a mapping now escape their keys through one `escape_key` helper -- a `MultiDict`, where [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) escaped an enum-derived key inline and nothing else, and a plain dict, where nothing was escaped. The json, tree and text writers are untouched. A non-`str` key is escaped too: a PCAP-NG decryption secrets block keys TLS key log entries by a raw bytes client random whose repr can carry all three characters. The upstream half is `JarryShaw/DictDumper#125`; this does not wait on it. The json writer's quoting defect stands, upstream as `JarryShaw/DictDumper#121` ([#772](https://github.com/JarryShaw/PyPCAPKit/issues/772)). -- the JSON writer emitted mapping keys unescaped, so the report of `test.pcapng` did not parse (`Expecting ':' delimiter`): its TLS key log is keyed by a `bytes` client random whose repr holds `"` and `\`. `escape_key` in `pcapkit.dumpkit.common` now JSON-escapes keys for JSON output, as [#772](https://github.com/JarryShaw/PyPCAPKit/issues/772) made it XML-escape them for plist, and the JSON reports of all 23 `examples/captures/` files now parse. The upstream defect, `JarryShaw/DictDumper#121`, stands ([#1152](https://github.com/JarryShaw/PyPCAPKit/issues/1152)). +- the JSON writer emitted mapping keys unescaped, so the report of `test.pcapng` did not parse (`Expecting ':' delimiter`): its TLS key log is keyed by a `bytes` client random whose repr holds `"` and `\`. `escape_key` in `pcapkit.dumpkit.common` now JSON-escapes keys for JSON output, as [#772](https://github.com/JarryShaw/PyPCAPKit/issues/772) made it XML-escape them for plist, and the JSON report of every `examples/captures/` file now parses. The upstream defect, `JarryShaw/DictDumper#121`, stands ([#1152](https://github.com/JarryShaw/PyPCAPKit/issues/1152)). The PCAP-NG end-to-end test of that report no longer skips; test-only ([#1795](https://github.com/JarryShaw/PyPCAPKit/issues/1795), [#1809](https://github.com/JarryShaw/PyPCAPKit/pull/1809)). - **a breaking change to** the dump schema: the output writers dropped or corrupted what the packets carried. An `OrderedMultiDict`, which every option list is, now dumps as a list of single-key entries in wire order ([#1305](https://github.com/JarryShaw/PyPCAPKit/pull/1305)). Inside an object the writers fall back on, a class, module or function is written as its qualified name rather than expanded, and an object met again on its own expansion path as `` ([#1372](https://github.com/JarryShaw/PyPCAPKit/issues/1372), [#1381](https://github.com/JarryShaw/PyPCAPKit/pull/1381)). - tree output raised on a `complex` value, and is now documented as lossy ([#1266](https://github.com/JarryShaw/PyPCAPKit/issues/1266), [#1366](https://github.com/JarryShaw/PyPCAPKit/pull/1366)). - plist dates are written to whole seconds, so `plistlib` reads them back ([#1448](https://github.com/JarryShaw/PyPCAPKit/issues/1448), [#1450](https://github.com/JarryShaw/PyPCAPKit/pull/1450)). @@ -113,42 +115,42 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Added -- three extraction engines: `engine='pypcap'` and `engine='pcap_ct'`, two independent distributions of the same `libpcap` interface, and `engine='pypcapfile'` ([#386](https://github.com/JarryShaw/PyPCAPKit/pull/386), [#405](https://github.com/JarryShaw/PyPCAPKit/pull/405)). They buy speed by doing less -- neither `pypcap` nor `pcap_ct` dissects at all, so they offer no reassembly or flow tracing, and `pypcapfile` has no IPv6 decoder. Install only **one** of `pypcap` and `pcap-ct`: both own the top-level `pcap` module, and with both present `pcap-ct` wins the import and the other becomes unselectable. The missing interface constants `PyPCAP`, `PCAP_CT` and `PyPCAPFile` are now exported alongside `DPKT`, `Scapy`, `PyShark` and `PCAPKit` ([#412](https://github.com/JarryShaw/PyPCAPKit/pull/412)). That makes seven built-in engines; 3.11 is the last interpreter on which every one can run, and even there two cannot coexist. +- the `engine='pypcap'` and `engine='pcap_ct'` extraction engines, independent distributions of the same `libpcap` interface, and `engine='pypcapfile'` ([#386](https://github.com/JarryShaw/PyPCAPKit/pull/386), [#405](https://github.com/JarryShaw/PyPCAPKit/pull/405)). They buy speed by doing less -- neither `pypcap` nor `pcap_ct` dissects at all, so they offer no reassembly or flow tracing, and `pypcapfile` has no IPv6 decoder. Install only **one** of `pypcap` and `pcap-ct`: both own the top-level `pcap` module, and with both present `pcap-ct` wins the import and the other becomes unselectable. The missing interface constants `PyPCAP`, `PCAP_CT` and `PyPCAPFile` are now exported alongside `DPKT`, `Scapy`, `PyShark` and `PCAPKit` ([#412](https://github.com/JarryShaw/PyPCAPKit/pull/412)). 3.11 is the last interpreter on which every built-in engine can run, and even there `pypcap` and `pcap-ct` cannot coexist. - `EngineBase.unsupported_reason`, a preflight every engine answers and `Extractor.run` consults before anything is imported. An engine that cannot run in the current environment now gives one warning naming the real cause (a Python version, a missing `tshark` or `libpcap`, the wrong `pcap` distribution) and a clean fall back to `pcapkit`'s own parser, rather than an error from inside the third-party package ([#396](https://github.com/JarryShaw/PyPCAPKit/pull/396), [#405](https://github.com/JarryShaw/PyPCAPKit/pull/405)). The same holds for an engine whose package is not installed at all ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045)). - `conflict` on the reassembly data models: absolute, inclusive ranges where two fragments claimed the same span with different bytes, previously lost silently on both the IP ([#482](https://github.com/JarryShaw/PyPCAPKit/pull/482)) and TCP ([#443](https://github.com/JarryShaw/PyPCAPKit/issues/443), [#478](https://github.com/JarryShaw/PyPCAPKit/pull/478)) paths. #### Changed - `layer=` and `protocol=` are honoured rather than inert. Both were read under the wrong names, so every value a caller passed was dropped into `**kwargs`; the CLI's `-L` also now validates its argument. The packet context reaches the schema layer for the first time, so a field the wire elides can be resolved from its enclosing packet ([#404](https://github.com/JarryShaw/PyPCAPKit/pull/404)). `follow_tcp_stream` dispatches on the engine type, where both branches of the old test were dead and the native adapter ran against every engine's frames ([#402](https://github.com/JarryShaw/PyPCAPKit/pull/402)). -- two reassembly and flow-tracing defaults moved ([#435](https://github.com/JarryShaw/PyPCAPKit/pull/435)), both visible to a caller. `Datagram.completed` widened from `bool` to a `Completion` enumeration (`COMPLETE`, `PARTIAL`, `TIMEOUT`); only `COMPLETE` is truthy, so `if datagram.completed:` is unaffected but `datagram.completed == True` no longer holds. TCP flow tracing is **bidirectional by default**, which merges each flow's two halves and closes one only once both have FINed -- 331 flows become 111 on the sample HTTP capture, the difference being single-frame stray tails; pass `trace_bidirectional=False` for the old behaviour. IP reassembly also gained the 60-second timeout [[RFC 1122](https://datatracker.ietf.org/doc/html/rfc1122), [RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200)], clocked off the capture's own timestamps, tunable with `reasm_timeout=`; TCP reassembly gets no timeout by default. `trace_analyse=` is new, and reassembles each traced flow's application layer. +- reassembly and flow-tracing defaults moved ([#435](https://github.com/JarryShaw/PyPCAPKit/pull/435)), visible to a caller. `Datagram.completed` widened from `bool` to a `Completion` enumeration (`COMPLETE`, `PARTIAL`, `TIMEOUT`); only `COMPLETE` is truthy, so `if datagram.completed:` is unaffected but `datagram.completed == True` no longer holds. TCP flow tracing is **bidirectional by default**, which merges each flow's two halves and closes one only once both have FINed -- 331 flows become 111 on the sample HTTP capture, the difference being single-frame stray tails; pass `trace_bidirectional=False` for the old behaviour. IP reassembly also gained the 60-second timeout [[RFC 1122](https://datatracker.ietf.org/doc/html/rfc1122), [RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200)], clocked off the capture's own timestamps, tunable with `reasm_timeout=`; TCP reassembly gets no timeout by default. `trace_analyse=` is new, and reassembles each traced flow's application layer. - conflicting TCP overlaps resolve first-write-wins, per [RFC 9293](https://datatracker.ietf.org/doc/html/rfc9293) section 3.10, where they had silently resolved last-write-wins ([#443](https://github.com/JarryShaw/PyPCAPKit/issues/443), [#478](https://github.com/JarryShaw/PyPCAPKit/pull/478)). A deliberate, narrow behaviour break: a conforming retransmission carries identical bytes. IP fragment reassembly keeps last-write-wins, because [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) specifies the opposite resolution, and records the disagreement instead ([#482](https://github.com/JarryShaw/PyPCAPKit/pull/482)). - renames with no compatibility alias left behind: `HoleDiscriptor` is spelled `HoleDescriptor` and its package alias `TCP_HoleDiscriptor` is `TCP_HoleDescriptor` ([#350](https://github.com/JarryShaw/PyPCAPKit/pull/350)). -- subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` ([#514](https://github.com/JarryShaw/PyPCAPKit/issues/514)). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias, which every built-in does (hence the public classes had **0** subclasses against the `*Base` classes' 9, 5, 2 and 3). **This breaks out-of-tree code that subclasses one of the four and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Two silent failures are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` (passing `name=` to a `Reassembly` subclass silently ignored the key, since `protocol=` is the real one), and so does `Dumper`'s `ext=` without `fmt=`. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass gained a class-level `registry` property mirroring `EnumSchema.registry`, and a `Dumper` subclass no longer touches the filesystem while its `class` statement runs. `Engine`'s keyword is `engine=` rather than the `name=` first shipped, because `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s parameters on Python 3.10 and raise `TypeError` before the hook runs. There is no `name=` alias: a keyword that works on some interpreters and not others is the trap being removed. -- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (all 13 derive from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its two siblings check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched ([#1020](https://github.com/JarryShaw/PyPCAPKit/pull/1020)). +- subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` ([#514](https://github.com/JarryShaw/PyPCAPKit/issues/514)). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias, which every built-in does (hence no public class had a subclass, while every built-in derives from its `*Base` class). **This breaks out-of-tree code that subclasses one of these classes and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Silent failures are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` (passing `name=` to a `Reassembly` subclass silently ignored the key, since `protocol=` is the real one), and so does `Dumper`'s `ext=` without `fmt=`. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass gained a class-level `registry` property mirroring `EnumSchema.registry`, and a `Dumper` subclass no longer touches the filesystem while its `class` statement runs. `Engine`'s keyword is `engine=` rather than the `name=` first shipped, because `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s parameters on Python 3.10 and raise `TypeError` before the hook runs. There is no `name=` alias: a keyword that works on some interpreters and not others is the trap being removed. +- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (every built-in derives from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its sibling internal registrars check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched ([#1020](https://github.com/JarryShaw/PyPCAPKit/pull/1020)). - extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output ([#420](https://github.com/JarryShaw/PyPCAPKit/pull/420)). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% (IP reassembly submits a datagram for every frame, fragmented or not) ([#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow dumper handed each record to a `Frame` constructor that re-dissected the whole stack to return the bytes it had just been given; options are no longer parsed twice either ([#427](https://github.com/JarryShaw/PyPCAPKit/pull/427)). -- **a breaking change to** `Extractor.engine` and `Extractor.record_header`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. The built-in engines subclass `EngineBase` directly, so none was ever an `Engine`, which adds only the `__init_subclass__` registration hook. Two things are visible to a type checker. Code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). +- **a breaking change to** `Extractor.engine` and `Extractor.record_header`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. The built-in engines subclass `EngineBase` directly, so none was ever an `Engine`, which adds only the `__init_subclass__` registration hook. A type checker sees the change: Code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). - the documentation of the engine base classes: the member stubs (`name`, `module`, `registry`, `extractor`, `unsupported_reason`, `run`, `read_frame`, `close` and the rest) moved from `Engine` to `EngineBase`, where they are defined, and `Engine` now documents only the `__init_subclass__` registration hook it adds, so the `EngineBase[_P]` that `Extractor.engine` returns leads to an API. Cross-references to `Engine.run`, `Engine.read_frame`, `Engine.close` and `Engine.unsupported_reason` now point at `EngineBase`, which stays out of `__all__`, as [#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016) settled ([#1024](https://github.com/JarryShaw/PyPCAPKit/issues/1024)). -- **a breaking change to** `register_mh_extension`: `schema` is keyword-only, as in the other fourteen schema-taking registrars ([#516](https://github.com/JarryShaw/PyPCAPKit/issues/516), [#526](https://github.com/JarryShaw/PyPCAPKit/pull/526)). +- **a breaking change to** `register_mh_extension`: `schema` is keyword-only, as in the other schema-taking registrars ([#516](https://github.com/JarryShaw/PyPCAPKit/issues/516), [#526](https://github.com/JarryShaw/PyPCAPKit/pull/526)). #### Fixed - TCP reassembly mixed absolute sequence numbers with buffer-relative slicing, so on any capture carrying a SYN with a realistic initial sequence number incomplete datagrams were dropped silently, and the `completed=False` branch of the public API was unreachable ([#349](https://github.com/JarryShaw/PyPCAPKit/issues/349), [#376](https://github.com/JarryShaw/PyPCAPKit/pull/376)). - `format='text'` raised `AttributeError` before writing anything, naming a `dictdumper.Text` that has never existed. It now points at `Tree`, as the `'txt'` alias beside it already did. -- `Extractor` had the ownership of its input stream inverted: a handle it opened itself, from `fin` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. `_cleanup` closed under `not self._flag_s` (the flag that gated the `open()`), and the `SeekableReader` wrapping a non-seekable input asked for `stream_closing=not self._flag_s` in the same wrong direction. The leak was one descriptor per extraction, the root cause of the `ResourceWarning` that reddened [#577](https://github.com/JarryShaw/PyPCAPKit/pull/577), [#596](https://github.com/JarryShaw/PyPCAPKit/pull/596) and [#600](https://github.com/JarryShaw/PyPCAPKit/pull/600) ([#606](https://github.com/JarryShaw/PyPCAPKit/issues/606) had only made the test immune). Both conditions now follow ownership via a single `Extractor._owns_input`, and a finaliser releases the handle of an extraction *abandoned* before end of file (`auto=False`, iterated part way and dropped); the two files [#610](https://github.com/JarryShaw/PyPCAPKit/issues/610) names emit no such warning. `__exit__` deliberately still closes unconditionally, since a caller who scopes an `Extractor` with `with` asked for that ([#610](https://github.com/JarryShaw/PyPCAPKit/issues/610)). -- `extract(..., no_eof=True)` never returned. End of stream was detected and `ExtractionWarning: EOF reached` fired, but all three loops that handle it (`record_frames`, `__next__`, `__call__`) read `if self._flag_n: continue`, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- `__main__` sets `no_eof` for `fin='-'`, and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one: `prepare` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (`Extractor._note_eof_progress`). A pipe is unaffected, since a read there *blocks* while the writer is open but idle. This also fixes `pcapkit -`, which hung on **any** finished stdin. **One deliberate narrowing**: a *seekable* input does not block, so a file still being appended to ends at the data present when the extraction reached it; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own. Pre-existing and untouched: an append landing *mid-record* raises `ValueError: read length must be non-negative or -1`. The end-to-end tests are bounded by a **child process** rather than `tests._support.time_limit`, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than a red test. Two guesses at the cause are ruled out: the parse path's two `except Exception` handlers are never entered during the spin, and the memory is the retry loop's `EOF reached` records, which the runner retains ([#620](https://github.com/JarryShaw/PyPCAPKit/issues/620)). -- `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and three dispatchable classes are all named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the three built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The guard deliberately reads "key present and the incumbent is a different class", not mere presence ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave all seven siblings the same identity guard): this function is the funnel all nine wrapper registrars end in, so registering one class under two codes (`register_tcp` then `register_udp`, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514). The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). -- two `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` named a bare `Type`, which Sphinx resolves against every class named `Type` in the project (five) rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. Both sites now spell it `~typing.Type[Dumper]`, one pair of the sites [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) lists ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). Four more bare `Type` sites, in hand-written `:type:` fields under `docs/source/pcapkit/foundation/`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). +- `Extractor` had the ownership of its input stream inverted: a handle it opened itself, from `fin` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. `_cleanup` closed under `not self._flag_s` (the flag that gated the `open()`), and the `SeekableReader` wrapping a non-seekable input asked for `stream_closing=not self._flag_s` in the same wrong direction. The leak was one descriptor per extraction, the root cause of the `ResourceWarning` that reddened [#577](https://github.com/JarryShaw/PyPCAPKit/pull/577), [#596](https://github.com/JarryShaw/PyPCAPKit/pull/596) and [#600](https://github.com/JarryShaw/PyPCAPKit/pull/600) ([#606](https://github.com/JarryShaw/PyPCAPKit/issues/606) had only made the test immune). Both conditions now follow ownership via a single `Extractor._owns_input`, and a finaliser releases the handle of an extraction *abandoned* before end of file (`auto=False`, iterated part way and dropped); the files [#610](https://github.com/JarryShaw/PyPCAPKit/issues/610) names emit no such warning. `__exit__` deliberately still closes unconditionally, since a caller who scopes an `Extractor` with `with` asked for that ([#610](https://github.com/JarryShaw/PyPCAPKit/issues/610)). +- `extract(..., no_eof=True)` never returned. End of stream was detected and `ExtractionWarning: EOF reached` fired, but every loop that handles it (`record_frames`, `__next__`, `__call__`) read `if self._flag_n: continue`, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- `__main__` sets `no_eof` for `fin='-'`, and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one: `prepare` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (`Extractor._note_eof_progress`). A pipe is unaffected, since a read there *blocks* while the writer is open but idle. This also fixes `pcapkit -`, which hung on **any** finished stdin. **One deliberate narrowing**: a *seekable* input does not block, so a file still being appended to ends at the data present when the extraction reached it; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own. Pre-existing and untouched: an append landing *mid-record* raises `ValueError: read length must be non-negative or -1`. The end-to-end tests are bounded by a **child process** rather than `tests._support.time_limit`, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than a red test. Guesses at the cause are ruled out: the parse path's `except Exception` handlers are never entered during the spin, and the memory is the retry loop's `EOF reached` records, which the runner retains ([#620](https://github.com/JarryShaw/PyPCAPKit/issues/620)). +- `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and several dispatchable classes are named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The guard deliberately reads "key present and the incumbent is a different class", not mere presence ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave its sibling registrars the same identity guard): this function is the funnel every wrapper registrar ends in, so registering one class under two codes (`register_tcp` then `register_udp`, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514). The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). +- the `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` that named a bare `Type`, which Sphinx resolves against every class named `Type` in the project rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. These sites now spell it `~typing.Type[Dumper]`, among the sites [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) lists ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). The other bare `Type` sites, in hand-written `:type:` fields under `docs/source/pcapkit/foundation/`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). - `register_protocol`'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (`incumbent is not protocol`, an identity check from [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)), but the message built both operands with a bare `repr()`. A factory defining a same-named closure-local class on every call (as `tests/protocols/test_construction_keyword_check_unit.py`'s `_protocol_class` does) produces two distinct classes sharing one `__module__` and `__qualname__`, so a real overwrite read "overwriting X with X." The message now compares the two reprs and, only when they coincide, appends each object's `id()`; `__module__`/`__qualname__` would not disambiguate, being what the repr already renders ([#710](https://github.com/JarryShaw/PyPCAPKit/issues/710)). -- five more registrars warned on mere key presence rather than an actual overwrite, outside the wording of [#718](https://github.com/JarryShaw/PyPCAPKit/issues/718)'s identity guard (`incumbent is not None and incumbent is not new`, landed by [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)), whose issue named only code-keyed registrars: `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, and `register_dumper` on both `Extractor` and `TraceFlow`. All five now compare the incumbent by identity before warning; both `register_dumper` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings ([#739](https://github.com/JarryShaw/PyPCAPKit/issues/739)). -- **six registrars** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the two `register_dumper` methods, and from inside `abc` for the other four. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in the five that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the seven protocol `register` classmethods is fixed separately by [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). -- a missing extraction engine package now gives one `EngineWarning` rather than two ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045), [#1046](https://github.com/JarryShaw/PyPCAPKit/pull/1046)). `Extractor.import_test` warned that the package was absent, then `Extractor.run` warned again when it fell back to the default engine; `run` now stays silent there, and `import_test` says what `run` used to: `engine () is not installed; using default engine instead`, the module name in backticks. Only code that matched `import_test`'s old wording, or that expects two warnings, breaks. -- three `Extractor` defects: `Extractor.output`'s `UnsupportedCall` named `format` rather than `output`; `make_name` looked an unknown format up in the `__output__` `defaultdict` and so inserted it; and the Scapy engine never set `_offmt`, so `Extractor.format` raised `AttributeError` under `engine='scapy'` ([#1095](https://github.com/JarryShaw/PyPCAPKit/issues/1095)). -- the DPKT, Scapy and PyShark engines call `Extractor.record_header()` at setup again, as they did until 2023, so the global-header step is shared with the default engine and sets `_offmt` for all four. **Their dumped output changes**: JSON, plist and tree reports gain the leading `Global Header` record (`Section Header 1` for PCAP-NG) the default engine already writes, identical to its, and `files=True` writes that record's file too; frame records are unchanged. `Extractor.format` now also answers on a capture with no frames. `format='pcap'` failing on every engine is a separate defect, [#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127) (below) ([#1104](https://github.com/JarryShaw/PyPCAPKit/issues/1104)). -- `format='pcap'` failed on every engine with a `TypeError` from `PCAPIO.__init__()`, missing `protocol`: `Extractor.__init__` created the writer before any global header was read, so `PCAPIO` had no link type. The header step now creates it through `Extractor._open_output`, and `run()` opens it for engines with no such step; a PCAP writer takes the frame itself and skips the `Global Header` record, so the default engine writes `in.pcap` back byte for byte. The third-party engines hand the writer no frame octets, so they fall back to JSON with a `FormatWarning`, as their flow tracing already does, and a PCAP-NG input raises `FormatError` ([#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127)). The written global header also dropped the input's `thiszone`, `sigfigs` and `snaplen`, so only 1 of 17 sample PCAP files came back byte for byte; the PCAP engine now passes all three through `_open_output` to `PCAPIO`, and all 17 do ([#1165](https://github.com/JarryShaw/PyPCAPKit/issues/1165)). +- more registrars warned on mere key presence rather than an actual overwrite, outside the wording of [#718](https://github.com/JarryShaw/PyPCAPKit/issues/718)'s identity guard (`incumbent is not None and incumbent is not new`, landed by [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)), whose issue named only code-keyed registrars: `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, and `register_dumper` on both `Extractor` and `TraceFlow`. Each now compares the incumbent by identity before warning; both `register_dumper` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings ([#739](https://github.com/JarryShaw/PyPCAPKit/issues/739)). +- **the class registrars listed below** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the `register_dumper` methods, and from inside `abc` for the others. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in those that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocol `register` classmethods is fixed separately by [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). +- a missing extraction engine package now gives a single `EngineWarning` rather than a duplicate ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045), [#1046](https://github.com/JarryShaw/PyPCAPKit/pull/1046)). `Extractor.import_test` warned that the package was absent, then `Extractor.run` warned again when it fell back to the default engine; `run` now stays silent there, and `import_test` says what `run` used to: `engine () is not installed; using default engine instead`, the module name in backticks. Only code that matched `import_test`'s old wording, or that expects the duplicate warning, breaks. +- `Extractor` defects: `Extractor.output`'s `UnsupportedCall` named `format` rather than `output`; `make_name` looked an unknown format up in the `__output__` `defaultdict` and so inserted it; and the Scapy engine never set `_offmt`, so `Extractor.format` raised `AttributeError` under `engine='scapy'` ([#1095](https://github.com/JarryShaw/PyPCAPKit/issues/1095)). +- the DPKT, Scapy and PyShark engines call `Extractor.record_header()` at setup again, as they did until 2023, so the global-header step is shared with the default engine and sets `_offmt` for each of them. **Their dumped output changes**: JSON, plist and tree reports gain the leading `Global Header` record (`Section Header 1` for PCAP-NG) the default engine already writes, identical to its, and `files=True` writes that record's file too; frame records are unchanged. `Extractor.format` now also answers on a capture with no frames. `format='pcap'` failing on every engine is a separate defect, [#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127) (below) ([#1104](https://github.com/JarryShaw/PyPCAPKit/issues/1104)). +- `format='pcap'` failed on every engine with a `TypeError` from `PCAPIO.__init__()`, missing `protocol`: `Extractor.__init__` created the writer before any global header was read, so `PCAPIO` had no link type. The header step now creates it through `Extractor._open_output`, and `run()` opens it for engines with no such step; a PCAP writer takes the frame itself and skips the `Global Header` record, so the default engine writes `in.pcap` back byte for byte. The third-party engines hand the writer no frame octets, so they fall back to JSON with a `FormatWarning`, as their flow tracing already does, and a PCAP-NG input raises `FormatError` ([#1127](https://github.com/JarryShaw/PyPCAPKit/issues/1127)). The written global header also dropped the input's `thiszone`, `sigfigs` and `snaplen`, so most sample PCAP files did not come back byte for byte; the PCAP engine now passes all three through `_open_output` to `PCAPIO`, and every one does ([#1165](https://github.com/JarryShaw/PyPCAPKit/issues/1165)). - IP reassembly reports only the octets it holds: a truncated fragment, a strict run and a zero-length datagram no longer report octets from the declared Total Length ([#1368](https://github.com/JarryShaw/PyPCAPKit/pull/1368)). Only the final fragment's partial block counts, and a datagram too long for its length field never completes ([#1582](https://github.com/JarryShaw/PyPCAPKit/pull/1582), [#1585](https://github.com/JarryShaw/PyPCAPKit/issues/1585), [#1598](https://github.com/JarryShaw/PyPCAPKit/pull/1598)). - TCP reassembly judges completion from each buffer's own gaps, compares sequence numbers modulo `2**32`, and holds a FIN until reordered data arrives ([#1375](https://github.com/JarryShaw/PyPCAPKit/pull/1375)). A TSO fragment's length is its captured length, and `RCVBT` covers the last block ([#1564](https://github.com/JarryShaw/PyPCAPKit/pull/1564)). - a second TCP flow with the same label truncated the first flow's file; a reused label now gets the next free suffix ([#1360](https://github.com/JarryShaw/PyPCAPKit/issues/1360), [#1371](https://github.com/JarryShaw/PyPCAPKit/pull/1371)). -- the engine, reassembly, traceflow and dumper registrars accept the built-in a name shipped with, which restores it ([#1363](https://github.com/JarryShaw/PyPCAPKit/issues/1363), [#1376](https://github.com/JarryShaw/PyPCAPKit/pull/1376)). +- the engine, reassembly, traceflow and dumper registrars accept the built-in each name shipped with, which restores it ([#1363](https://github.com/JarryShaw/PyPCAPKit/issues/1363), [#1376](https://github.com/JarryShaw/PyPCAPKit/pull/1376)). - PCAP-NG blocks are numbered across the whole file, not per section ([#1256](https://github.com/JarryShaw/PyPCAPKit/issues/1256), [#1426](https://github.com/JarryShaw/PyPCAPKit/pull/1426)). - `format='pcap'` is kept when an unavailable engine falls back to the default one ([#1264](https://github.com/JarryShaw/PyPCAPKit/issues/1264), [#1430](https://github.com/JarryShaw/PyPCAPKit/pull/1430)). - `extract()` accepts a file object with no `name` as `fin` ([#1506](https://github.com/JarryShaw/PyPCAPKit/issues/1506), [#1509](https://github.com/JarryShaw/PyPCAPKit/pull/1509)), and reads every frame from a pipe or other non-seekable stream ([#1586](https://github.com/JarryShaw/PyPCAPKit/issues/1586), [#1595](https://github.com/JarryShaw/PyPCAPKit/pull/1595)). @@ -165,16 +167,16 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and #### Added - ESP parsing and construction [[RFC 4303](https://datatracker.ietf.org/doc/html/rfc4303)], with optional payload decryption and ICV verification through `cryptography` (`pip install pypcapkit[crypto]`) ([#378](https://github.com/JarryShaw/PyPCAPKit/pull/378)). Keys reach the dissector through a new caller-state channel, `pcapkit.corekit.context`, surfaced as the `context=` keyword on `extract()` and `Extractor`, carrying an `esp.SecurityAssociation`. Nothing here raises: with no association the SPI and sequence number are still reported and the ciphertext is left opaque with `status=NO_SA`, and a failed decryption or ICV check is recorded on the parsed result the same way. Extended Sequence Numbers, TFC padding and anti-replay are not implemented, and an unsupported cipher or MAC is refused with a clear error rather than half-processed. -- SCTP as a transport protocol [[RFC 9260](https://datatracker.ietf.org/doc/html/rfc9260)]: all 13 chunk types, 8 chunk parameters and 13 error causes, with the CRC32c both recorded and verifiable (it covers the SCTP packet alone, with no IP pseudo-header). Upper layers register on the DATA chunk's Payload Protocol Identifier through `register_sctp`, not on a port ([#379](https://github.com/JarryShaw/PyPCAPKit/pull/379)). -- NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) ([#251](https://github.com/JarryShaw/PyPCAPKit/discussions/251), [#417](https://github.com/JarryShaw/PyPCAPKit/pull/417)). Decoding is generic by ASN.1 shape rather than per-procedure, so all 81 elementary procedures and 438 protocol IEs work and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` was at first left out of the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module -- until [#910](https://github.com/JarryShaw/PyPCAPKit/issues/910) counted it among `all`'s core addons, so `all` installs it (the `all` entry under *Project infrastructure*). -- the Mobility Header registry, completed ([#383](https://github.com/JarryShaw/PyPCAPKit/pull/383), [#437](https://github.com/JarryShaw/PyPCAPKit/pull/437)): the [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options, then all 24 registered message data types, all 71 registered options (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families) and all 4 CGA extensions. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the four extensions round-trip end to end. +- SCTP as a transport protocol [[RFC 9260](https://datatracker.ietf.org/doc/html/rfc9260)]: every chunk type, chunk parameter and error cause it defines, with the CRC32c both recorded and verifiable (it covers the SCTP packet alone, with no IP pseudo-header). Upper layers register on the DATA chunk's Payload Protocol Identifier through `register_sctp`, not on a port ([#379](https://github.com/JarryShaw/PyPCAPKit/pull/379)). +- NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) ([#251](https://github.com/JarryShaw/PyPCAPKit/discussions/251), [#417](https://github.com/JarryShaw/PyPCAPKit/pull/417)). Decoding is generic by ASN.1 shape rather than per-procedure, so every elementary procedure and protocol IE works and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` was at first left out of the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module -- until [#910](https://github.com/JarryShaw/PyPCAPKit/issues/910) counted it among `all`'s core addons, so `all` installs it (the `all` entry under *Project infrastructure*). +- the Mobility Header registry, completed ([#383](https://github.com/JarryShaw/PyPCAPKit/pull/383), [#437](https://github.com/JarryShaw/PyPCAPKit/pull/437)): the [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options, then every registered message data type, every registered option (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families) and every CGA extension. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the CGA extensions round-trip end to end. - dispatch entries for dissectors that existed but were reachable from no registry ([#436](https://github.com/JarryShaw/PyPCAPKit/pull/436)): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, `L2TPv2` on UDP 1701 and OSPF at `TransType` 89. `VLAN` became an abstract base with `C_Tag` (802.1Q) and `S_Tag` (802.1ad) as concrete subclasses, so a Q-in-Q frame no longer collapses into one opaque `Raw`; `L2TP` likewise became a base, with `L2TPv2` carrying the [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) implementation. -- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other four families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) needs; [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) turned out to need a missing class instead (115 is [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3 over IP, not [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661); see the corresponding **Fixed** entry below), so its worked example names `L2TPv3` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). +- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other registrar families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) needs; [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) turned out to need a missing class instead (115 is [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3 over IP, not [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661); see the corresponding **Fixed** entry below), so its worked example names `L2TPv3` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). - `tests/protocols/test_dispatch_reachability_unit.py`, the coverage [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) asked for: every `ProtocolBase` descendant whose `__index__` returns an enum member must be reachable under that code in the registry its enum *type* designates, read from the same `_CODE_DESTINATIONS` table backing `code=` so the two cannot drift. It walks the classes rather than the registry entries, so it catches a class nothing registered, the shape in which `OSPF` once shipped ([#548](https://github.com/JarryShaw/PyPCAPKit/issues/548)). - `tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py`, covering all three MP_JOIN layouts through the *public* constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the statement order, and controls for the parse path and the flag-independent options. The old cases bypassed the public path, so full coverage coexisted with a broken one. The stale `tcp-mptcp/MP_JOIN` entry is deleted from `EXPECTED_FAILURES` ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). - `tests/protocols/test_option_generator_tcp_base_unit.py`, asserting that every `TCP_BASE` key is a parameter `TCP.make` declares (derived from `inspect.signature`) and that the built segment carries the stated header in both data model and packed octets ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). -- ILNP nonce sizing coverage in `tests/protocols/internet/test_ipv6_extension_unit.py`, one test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over ten nonces, most with a bit length that is *not* a multiple of eight, where floor division and the ceiling disagree ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). -- `SOLUTION` parameter width coverage in `tests/protocols/internet/test_hip_unit.py`: eight widths asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) requires of HIPv1. Five widths are deliberately *not* multiples of eight, where the defective `ceil(bits / 4)` coincides with the correct width ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). +- ILNP nonce sizing coverage in `tests/protocols/internet/test_ipv6_extension_unit.py`, one test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over a spread of nonces, most with a bit length that is *not* a multiple of eight, where floor division and the ceiling disagree ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). +- `SOLUTION` parameter width coverage in `tests/protocols/internet/test_hip_unit.py`: widths asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) requires of HIPv1. Some widths are deliberately *not* multiples of eight, where the defective `ceil(bits / 4)` differs from the correct width ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). - the BSD loopback layer for `LINKTYPE_NULL` and `LINKTYPE_LOOP`, and dissection of `LINKTYPE_RAW` by the version nibble ([#1594](https://github.com/JarryShaw/PyPCAPKit/pull/1594)). - one home for each RFC- or format-fixed constant spelled out in more than one module -- the IPv4, IPv6, AH and TCP header lengths and offsets, `IP_MAX_LENGTH`, and the PCAP and PCAP-NG framing constants -- in the module that owns the protocol ([#1739](https://github.com/JarryShaw/PyPCAPKit/issues/1739), [#1752](https://github.com/JarryShaw/PyPCAPKit/pull/1752)). @@ -202,8 +204,9 @@ Layer-limited extraction: `layer='link'` no longer sets the termination flag on - **a breaking change to** `OSPF_CrytographicAuthentication`: the misspelling is corrected to `OSPF_CryptographicAuthentication`, and `CrytographicAuthentication` in `pcapkit.protocols.schema.application.ospf` and `pcapkit.protocols.data.application.ospf` to `CryptographicAuthentication`, with no alias left for the old spelling. Both OSPF names also move to the application group of `schema.__all__` and `data.__all__`, after [#719](https://github.com/JarryShaw/PyPCAPKit/issues/719)'s move ([#1090](https://github.com/JarryShaw/PyPCAPKit/issues/1090)). - **a breaking change to** a declared length that runs past the data: it is refused with `ProtocolError` rather than accepted and zero-filled on rebuild, so a malformed option looks different in `info` ([#1325](https://github.com/JarryShaw/PyPCAPKit/issues/1325), [#1398](https://github.com/JarryShaw/PyPCAPKit/pull/1398)). - dissection is bounded: a frame at 64 layers ([#1610](https://github.com/JarryShaw/PyPCAPKit/issues/1610), [#1620](https://github.com/JarryShaw/PyPCAPKit/pull/1620)) and an IPv6 extension-header chain at 32 headers ([#1604](https://github.com/JarryShaw/PyPCAPKit/issues/1604), [#1608](https://github.com/JarryShaw/PyPCAPKit/pull/1608)). Both limits are `PCAPKIT_FRAME_LAYER_LIMIT` and `PCAPKIT_EXTENSION_HEADER_LIMIT` in `pcapkit.utilities.config` ([#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). Rebuilding a frame cut at the chain bound warns once ([#1619](https://github.com/JarryShaw/PyPCAPKit/issues/1619), [#1630](https://github.com/JarryShaw/PyPCAPKit/pull/1630)). -- **a breaking change to** IPv6 extension headers: an unrecognised or unparsable one is parsed generically, per [RFC 6564](https://datatracker.ietf.org/doc/html/rfc6564), rather than collapsing the packet to `Raw` ([#891](https://github.com/JarryShaw/PyPCAPKit/issues/891), [#904](https://github.com/JarryShaw/PyPCAPKit/pull/904)). `IPv6_Ext` does it, and is also the shared base of all eight extension headers ([#917](https://github.com/JarryShaw/PyPCAPKit/issues/917), [#924](https://github.com/JarryShaw/PyPCAPKit/pull/924)). +- **a breaking change to** IPv6 extension headers: an unrecognised or unparsable one is parsed generically, per [RFC 6564](https://datatracker.ietf.org/doc/html/rfc6564), rather than collapsing the packet to `Raw` ([#891](https://github.com/JarryShaw/PyPCAPKit/issues/891), [#904](https://github.com/JarryShaw/PyPCAPKit/pull/904)). `IPv6_Ext` does it, and is also the shared base of every extension header ([#917](https://github.com/JarryShaw/PyPCAPKit/issues/917), [#924](https://github.com/JarryShaw/PyPCAPKit/pull/924)). - **a breaking change to** `pack()`: it no longer keeps the header it builds on the instance it is called on, which in 1.4.1 too left `schema` disagreeing with `info`: `UDP(srcport=1234).pack()` reset `schema.srcport` to 0 while `info.srcport` kept 1234. Once an instance has been parsed or constructed, `pack()` builds on a shallow copy, so `schema`, `info` and what `make()` sets, such as TCP's `connection`, stay as they were. The `Frame`, `PCAPNG` and `HTTPv2` overrides do the same, and re-packing a PCAP-NG block that carries an option allowed only once no longer raises. Code that relied on `pack()` to re-sync `schema` calls `make()` or constructs a new instance ([#1769](https://github.com/JarryShaw/PyPCAPKit/issues/1769), [#1791](https://github.com/JarryShaw/PyPCAPKit/pull/1791)). +- **a breaking change to** the HOPOPT, IPv6-Opts and Mobility Header padding: two octets are one `PadN` with no data (`01 00`) rather than two `Pad1` options, as [RFC 8200 Section 4.2](https://datatracker.ietf.org/doc/html/rfc8200#section-4.2) and [RFC 6275 Section 6.2.2](https://datatracker.ietf.org/doc/html/rfc6275#section-6.2.2) ask. Only a freshly built header with exactly two octets of padding changes; a parsed one that carried two `Pad1` options still rebuilds byte for byte ([#1793](https://github.com/JarryShaw/PyPCAPKit/issues/1793), [#1812](https://github.com/JarryShaw/PyPCAPKit/pull/1812)). #### Fixed @@ -211,70 +214,70 @@ Layer-limited extraction: `layer='link'` no longer sets the termination flag on - the identical defect one layer up, in the schema layer's `EnumSchema.registry`: `Option.registry[code]` for an unregistered `code` inserted the default schema under that code, so a single lookup made an unassigned TCP option number such as `156` read back as registered for the rest of the process. `EnumSchema.__enum__` is now built (or, when a subclass seeds it manually, as `PCAPNG.Option`'s namespaced mapping and `TCP.MPTCP`'s plain one do) as a retention-safe mapping that still returns the registered default on a miss but stops recording it; `.registry` returns the same object as before, so nothing holding a reference is affected ([#555](https://github.com/JarryShaw/PyPCAPKit/issues/555)). - on Python 3.10 and older, no `Schema` subclass got its own `_abc_impl`: all fell through to `collections.abc.Mapping`'s, so a single `isinstance` or `issubclass` answer poisoned every later question about that class for the rest of the process. A terminating PCAP-NG `EndRecord` tested `True` as an `IPv4Record` ([#439](https://github.com/JarryShaw/PyPCAPKit/issues/439)). - construction, broken in several places at once: the generated typed `__init__` was never installed, so `__post_init__` did not run and a schema built from a subset of its fields could not be packed -- `UDP(srcport=53, dstport=5353)` now packs ([#430](https://github.com/JarryShaw/PyPCAPKit/pull/430)); IPv6 and Mobility Header option padding was wrong, leaving construction wholly broken ([#398](https://github.com/JarryShaw/PyPCAPKit/pull/398)); `HTTP.make` called the versioned `make` unbound, so every real call raised `TypeError` ([#452](https://github.com/JarryShaw/PyPCAPKit/issues/452), [#462](https://github.com/JarryShaw/PyPCAPKit/pull/462)); and `IPv4._make_data` returned the fragment offset in octets where the wire wants 8-octet units, and read `data.options` on a packet that has none ([#494](https://github.com/JarryShaw/PyPCAPKit/issues/494), [#499](https://github.com/JarryShaw/PyPCAPKit/pull/499)). -- a truncated or under-declared area no longer parses "successfully", and no longer wedges the process. The option and list loops could spin forever with no exception on a truncated area, reachable from untrusted input through HOPOPT, IPv6-Opts, MH, HIP and SCTP; each iteration must now advance the stream by at least one octet, and the error names the option, the offset and the octets remaining ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431), [#432](https://github.com/JarryShaw/PyPCAPKit/pull/432)). Separately, wire-derived lengths in `ipv6_opts`, CALIPSO, MPL, REG_INFO and four HIP list callbacks underflowed below zero, which `ListField`'s own `while length > 0` turned into a silent empty list; they are floored and raise instead ([#449](https://github.com/JarryShaw/PyPCAPKit/pull/449), [#456](https://github.com/JarryShaw/PyPCAPKit/pull/456), [#460](https://github.com/JarryShaw/PyPCAPKit/pull/460), [#463](https://github.com/JarryShaw/PyPCAPKit/issues/463)). +- a truncated or under-declared area no longer parses "successfully", and no longer wedges the process. The option and list loops could spin forever with no exception on a truncated area, reachable from untrusted input through HOPOPT, IPv6-Opts, MH, HIP and SCTP; each iteration must now advance the stream by at least one octet, and the error names the option, the offset and the octets remaining ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431), [#432](https://github.com/JarryShaw/PyPCAPKit/pull/432)). Separately, wire-derived lengths in `ipv6_opts`, CALIPSO, MPL, REG_INFO and HIP list callbacks underflowed below zero, which `ListField`'s own `while length > 0` turned into a silent empty list; they are floored and raise instead ([#449](https://github.com/JarryShaw/PyPCAPKit/pull/449), [#456](https://github.com/JarryShaw/PyPCAPKit/pull/456), [#460](https://github.com/JarryShaw/PyPCAPKit/pull/460), [#463](https://github.com/JarryShaw/PyPCAPKit/issues/463)). - field widths and units, each measured against the specification: HIP's `TRANSPORT_FORMAT_LIST`, `NAT_TRAVERSAL_MODE` and `ESP_TRANSFORM` list entries are two octets, not one [[RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401), [RFC 5770](https://datatracker.ietf.org/doc/html/rfc5770), [RFC 7402](https://datatracker.ietf.org/doc/html/rfc7402)] ([#463](https://github.com/JarryShaw/PyPCAPKit/issues/463), [#472](https://github.com/JarryShaw/PyPCAPKit/issues/472)); the MN-ID option sizes from its subtype, not from `identifier`'s Python type ([#448](https://github.com/JarryShaw/PyPCAPKit/issues/448), [#464](https://github.com/JarryShaw/PyPCAPKit/pull/464), [#467](https://github.com/JarryShaw/PyPCAPKit/issues/467)); IPv6-Route's `Hdr Ext Len` is computed in 8-octet units on both sides [[RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200)] ([#487](https://github.com/JarryShaw/PyPCAPKit/issues/487), [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489)); and the Fast Binding Update and Acknowledgment Lifetimes are plain seconds [[RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568)], not [RFC 6275](https://datatracker.ietf.org/doc/html/rfc6275)'s four-second units ([#502](https://github.com/JarryShaw/PyPCAPKit/pull/502)). -- the last four call sites where a `_make_*` method converts an address itself, ahead of the schema, so the same `bool`-as-`int` mistake reached them, unguarded by the schema-layer fix above: `ARP._make_proto_resolve`, `IPv6_Route._make_data_type_rpl` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the `dst` parameter of `IPv6_Route.make`, and the latent `OSPF._make_id_numbers`. All four now go through `parse_ip_address`, the helper [#539](https://github.com/JarryShaw/PyPCAPKit/pull/539) and [#552](https://github.com/JarryShaw/PyPCAPKit/issues/552) route their own sites through, completing the programme [#481](https://github.com/JarryShaw/PyPCAPKit/pull/481) and [#508](https://github.com/JarryShaw/PyPCAPKit/issues/508) started. Pinning `version=6` on the two `IPv6_Route` sites is a second, smaller behaviour change: both converted a plain integer through the family-inferring `ipaddress.ip_address`, so `ip=[258]` packed a 4-octet IPv4 address (`0.0.1.2`) inside an IPv6-only header; it now packs the 16-octet `::102` ([#540](https://github.com/JarryShaw/PyPCAPKit/issues/540)). -- PCAP and PCAP-NG output and parsing: `bytes(frame)` returned the *next* frame's octets, `files=True` wrote names like `Frame 1..json`, and a PCAP-NG `timestamp_epoch` was shifted by the reading host's timezone ([#403](https://github.com/JarryShaw/PyPCAPKit/pull/403)); seven further parser defects ([#341](https://github.com/JarryShaw/PyPCAPKit/issues/341)--[#347](https://github.com/JarryShaw/PyPCAPKit/issues/347), [#371](https://github.com/JarryShaw/PyPCAPKit/pull/371)) and, on the write path, four more block-parsing ones ([#388](https://github.com/JarryShaw/PyPCAPKit/pull/388)); `BitField` packed every named bit as set ([#359](https://github.com/JarryShaw/PyPCAPKit/issues/359), [#374](https://github.com/JarryShaw/PyPCAPKit/pull/374)); the extension-header walk failed to advance past the last IPv6 extension header ([#348](https://github.com/JarryShaw/PyPCAPKit/issues/348), [#373](https://github.com/JarryShaw/PyPCAPKit/pull/373)); and IPv6 fragment offsets went unscaled, with reassembly keyed on the flow label -- optional, and routinely zero, so distinct datagrams collapsed together -- rather than on the fragment identification ([#389](https://github.com/JarryShaw/PyPCAPKit/pull/389)). +- the remaining call sites where a `_make_*` method converts an address itself, ahead of the schema, so the same `bool`-as-`int` mistake reached them, unguarded by the schema-layer fix above: `ARP._make_proto_resolve`, `IPv6_Route._make_data_type_rpl` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the `dst` parameter of `IPv6_Route.make`, and the latent `OSPF._make_id_numbers`. Each now goes through `parse_ip_address`, the helper [#539](https://github.com/JarryShaw/PyPCAPKit/pull/539) and [#552](https://github.com/JarryShaw/PyPCAPKit/issues/552) route their own sites through, completing the programme [#481](https://github.com/JarryShaw/PyPCAPKit/pull/481) and [#508](https://github.com/JarryShaw/PyPCAPKit/issues/508) started. Pinning `version=6` on the `IPv6_Route` sites is a second, smaller behaviour change: both converted a plain integer through the family-inferring `ipaddress.ip_address`, so `ip=[258]` packed a 4-octet IPv4 address (`0.0.1.2`) inside an IPv6-only header; it now packs the 16-octet `::102` ([#540](https://github.com/JarryShaw/PyPCAPKit/issues/540)). +- PCAP and PCAP-NG output and parsing: `bytes(frame)` returned the *next* frame's octets, `files=True` wrote names like `Frame 1..json`, and a PCAP-NG `timestamp_epoch` was shifted by the reading host's timezone ([#403](https://github.com/JarryShaw/PyPCAPKit/pull/403)); further parser defects ([#341](https://github.com/JarryShaw/PyPCAPKit/issues/341)--[#347](https://github.com/JarryShaw/PyPCAPKit/issues/347), [#371](https://github.com/JarryShaw/PyPCAPKit/pull/371)) and, on the write path, more block-parsing ones ([#388](https://github.com/JarryShaw/PyPCAPKit/pull/388)); `BitField` packed every named bit as set ([#359](https://github.com/JarryShaw/PyPCAPKit/issues/359), [#374](https://github.com/JarryShaw/PyPCAPKit/pull/374)); the extension-header walk failed to advance past the last IPv6 extension header ([#348](https://github.com/JarryShaw/PyPCAPKit/issues/348), [#373](https://github.com/JarryShaw/PyPCAPKit/pull/373)); and IPv6 fragment offsets went unscaled, with reassembly keyed on the flow label -- optional, and routinely zero, so distinct datagrams collapsed together -- rather than on the fragment identification ([#389](https://github.com/JarryShaw/PyPCAPKit/pull/389)). - `TCP._make_mptcp_addaddr` could not build an `ADD_ADDR` option end to end: its `kind=`/`length=` arguments were rejected with `UnknownFieldWarning` and dropped, and `.pack()` then raised `KeyError: 'length'` from `port`'s own condition, `pkt['length'] in (10, 22)`. The cause was one layer up: `MPTCP`, the base of every Multipath TCP subtype schema, declared `kind` and `length` only under `typing.TYPE_CHECKING` rather than as real fields, unlike `Option`. That dropped `kind=`/`length=` for every `_make_mptcp_*` constructor, so `MPTCP` now declares both for real ([#541](https://github.com/JarryShaw/PyPCAPKit/issues/541)). The missing fields broke parsing too: a subtype schema's leading field read the `kind` octet itself, an off-by-two in field alignment rather than a wire-format change (a correct sender's octets were always right). Spec-correct `ADD_ADDR` and `MP_PRIO` options failed with `FieldError: TCP: [OptNo 30] 3 invalid IP version` and `KeyError: 'length'`; both parse now. -- two dropped-keyword/wrong-cast defects flagged in review during this release and filed only as [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556): HIP's `_make_param_encrypted` passed `cipher=` to a schema with no such field, so an AES-cipher `ENCRYPTED` parameter built through `make` packed without its IV; and IPv6-Route's `RPL.post_process`, which runs on every `Schema.pack`, assumed `self.addresses` was still the concatenated `bytes` a parse leaves, and raised slicing the `list[bytes]` a `make`-built multi-address header holds ([#556](https://github.com/JarryShaw/PyPCAPKit/issues/556)). -- two more defects [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) exposed by letting construction reach code that had never run. `MPTCP.subtype` was still `typing.TYPE_CHECKING`-only, so `TCP(options=[(Enum_Option.Multipath_TCP, ...)])` raised `AttributeError: ... has no attribute 'subtype'` for every subtype but `MP_JOIN`: the convenience constructor reads the schema straight back through `_read_mptcp_*` with no byte round trip, so `_MPTCP.post_process`, the only code that set `subtype`, never ran. Fixed on the construction path (`TCP._make_mode_mp`) rather than by adding a third real field the way `kind`/`length` got in [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541): `subtype` is already packed as 4 bits of each subtype's own `test` bitfield, and a second field for the same bits would double-encode them or need a "derive, don't pack" field kind this library lacks ([#566](https://github.com/JarryShaw/PyPCAPKit/issues/566)). Separately, `_make_mptcp_capable` wrote `length=20 if rkey is None else 32` where [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 gives 12 and 20, and `MPTCPCapable.rkey`'s condition (`pkt['length'] != 32`) dropped the receiver's key for exactly the length the maker used for "key present": a key-absent MP_CAPABLE could not be built, and a key-present one lost its key on the wire. Both, and the guard in `_read_mptcp_capable`, now agree on 12/20. **This changes MP_CAPABLE's packed output**: a 20-octet key-present option built or parsed under the old code becomes 12 octets with no key, or 20 octets with the key present, depending on which the caller meant ([#567](https://github.com/JarryShaw/PyPCAPKit/issues/567)). -- TCP's MP_CAPABLE now parses and builds all five lengths of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where `_read_mptcp_capable` raised `ProtocolError: TCP: [OptNo 30] invalid format` for anything but 12 and 20. In the schema `skey` is now present only for `length > 4` and `rkey` for `length > 12`, and two fields are new: `dl_len` (16 bits, `length > 20`) and `checksum` (2 octets, `length > 22`). `_make_mptcp_capable` takes `dl_len` and `checksum` too, where it used to drop them silently through `**kwargs`, and picks the length from how many of `skey`, `rkey`, `dl_len` and `checksum` are not `None`. A field given without the one before it raises `ProtocolError`. The data model `MPTCPCapable` gains `dl_len` and `checksum`, and `skey` becomes `Optional[int]`: code that constructs it directly must now pass both new fields, and `to_dict()` carries two more keys ([#1042](https://github.com/JarryShaw/PyPCAPKit/issues/1042)). -- IPv6-Route's RPL routing data was five octets wide at the front where [RFC 6554](https://datatracker.ietf.org/doc/html/rfc6554) section 3 gives four, with three further defects behind it. `CmprI`, `CmprE` and `Pad` are each 4 bits sharing a 32-bit word with a 20-bit `Reserved`, but the schema declared `cmpr_i` and `cmpr_e` as whole octets, so a constructed two-address header packed to 41 octets while its `Hdr Ext Len` of 5 declared 48. Correcting the width is a **wire-format change** and reshapes the schema: `RPL(cmpr_i=..., cmpr_e=...)` is now `RPL(cmpr={'cmpr_i': ..., 'cmpr_e': ...})`, beside the existing `pad={'pad_len': ...}`. Behind it, the reader's `header.length % 16` guard read `Hdr Ext Len` as an octet count and assumed 16-octet addresses, which an SRH carries only when `CmprI` and `CmprE` are both 0 -- the unit confusion [#487](https://github.com/JarryShaw/PyPCAPKit/issues/487) fixed for Source Route and Type 2, left by [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489) for want of a working RPL round trip to validate against. It is replaced by section 4.2's address-count arithmetic, `n = (((Hdr Ext Len * 8) - Pad - (16 - CmprE)) / (16 - CmprI)) + 1`, which the reader now requires to be non-negative and whole. `RPL.post_process` subtracted `pad_len` a second time from a buffer whose length callback had already taken it off, losing one address per `16 - CmprI` octets of padding, and set `ip` only when it had parsed octets, so `IPv6_Route(type=..., data={'ip': [...]})` raised a bare `AttributeError: 'RPL' object has no attribute 'ip'`. And `_make_data_type_rpl` computed `Pad` as `8 - length % 8` without the outer `% 8`, so an aligned address vector got a full 8 octets of padding where section 3 requires `Pad` to be 0 when `CmprI` and `CmprE` are both 0. The narrower `cmpr_i` also removes a latent divide-by-zero (a whole octet could hold 16, making `16 - CmprI` zero). `ipv6-route-type/RPL_Source_Route_Header` round-trips and its `EXPECTED_FAILURES` entry is deleted; none of this has been checked against a real RPL capture ([#564](https://github.com/JarryShaw/PyPCAPKit/issues/564)). +- dropped-keyword/wrong-cast defects flagged in review during this release and filed only as [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556): HIP's `_make_param_encrypted` passed `cipher=` to a schema with no such field, so an AES-cipher `ENCRYPTED` parameter built through `make` packed without its IV; and IPv6-Route's `RPL.post_process`, which runs on every `Schema.pack`, assumed `self.addresses` was still the concatenated `bytes` a parse leaves, and raised slicing the `list[bytes]` a `make`-built multi-address header holds ([#556](https://github.com/JarryShaw/PyPCAPKit/issues/556)). +- further defects [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) exposed by letting construction reach code that had never run. `MPTCP.subtype` was still `typing.TYPE_CHECKING`-only, so `TCP(options=[(Enum_Option.Multipath_TCP, ...)])` raised `AttributeError: ... has no attribute 'subtype'` for every subtype but `MP_JOIN`: the convenience constructor reads the schema straight back through `_read_mptcp_*` with no byte round trip, so `_MPTCP.post_process`, the only code that set `subtype`, never ran. Fixed on the construction path (`TCP._make_mode_mp`) rather than by adding a third real field the way `kind`/`length` got in [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541): `subtype` is already packed as 4 bits of each subtype's own `test` bitfield, and a second field for the same bits would double-encode them or need a "derive, don't pack" field kind this library lacks ([#566](https://github.com/JarryShaw/PyPCAPKit/issues/566)). Separately, `_make_mptcp_capable` wrote `length=20 if rkey is None else 32` where [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 gives 12 and 20, and `MPTCPCapable.rkey`'s condition (`pkt['length'] != 32`) dropped the receiver's key for exactly the length the maker used for "key present": a key-absent MP_CAPABLE could not be built, and a key-present one lost its key on the wire. Both, and the guard in `_read_mptcp_capable`, now agree on 12/20. **This changes MP_CAPABLE's packed output**: a 20-octet key-present option built or parsed under the old code becomes 12 octets with no key, or 20 octets with the key present, depending on which the caller meant ([#567](https://github.com/JarryShaw/PyPCAPKit/issues/567)). +- TCP's MP_CAPABLE now parses and builds all five lengths of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where `_read_mptcp_capable` raised `ProtocolError: TCP: [OptNo 30] invalid format` for anything but 12 and 20. In the schema `skey` is now present only for `length > 4` and `rkey` for `length > 12`, and the fields `dl_len` (16 bits, `length > 20`) and `checksum` (2 octets, `length > 22`) are new. `_make_mptcp_capable` takes `dl_len` and `checksum` too, where it used to drop them silently through `**kwargs`, and picks the length from how many of `skey`, `rkey`, `dl_len` and `checksum` are not `None`. A field given without the one before it raises `ProtocolError`. The data model `MPTCPCapable` gains `dl_len` and `checksum`, and `skey` becomes `Optional[int]`: code that constructs it directly must now pass both new fields, and `to_dict()` carries the new keys ([#1042](https://github.com/JarryShaw/PyPCAPKit/issues/1042)). +- IPv6-Route's RPL routing data was five octets wide at the front where [RFC 6554](https://datatracker.ietf.org/doc/html/rfc6554) section 3 gives four, with further defects behind it. `CmprI`, `CmprE` and `Pad` are each 4 bits sharing a 32-bit word with a 20-bit `Reserved`, but the schema declared `cmpr_i` and `cmpr_e` as whole octets, so a constructed two-address header packed to 41 octets while its `Hdr Ext Len` of 5 declared 48. Correcting the width is a **wire-format change** and reshapes the schema: `RPL(cmpr_i=..., cmpr_e=...)` is now `RPL(cmpr={'cmpr_i': ..., 'cmpr_e': ...})`, beside the existing `pad={'pad_len': ...}`. Behind it, the reader's `header.length % 16` guard read `Hdr Ext Len` as an octet count and assumed 16-octet addresses, which an SRH carries only when `CmprI` and `CmprE` are both 0 -- the unit confusion [#487](https://github.com/JarryShaw/PyPCAPKit/issues/487) fixed for Source Route and Type 2, left by [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489) for want of a working RPL round trip to validate against. It is replaced by section 4.2's address-count arithmetic, `n = (((Hdr Ext Len * 8) - Pad - (16 - CmprE)) / (16 - CmprI)) + 1`, which the reader now requires to be non-negative and whole. `RPL.post_process` subtracted `pad_len` a second time from a buffer whose length callback had already taken it off, losing one address per `16 - CmprI` octets of padding, and set `ip` only when it had parsed octets, so `IPv6_Route(type=..., data={'ip': [...]})` raised a bare `AttributeError: 'RPL' object has no attribute 'ip'`. And `_make_data_type_rpl` computed `Pad` as `8 - length % 8` without the outer `% 8`, so an aligned address vector got a full 8 octets of padding where section 3 requires `Pad` to be 0 when `CmprI` and `CmprE` are both 0. The narrower `cmpr_i` also removes a latent divide-by-zero (a whole octet could hold 16, making `16 - CmprI` zero). `ipv6-route-type/RPL_Source_Route_Header` round-trips and its `EXPECTED_FAILURES` entry is deleted; none of this has been checked against a real RPL capture ([#564](https://github.com/JarryShaw/PyPCAPKit/issues/564)). - `L2TPv2` parsed any version nibble, so an L2TPv3 datagram was reported as v2 with a tunnel and session ID read out of v3's Control Connection ID. [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) §3.1 fixes `Ver` at 2 and reserves 1 for L2F, and `L2TPv2.version` documented that any other value "is a different protocol reached through a different class", but nothing enforced it, so the hard-coded `Literal[2]` property and `info.version` answered 2 and 3 on the same octets. `read` now raises `ProtocolError`, degrading the payload to `Raw` through the existing `beholder` path with the reason recorded. **This affects real captures**: [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) §4.1.2 puts L2TPv3 on port 1701 too, the port `UDP.__proto__` binds, so `Ethernet:IPv4:UDP:L2TPv2:Raw` with invented field values becomes `Ethernet:IPv4:UDP:Raw` with the octets preserved. This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which reported `TransType.L2TP` (115) as registered nowhere and proposed binding `L2TPv2` there. That binding is wrong rather than awkward: [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) §4.1.1 gives 115 to *L2TPv3 over IP*, whose session header carries **no version nibble at all**, so a v2 parser cannot detect that the datagram is not its own. 115 is a missing *class*, not a missing registration, and stays unbound until an `L2TPv3` class exists; no dissector was invented to fill it. The reasoning is recorded in `pcapkit.protocols.link.l2tp`, and `register_protocol_code`'s worked example, which named `L2TPv2` at 115, names `L2TPv3` instead ([#548](https://github.com/JarryShaw/PyPCAPKit/issues/548)). - building any `MP_JOIN` option raised `AttributeError: 'TCP' object has no attribute '_flags'`. `TCP.make` constructed the options *before* assigning the `self._flags` that the option makers read, and `_make_mptcp_join` branches on it to choose among the three layouts of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2 (figure 5, SYN, 12 octets; figure 6, SYN/ACK, 16; figure 7, ACK, 24). The parse path was never affected, since `read` assigns the flags first. Fixed by hoisting the flag resolution above the `_make_tcp_options` call, leaving only the data-offset computation, which needs the options' total length, after it. **Not** fixed by a zero default for `_flags`, which would have been worse: `Protocol.pack` is public and calls `make`, so an instance that had parsed a segment had the attribute set and silently built the option for the segment it had *read*. A parsed MP_JOIN-SYN instance asked to pack an MP_JOIN-ACK segment emitted an ACK header carrying figure 5's 12-octet SYN option, with the caller's 20-octet HMAC replaced by an all-zero token and nonce, and nothing raised. Hoisting also made one branch reachable for the first time, an MP_JOIN on a segment with neither SYN nor ACK: the accumulator was seeded with `cast('Enum_Flags', 0)`, and `typing.cast` being a runtime no-op, `self._flags` stayed a plain `int` and the first membership test raised `TypeError` rather than the documented `ProtocolError`. It is now seeded with `Enum_Flags(0)`, a real flagless member that equals `0` and ORs identically. The read path's seed is unchanged: a flagless MP_JOIN is rejected by `mptcp_data_selector` before `_read_mptcp_join` runs ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). -- the ILNP Nonce option builders in `HOPOPT` and `IPv6_Opts` sized the option with `math.ceil(nonce.bit_length() // 8)`, floor division dressed as a ceiling (`//` floors and `math.ceil` of an `int` is a no-op). The nonce is packed by a `NumberField` whose width *is* the declared `len`, so an under-declared length silently truncated the nonce on the wire. Every nonce whose bit length was not a multiple of eight was affected, and **small values were the worst case rather than boundary values**: any nonce below 256 was declared as *zero* octets and dropped (`nonce=9` packed to `b'\x8b\x00'` and parsed back as `0`). Fixed to `max(1, math.ceil(nonce.bit_length() / 8))`, the form used at five other sizing sites in `hip.py` and `mh.py`. The one-octet floor is the `mh.py` convention and is load-bearing because `nonce` defaults to `0`, whose bit length is `0`: without it the default builds an option with no Nonce Value field at all, collapsing "the nonce is 0" into "there is no nonce" when [RFC 6744](https://datatracker.ietf.org/doc/html/rfc6744) gives the option that field. The read path takes its width from the `len` octet on the wire and was never affected ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). +- the ILNP Nonce option builders in `HOPOPT` and `IPv6_Opts` sized the option with `math.ceil(nonce.bit_length() // 8)`, floor division dressed as a ceiling (`//` floors and `math.ceil` of an `int` is a no-op). The nonce is packed by a `NumberField` whose width *is* the declared `len`, so an under-declared length silently truncated the nonce on the wire. Every nonce whose bit length was not a multiple of eight was affected, and **small values were the worst case rather than boundary values**: any nonce below 256 was declared as *zero* octets and dropped (`nonce=9` packed to `b'\x8b\x00'` and parsed back as `0`). Fixed to `max(1, math.ceil(nonce.bit_length() / 8))`, the form used at the other sizing sites in `hip.py` and `mh.py`. The one-octet floor is the `mh.py` convention and is load-bearing because `nonce` defaults to `0`, whose bit length is `0`: without it the default builds an option with no Nonce Value field at all, collapsing "the nonce is 0" into "there is no nonce" when [RFC 6744](https://datatracker.ietf.org/doc/html/rfc6744) gives the option that field. The read path takes its width from the `len` octet on the wire and was never affected ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). - `httpv1`'s `_RE_METHOD` was unanchored and `re.match` anchors only at the start, so it prefix-matched, and the request-line reader passed the whole `para1` to `Method.get` rather than the captured `method` group. So `b'Get'` matched on the single character `G`, satisfied the guard that decides a start-line is a request, and handed the mixed-case token to a lookup that raised on it. Fixing either half alone still gives a wrong answer (normalising the lookup would parse `b'Get'` as `GET` off a one-character match; passing the group would parse it as a method named `G`). The pattern is anchored at both ends and the captured group is looked up, so a non-method token is a malformed request line rather than a mis-parsed one. Method tokens are case-sensitive per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so no `re.I` was added: `GET` parses, `Get` and `get` are rejected ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). -- `_RE_STATUS` in the same reader had the same unanchored prefix defect, found by auditing `_RE_METHOD`'s siblings, and escaped as the wrong exception type. The pattern is only a guard (the value comes from `int(para2)` on the raw token), so a prefix match let a malformed status past the guard and out of `int()` uncaught, where `_read_http_header` documents `ProtocolError`: `200x` raised `ValueError: invalid literal for int()` and `2000` raised `ValueError: 2000 is not a valid StatusCode`; both are now `ProtocolError`. [RFC 9112 Section 4](https://datatracker.ietf.org/doc/html/rfc9112#section-4) gives `status-code = 3DIGIT`, exactly three, so the anchor is what the grammar said (the production lives in HTTP/1.1 because `status-code` is part of its `status-line`; [RFC 9110 Section 15](https://datatracker.ietf.org/doc/html/rfc9110#section-15) covers code semantics and the registry). `_RE_VERSION` was audited and is safe, since both call sites read the captured group ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). -- as a consequence of the above, the unhandled `MemoryError` at `pcapkit/protocols/protocol.py:1411` on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack` eleven lines earlier spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). +- `_RE_STATUS` in the same reader had the same unanchored prefix defect, found by auditing `_RE_METHOD`'s siblings, and escaped as the wrong exception type. The pattern is only a guard (the value comes from `int(para2)` on the raw token), so a prefix match let a malformed status past the guard and out of `int()` uncaught, where `_read_http_header` documents `ProtocolError`: `200x` raised `ValueError: invalid literal for int()` and `2000` raised `ValueError: 2000 is not a valid StatusCode`; both are now `ProtocolError`. [RFC 9112 Section 4](https://datatracker.ietf.org/doc/html/rfc9112#section-4) gives `status-code = 3DIGIT`, exactly three, so the anchor is what the grammar said (the production lives in HTTP/1.1 because `status-code` is part of its `status-line`; [RFC 9110 Section 15](https://datatracker.ietf.org/doc/html/rfc9110#section-15) covers code semantics and the registry). `_RE_VERSION` was audited and is safe, since every call site reads the captured group ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). +- as a consequence of the above, the unhandled `MemoryError` in `pcapkit/protocols/protocol.py`'s `self._file.read()` call on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). - documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, whereas a `SwitchField` always resolves to a concrete field ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). - the one assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, red on `main` since [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) merged. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - the HIP `SOLUTION` builder sized the parameter with `4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)`, an invalid shorthand for two fields. [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5) gives `Length` as `4 + RHASH_len / 4` over a `Random #I` and a `Puzzle solution #J` of `RHASH_len / 8` octets **each**, and `/ 4` equals twice `/ 8` only because `RHASH_len` is a whole number of octets. For an arbitrary `bit_length()` the builder emitted an *odd* contents width, which its own reader refused: `_read_param_solution`'s `(schema.len - 4) % 2` guard exists because `SolutionParameter` splits that width into two equal halves. `random=0x1` with `solution=0xfff` declared `len=7` and raised `ProtocolError: HIPv2: [ParamNo 321] invalid format` on input the library had produced. The undersized length was worse: both fields take their width from that `len`, so `solution=0xfff` was packed into one octet and came back as `0xff`, silently. Fixed to `4 + 2 * math.ceil(max(...) / 8)`, the form the sibling `PUZZLE` builder uses, which is even by construction. Loosening the reader was rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). - `TCP.read` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. One dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). - **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading** `frame.len` **or** `frame.cap_len` **from a** `.pcap` **gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). -- the HIP `PUZZLE` and `SOLUTION` builders derived three wire-format values from the payload value instead of the data model, and all three were wrong. **This changes two public data models and the octets both parameters emit.** The width of `Random #I` and `Puzzle solution #J` came from `int.bit_length()` alone, so every leading zero octet was dropped on re-serialisation: a `SOLUTION` read with `Length = 20` rebuilt as `Length = 6`, a `PUZZLE` with `Length = 12` as `Length = 5`, silently. That width is `RHASH_len / 8` octets [[RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4), [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5)], a property of the Responder's HIT Suite, so both data models now carry it as `rhash_len` and both builders prefer it; a new `rhash_len=` keyword states it for a build from scratch, which under HIPv2 is the only place it can come from, since `RSA,DSA/SHA-256` is the REQUIRED HIT Suite [[RFC 7401 Section 5.2.10](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.10)] and makes the field 32 octets rather than 8. On the mandatory path about one parameter in 256 has a zero top octet and lost it ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653)). `SOLUTION`'s second contents octet is `Reserved`, "zero when sent, ignored when received" [[RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5), and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) identically], not the `Lifetime` that only [RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4) defines, which pcapkit encoded there as `2^(value - 32)` seconds. It wrote `0x20`, `0x21`, `0x25` or `0x2b` into a field the RFC requires to be zero, and could not write that zero: a conformant `SOLUTION` with `Reserved` `0x00` parsed to `timedelta(0)` and could not be re-serialised, escaping a bare `ValueError` from `math.log2(0.0)` that was not a `BaseError`. The field is renamed `reserved`, defaults to zero, and is carried verbatim across a round trip ([#654](https://github.com/JarryShaw/PyPCAPKit/issues/654)). And neither builder read its own `version` keyword, so `version=1` and `version=2` computed identical lengths, where [RFC 5201 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.4) and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) state both fields as literally 8 bytes and `Length` as literally 12 and 20. Under HIPv1 each builder accepted only a `bit_length()` of 57..64 and built, for anything narrower, a parameter this library's reader rejects -- the shape of [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608), in the `PUZZLE` builder that [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629) never opened ([#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). The two remaining `math.log2` sites, both in `PUZZLE` where a lifetime is real, raise `ProtocolError` (`(BaseError, ValueError)`, so a caller written around the bare exception still catches it). Migration: `SolutionParameter.lifetime` is now `reserved` and an `int` rather than a `timedelta`; `PuzzleParameter` and `SolutionParameter` gained a required `rhash_len`; and `_make_param_solution` no longer takes `lifetime=`. All three were reachable end to end only once [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608) was fixed in [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629), which removed the parity guard that had failed these rebuilds loudly; the round trip then stopped raising and started quietly emitting a different parameter, which is why they were fixed together ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653), [#654](https://github.com/JarryShaw/PyPCAPKit/issues/654), [#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). -- every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **45 of the 46** HIP parameters; `LOCATOR_SET` is excluded, below. **This changes the octets those 45 parameters write, and the** `length` **their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; all 95 padding sites computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. 94 of the 95 sites (45 of 46 `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and 49 of 49 record lengths in `pcapkit/protocols/internet/hip.py`) are now one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because two defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other 44 move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). -- two HTTP/2 flag defects, one on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only one of the six `_make_http_*` methods that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the five sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at all six of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the five frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since all five pass `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: three payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). +- the HIP `PUZZLE` and `SOLUTION` builders derived wire-format values from the payload value instead of the data model, and each was wrong. **This changes the** `PuzzleParameter` **and** `SolutionParameter` **data models and the octets both parameters emit.** The width of `Random #I` and `Puzzle solution #J` came from `int.bit_length()` alone, so every leading zero octet was dropped on re-serialisation: a `SOLUTION` read with `Length = 20` rebuilt as `Length = 6`, a `PUZZLE` with `Length = 12` as `Length = 5`, silently. That width is `RHASH_len / 8` octets [[RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4), [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5)], a property of the Responder's HIT Suite, so both data models now carry it as `rhash_len` and both builders prefer it; a new `rhash_len=` keyword states it for a build from scratch, which under HIPv2 is the only place it can come from, since `RSA,DSA/SHA-256` is the REQUIRED HIT Suite [[RFC 7401 Section 5.2.10](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.10)] and makes the field 32 octets rather than 8. On the mandatory path about one parameter in 256 has a zero top octet and lost it ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653)). `SOLUTION`'s second contents octet is `Reserved`, "zero when sent, ignored when received" [[RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5), and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) identically], not the `Lifetime` that only [RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4) defines, which pcapkit encoded there as `2^(value - 32)` seconds. It wrote `0x20`, `0x21`, `0x25` or `0x2b` into a field the RFC requires to be zero, and could not write that zero: a conformant `SOLUTION` with `Reserved` `0x00` parsed to `timedelta(0)` and could not be re-serialised, escaping a bare `ValueError` from `math.log2(0.0)` that was not a `BaseError`. The field is renamed `reserved`, defaults to zero, and is carried verbatim across a round trip ([#654](https://github.com/JarryShaw/PyPCAPKit/issues/654)). And neither builder read its own `version` keyword, so `version=1` and `version=2` computed identical lengths, where [RFC 5201 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.4) and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) state both fields as literally 8 bytes and `Length` as literally 12 and 20. Under HIPv1 each builder accepted only a `bit_length()` of 57..64 and built, for anything narrower, a parameter this library's reader rejects -- the shape of [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608), in the `PUZZLE` builder that [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629) never opened ([#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). The remaining `math.log2` sites, in `PUZZLE` where a lifetime is real, raise `ProtocolError` (`(BaseError, ValueError)`, so a caller written around the bare exception still catches it). Migration: `SolutionParameter.lifetime` is now `reserved` and an `int` rather than a `timedelta`; `PuzzleParameter` and `SolutionParameter` gained a required `rhash_len`; and `_make_param_solution` no longer takes `lifetime=`. Each was reachable end to end only once [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608) was fixed in [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629), which removed the parity guard that had failed these rebuilds loudly; the round trip then stopped raising and started quietly emitting a different parameter, which is why they were fixed together ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653), [#654](https://github.com/JarryShaw/PyPCAPKit/issues/654), [#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). +- every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **every HIP parameter but** `LOCATOR_SET`, excluded below. **This changes the octets those parameters write, and the** `length` **their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; every padding site computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. Every site but `LOCATOR_SET`'s (the `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and the record lengths in `pcapkit/protocols/internet/hip.py`) now goes through one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other parameters move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). +- HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only `_make_http_*` method that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at each of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since each passes `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). - four MPTCP error messages carried a doubled separator, rendering as `TCP: : [OptNo 30] 1: invalid flags combination`. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty field reads as a value that failed to interpolate. Four sites (`_read_mptcp_join` and `_make_mptcp_join` for the invalid flag combination, both guards of `_make_mptcp_dss` for missing required fields), against 28 messages in the same file spelled correctly; now 32 correct against 0 doubled. Pre-existing since 2023-04-10. The new test asserts each message in full, since the existing `assertIn` on the body passes either side of the change ([#649](https://github.com/JarryShaw/PyPCAPKit/issues/649)). -- three names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. All three are now in their module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as twenty-one sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values, and reports exactly those three findings on the unfixed tree. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). -- every PCAP-NG packet block now carries the captured octets it declares. All four Enhanced Packet Blocks of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those three, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack` extracted them from the block schema, and `ProtocolBase.__init__` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the three types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). -- an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. Three payload length callbacks in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. The same lambda also governs **packing**: an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte. The one other conditional of this shape, `TSOption.remainder` in `pcapkit/protocols/schema/internet/ipv4.py`, is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). -- the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register` was a bare `cls.__enum__[code] = schema`, and is the schema half of 14 public registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's two branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The seven code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of the three takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its 327 registry writes (one being `R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690)) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). -- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). A second route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the eight unwrapped `length - N` spans and eight `__option_padding__` fields: `_TextField.__call__` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; two others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Three bare exceptions in `SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The five unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the fifth and last registrar with no overwrite guard: `Option.register` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the seven parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the three decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and eight short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). -- HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- two independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). A second defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), leaving 43 entries. `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. +- names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. Each is now in its module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as the sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values, and reports exactly those findings on the unfixed tree. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). +- every PCAP-NG packet block now carries the captured octets it declares. All four Enhanced Packet Blocks of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those three, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack` extracted them from the block schema, and `ProtocolBase.__init__` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the packet types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). +- an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. The payload length callbacks for those frames in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. The same lambda also governs **packing**: an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte. The one other conditional of this shape, `TSOption.remainder` in `pcapkit/protocols/schema/internet/ipv4.py`, is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). +- the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register` was a bare `cls.__enum__[code] = schema`, and is the schema half of the public option registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of them takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its registry writes (`R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), among them) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). +- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). A second route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the unwrapped `length - N` spans and the `__option_padding__` fields: `_TextField.__call__` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Three bare exceptions in `SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the last registrar with no overwrite guard: `Option.register` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and some short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). +- HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). A second defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690). `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. - `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at four sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for all four input classes. All four now raise `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so a fifth bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787), [#789](https://github.com/JarryShaw/PyPCAPKit/pull/789)). - a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: `b'MESSAGE=hello'` plus three NULs returned `MESSAGE == 'hello\x00\x00\x00'`. The padding-only-line guard from [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) catches padding only on a line of its own, which needs the preceding field to have ended with a real newline; without one `readline()` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 octets, so at most the trailing three octets of a terminator-less line are stripped, with a `SchemaWarning`, and anything past that is left as data. The gate reads the *raw* line, since a real trailing newline, or any other raw last octet including ASCII whitespace, proves the padding is zero ([#794](https://github.com/JarryShaw/PyPCAPKit/issues/794)). `examples/generators/pcapng.py`'s `_journal_entry` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s `not line.strip(b'\x00')` guard and the mechanism ([#791](https://github.com/JarryShaw/PyPCAPKit/issues/791)). -- `httpv2.HTTP.read` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack`/ `FieldBase.length` machinery shared by ten schema modules, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). +- `httpv2.HTTP.read` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack`/ `FieldBase.length` machinery the schema modules share, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). - `SystemdJournalExportBlock.post_process` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`; a missing or non-`b'\n'` octet ends the entry with a `SchemaWarning` instead of resynchronising at the wrong offset. `test_journal_fields_following_a_binary_field_are_not_discarded` fails against the unfixed code with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). - the same block's entries were split on a bare `b'\n\n'`, which shreds a binary field whose value contains that pair. Walking the entry instead of splitting it introduced a regression on first review: a `malformed` flag broke the *outer* per-block loop on a bad terminator, ending the whole block and dropping every well-formed entry behind it. Fixed by dropping `malformed`, leaving the per-entry `break` as the only effect. A second gap closed in the same change: a trailing separator on the block's last octet was indistinguishable from EOF and swallowed, so a rebuild wrote `length` one octet short ([#723](https://github.com/JarryShaw/PyPCAPKit/issues/723)). -- **a breaking change to** four `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). All four now peek at what `.get()` would consult and call it only once the value resolves; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and 6 of the other 7 namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: 14 of 23 captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves all 14 byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). +- **a breaking change to** four `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). All four now peek at what `.get()` would consult and call it only once the value resolves; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and most other namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: some sample captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). - `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the first two left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). -- `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and nine genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: TCP:80/8080 traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). -- **a breaking change to** 8 more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 1 of 2 (`AppType` is PR 2, [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) below), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. The first five convert 12 unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; the three with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a round-2 review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (a second, independent mint site) now build an unregistered member. `FEATCode`'s crawler now declares all 15 real FEAT-code values from the live IANA table instead of minting 10 of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are PR 2's. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s two paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). -- `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. All seven `Raises:` clauses already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Two observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). -- `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Two more guards named the wrong option: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). +- `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and the genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: `TCP:80/8080` traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). +- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. The first five convert 12 unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; the three with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a round-2 review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (a second, independent mint site) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). +- `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. Every such `Raises:` clause already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). +- `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Other guards named the wrong option too: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). - `httpv1.HTTP` refused every HTTP/1 message with no header fields, which [[RFC 9112 Section 2.1](https://datatracker.ietf.org/doc/html/rfc9112#section-2.1)] allows (`*( field-line CRLF )`): `b'GET / HTTP/1.0\r\n\r\n'` raised `ProtocolError: HTTP: invalid format`, because `_read_http_header` treated a header with no CRLF as malformed. So `HTTPv1(method='GET', uri='/', http_version='1.1')` failed to read back the bytes its own `make` built, and `HTTP._guess_version` answered `unknown HTTP version`, since `_test_start_line` mirrored the same check. A CRLF-less header is now a start line with zero fields, in the parser and the predicate alike. That check was also what refused the HTTP/2 connection preface, whose header is exactly `PRI * HTTP/2.0`; it is now refused by name instead, to the same message ([#1041](https://github.com/JarryShaw/PyPCAPKit/issues/1041)). - `httpv1.HTTP.from_data` could not rebuild an HTTP/1 message whose body is empty: `HTTP.from_data(info)` for `b'GET / HTTP/1.1\r\nHost: a\r\n\r\n'` raised `TypeError: can't concat NoneType to bytes`. `read` records an empty body as `None`, `_make_data` passed it through, and `make` concatenated it. `_make_data` now hands `make` `b''` for a `None` body; `info.body` is still `None`, so readers see no change. `httpv2` is unaffected: its empty-payload frames already round-trip ([#1050](https://github.com/JarryShaw/PyPCAPKit/issues/1050)). - `IPv6._read_ip_hextet` sliced the traffic class from `hex[0:2]` and the flow label from `hex[2:]`, both including the version nibble, so `6abcdef0` read as `(6, 0x6a, 0xbcdef0)`; it now reads `(6, 0xab, 0xcdef0)`. The parser reads these fields through the schema, so only a direct call was affected ([#1094](https://github.com/JarryShaw/PyPCAPKit/issues/1094)). - the Mobility Header Message ID option wrote a zero fraction for every timestamp, and `MesgIDOption.post_process` ignored the fraction on read. `_make_opt_mesg_id` now writes `round(frac * 2**32)`, clamped to 32 bits, and the reader adds `fraction / 2**32` seconds, so `timestamp` keeps sub-second precision ([#1093](https://github.com/JarryShaw/PyPCAPKit/issues/1093)). -- three HIP parameter defects. `HOST_ID` packed DI-Type and DI-Length as a 4-octet `BitField` where [RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401) section 5.2.9 has one 16-bit word, so every built record carried two surplus octets and read back the wrong algorithm ([#1118](https://github.com/JarryShaw/PyPCAPKit/issues/1118)); with it, `make_samples.py` builds every HIP parameter. `HIP_CIPHER` warned at six Cipher IDs, which section 5.2.8 allows; it now warns only above six and keeps the first six, as a recipient must. The maker is unchanged ([#1092](https://github.com/JarryShaw/PyPCAPKit/issues/1092)). And the HIPv2-only `DH_GROUP_LIST`, `HIP_CIPHER`, `HIT_SUITE_LIST` and `TRANSPORT_FORMAT_LIST` were built and parsed in a HIPv1 packet; at version 1 the reader and the maker now raise `ProtocolError`, as `R1_Counter` and `HIP_TRANSFORM` already do on a version mismatch ([#1133](https://github.com/JarryShaw/PyPCAPKit/issues/1133)). -- the PCAP `Header` could not rebuild itself. `_make_magic`'s explicit-endian branch ignored `nanosecond` ([#1096](https://github.com/JarryShaw/PyPCAPKit/issues/1096)), and `_make_data` returned a `magic_number` that `make()` does not take -- one of the three keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry above reports -- so `from_data` dropped it with an `UnknownFieldWarning` and rebuilt every header with the host-default `d4c3b2a1` magic. `_make_data` now returns `byteorder` and `nanosecond`, and every byte order and timestamp resolution rebuilds unchanged. `Header._read_protos` also read four octets whatever `size` asked for ([#1125](https://github.com/JarryShaw/PyPCAPKit/issues/1125)). And `_make_magic` honoured `lilendian` and `bigendian` only as a pair, so `Header(bigendian=True)` wrote `d4c3b2a1` on a little-endian host; either flag alone now picks the byte order and overrides `byteorder`, and equal flags still raise `EndianError` ([#1144](https://github.com/JarryShaw/PyPCAPKit/issues/1144)). +- HIP parameter defects. `HOST_ID` packed DI-Type and DI-Length as a 4-octet `BitField` where [RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401) section 5.2.9 has one 16-bit word, so every built record carried two surplus octets and read back the wrong algorithm ([#1118](https://github.com/JarryShaw/PyPCAPKit/issues/1118)); with it, `make_samples.py` builds every HIP parameter. `HIP_CIPHER` warned at six Cipher IDs, which section 5.2.8 allows; it now warns only above six and keeps the first six, as a recipient must. The maker is unchanged ([#1092](https://github.com/JarryShaw/PyPCAPKit/issues/1092)). And the HIPv2-only `DH_GROUP_LIST`, `HIP_CIPHER`, `HIT_SUITE_LIST` and `TRANSPORT_FORMAT_LIST` were built and parsed in a HIPv1 packet; at version 1 the reader and the maker now raise `ProtocolError`, as `R1_Counter` and `HIP_TRANSFORM` already do on a version mismatch ([#1133](https://github.com/JarryShaw/PyPCAPKit/issues/1133)). +- the PCAP `Header` could not rebuild itself. `_make_magic`'s explicit-endian branch ignored `nanosecond` ([#1096](https://github.com/JarryShaw/PyPCAPKit/issues/1096)), and `_make_data` returned a `magic_number` that `make()` does not take -- one of the keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry above reports -- so `from_data` dropped it with an `UnknownFieldWarning` and rebuilt every header with the host-default `d4c3b2a1` magic. `_make_data` now returns `byteorder` and `nanosecond`, and every byte order and timestamp resolution rebuilds unchanged. `Header._read_protos` also read four octets whatever `size` asked for ([#1125](https://github.com/JarryShaw/PyPCAPKit/issues/1125)). And `_make_magic` honoured `lilendian` and `bigendian` only as a pair, so `Header(bigendian=True)` wrote `d4c3b2a1` on a little-endian host; either flag alone now picks the byte order and overrides `byteorder`, and equal flags still raise `EndianError` ([#1144](https://github.com/JarryShaw/PyPCAPKit/issues/1144)). - TCP's NS flag was written by `make()` but never parsed: it had been commented out of `Flags`, the reader and `_make_data`, with no reason recorded. Per the owner's ruling it is exposed rather than removed, so `info.flags.ns` exists and `from_data` keeps the bit ([#1098](https://github.com/JarryShaw/PyPCAPKit/issues/1098)). -- `from_data` round trips. `Frame._make_data` returned `ts_src` and `L2TPv2._make_data` `prio`, the other two keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry reports, so `from_data` discarded both with an `UnknownFieldWarning` and an L2TPv2 priority bit was lost; they are now `ts_sec` and `priority` ([#1097](https://github.com/JarryShaw/PyPCAPKit/issues/1097)). And `ProtocolBase.from_data` took no keywords, so `Frame.from_data` and `PCAPNG.from_data`, which need `num`/`header` and `num`/`sct`/`ctx`, could not be called at all. Extra keywords now reach the construction, overriding `_make_data`'s, under the construction-keyword check, so a typo raises `UnsupportedCall` ([#1126](https://github.com/JarryShaw/PyPCAPKit/issues/1126)). `HTTP.from_data` always raised `invalid HTTP version: 0`, looking up a `version` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's `_make_data` also returned the payload `length`, which `ProtocolBase` takes as the parse length, so even a direct call failed as shorter than the 9-octet frame header; `make` now computes it, and all 231 HTTP frames in three sample captures rebuild byte for byte ([#1154](https://github.com/JarryShaw/PyPCAPKit/issues/1154)). +- `from_data` round trips. `Frame._make_data` returned `ts_src` and `L2TPv2._make_data` `prio`, the other keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry reports, so `from_data` discarded both with an `UnknownFieldWarning` and an L2TPv2 priority bit was lost; they are now `ts_sec` and `priority` ([#1097](https://github.com/JarryShaw/PyPCAPKit/issues/1097)). And `ProtocolBase.from_data` took no keywords, so `Frame.from_data` and `PCAPNG.from_data`, which need `num`/`header` and `num`/`sct`/`ctx`, could not be called at all. Extra keywords now reach the construction, overriding `_make_data`'s, under the construction-keyword check, so a typo raises `UnsupportedCall` ([#1126](https://github.com/JarryShaw/PyPCAPKit/issues/1126)). `HTTP.from_data` always raised `invalid HTTP version: 0`, looking up a `version` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's `_make_data` also returned the payload `length`, which `ProtocolBase` takes as the parse length, so even a direct call failed as shorter than the 9-octet frame header; `make` now computes it, and every HTTP frame in the sample captures rebuilds byte for byte ([#1154](https://github.com/JarryShaw/PyPCAPKit/issues/1154)). - Quick-Start options did not round-trip. The IPv4, HOPOPT and IPv6-Opts readers took the function from `schema.func`, which only a parsed schema carries, so a built option raised `AttributeError`; HOPOPT and IPv6-Opts also sized the 8-octet option as 5, misreading the nonce with a `packet length < 0` warning ([#1091](https://github.com/JarryShaw/PyPCAPKit/issues/1091)). TCP's Quick-Start Response reader read one octet past the option, and its maker produced a negative rate code below 40 kbps; such a rate now rounds down to zero, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) permits ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)). - `Ethernet`, `ARP` and `PCAPNG` packed MAC addresses as ASCII hex text. `Ethernet._make_mac_addr` returned the separator-stripped hex string and `BytesField(length=6)` truncated it, so `Ethernet()` put `303030303030` on the wire for an all-zero address ([#1123](https://github.com/JarryShaw/PyPCAPKit/issues/1123)). `ARP._make_addr_resolve`, `PCAPNG._make_mac_addr` and `_make_eui_addr` did the same; all now decode the hex to octets, and `from_data` rebuilds each byte for byte ([#1134](https://github.com/JarryShaw/PyPCAPKit/issues/1134)). - TCP's User Timeout maker wrote `length=3` where [RFC 5482](https://datatracker.ietf.org/doc/html/rfc5482) and the reader require 4, so every option it built failed to parse with `ProtocolError` ([#1120](https://github.com/JarryShaw/PyPCAPKit/issues/1120)). -- **a breaking change to** the HTTP/2 frame `length`: it counted the 9-octet frame header, where [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) section 4.1 counts the payload alone, so `make()` wrote `00000e` for a 5-octet PRIORITY frame and `read` rejected RFC-conformant frames -- and, since `_read_http_priority` checked for 9, pcapkit's own PRIORITY frame. The length and every per-type check now exclude the header. Captures written to the old convention no longer parse; nine frames packed by Scapy's `contrib.http2` now parse and rebuild byte for byte ([#1121](https://github.com/JarryShaw/PyPCAPKit/issues/1121)). +- **a breaking change to** the HTTP/2 frame `length`: it counted the 9-octet frame header, where [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) section 4.1 counts the payload alone, so `make()` wrote `00000e` for a 5-octet PRIORITY frame and `read` rejected RFC-conformant frames -- and, since `_read_http_priority` checked for 9, pcapkit's own PRIORITY frame. The length and every per-type check now exclude the header. Captures written to the old convention no longer parse; frames packed by Scapy's `contrib.http2` now parse and rebuild byte for byte ([#1121](https://github.com/JarryShaw/PyPCAPKit/issues/1121)). - **a breaking change to** `ARP.make()`: `hlen` and `plen` are derived from the packed `sha` and `spa` when omitted, and an address that disagrees with its length field raises `ProtocolError` where it used to be truncated silently -- `ARP(ptype=IPv6, spa='::1')` wrote `plen=4` and four octets of the address. A non-IP `str` protocol address packs as hex octets rather than ASCII text, as hardware addresses do since [#1134](https://github.com/JarryShaw/PyPCAPKit/issues/1134), and non-hex input raises `ProtocolError` ([#1139](https://github.com/JarryShaw/PyPCAPKit/issues/1139)). - HTTP/2 `SETTINGS` built each setting pair as a `SchemaField` with no length, so a well-formed frame warned `packet length < 0` twice per pair. The item now has its fixed 6-octet size, as TCP SACK and SCTP gap-ack items already do ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140)). - `HTTP._guess_version` took almost any binary for HTTP/2 on its fall-through: nine zero octets read as a DATA frame. The HTTP/2 trial now runs only on a header with a type [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) defines, no undefined flag, the reserved bit clear and a stream that type is sent on. Over 5,000 fuzzed zero-heavy buffers, HTTP/2 labels fell from 566 to 101, every one a legal DATA header. Explicit `version=2` parsing is unchanged ([#1141](https://github.com/JarryShaw/PyPCAPKit/issues/1141)). -- **a breaking change to** `TCP.make()`'s options: `_make_tcp_options` dropped the `NOP` and `EOOL` options it was given and padded each other option to 4 octets separately, so `from_data` rebuilt none of the 1,117 segments in `http.pcap` with their original header octets. Options are now emitted as given and padded once at the end -- zero octets after a trailing `EOOL`, otherwise `NOP` options and an `EOOL` -- so a built header can be shorter than before; every segment in six sample captures rebuilds, and `options-*.pcap` regenerate unchanged ([#1151](https://github.com/JarryShaw/PyPCAPKit/issues/1151)). +- **a breaking change to** `TCP.make()`'s options: `_make_tcp_options` dropped the `NOP` and `EOOL` options it was given and padded each other option to 4 octets separately, so `from_data` rebuilt none of the segments in `http.pcap` with their original header octets. Options are now emitted as given and padded once at the end -- zero octets after a trailing `EOOL`, otherwise `NOP` options and an `EOOL` -- so a built header can be shorter than before; every segment in the sample captures measured rebuilds, and `options-*.pcap` regenerate unchanged ([#1151](https://github.com/JarryShaw/PyPCAPKit/issues/1151)). - **a breaking change to** `from_data` on a capture truncated by snaplen: `IPv4`, `IPv6` and `UDP` rebuilt their length field from the payload at hand, writing the recorded length, so `big_endian.pcap` frame 3 came back as `45000052` where `450004a2` was on the wire. `_make_data` now passes the parsed length through to `make()`'s new `total_length` (IPv4, UDP) or `payload_length` (IPv6) keyword, which computes the length as before when omitted, so `IPv4(payload=...)` is unchanged ([#1155](https://github.com/JarryShaw/PyPCAPKit/issues/1155)). - **a breaking change to** the layer chain of a capture cut inside a header: a next layer parsed from too few captured octets was zero-filled, and `from_data` rebuilt the fabricated octets. `IPv4.read` handed UDP the declared payload length when nothing past the IPv4 header was captured, so `test.pcapng` frame 3 rebuilt with 8 invented UDP octets ([#1167](https://github.com/JarryShaw/PyPCAPKit/issues/1167)). Every `_import_next_layer` now goes through one guard: no captured octets gives `NoPayload`, and fewer than the next layer's `__length_hint__` gives `Raw` unless the parsed header fits anyway, as a 12-octet L2TPv2 ZLB does. A header-only IPv6 packet is `IPv6` rather than `IPv6:UDP:Raw`, and frame 3 is `Ethernet:Raw`, rebuilt byte for byte ([#1170](https://github.com/JarryShaw/PyPCAPKit/issues/1170)). The guard needed honest hints, so `IPv6_Route` (4), `MH` (6), `HOPOPT` and `IPv6_Opts` (2) now hint their 8-octet minimum header ([#1173](https://github.com/JarryShaw/PyPCAPKit/issues/1173)), and so does the `IPv6_Ext` fallback, which hinted 2 ([#1178](https://github.com/JarryShaw/PyPCAPKit/issues/1178)). - **a breaking change to** `L2TPv2`, against [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) section 3.1. `Type` was inverted; it is now `Data = 0` and `Control = 1`, regenerated in `pcapkit.const` from the fixed `pcapkit.vendor` table, so the same bytes parse to the other `flags.type`, and a default `make()` writes T=0 where it wrote T=1. A control message now gets `L` and a Length field automatically, and the offset pad, which `from_data` zero-filled, is kept as `info.padding` -- a new key in `info` and in dumped output ([#1175](https://github.com/JarryShaw/PyPCAPKit/issues/1175)). `from_data` also cleared `L` and dropped the Length field: `_make_data` returned it as `length`, which `ProtocolBase` takes as the parse length. `make()`'s `length` keyword is now `total_length`, written as given; a call still passing `length` loses `L` silently ([#1172](https://github.com/JarryShaw/PyPCAPKit/issues/1172)). `make()` also gains `length_flag` and `padding`, and raises `ProtocolError` for a `total_length` with `length_flag=False`. -- **a breaking change to** `IPv6.from_data`: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden `__exthdr__` key, absent from `to_dict` and dumped output, and re-emitted ahead of the upper layer. Of 1,602 sample frames, 130 that rebuilt wrong now rebuild exactly, and 50 that rebuilt with wrong bytes raised `AttributeError` instead ([#1177](https://github.com/JarryShaw/PyPCAPKit/issues/1177)). Those were two older defects, now fixed: `HIP.alias` read `_info` before `read` had set it, which also broke a standalone HIP parse and `HIP.from_data` ([#1184](https://github.com/JarryShaw/PyPCAPKit/issues/1184)), and the MPL option's maker called `bit_length()` on the `IPv6Address` seed of an S=0 option. MPL in `HOPOPT` and `IPv6_Opts` now takes `S` from the parsed option or a new `seed_type` keyword, and a missing or oversized seed raises `ProtocolError` ([#1185](https://github.com/JarryShaw/PyPCAPKit/issues/1185)). With both fixed, all 179 IPv6 frames of the five `options-*.pcap` captures rebuild byte for byte. +- **a breaking change to** `IPv6.from_data`: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden `__exthdr__` key, absent from `to_dict` and dumped output, and re-emitted ahead of the upper layer. Sample frames that rebuilt wrong now rebuild exactly, and those that rebuilt with wrong bytes raised `AttributeError` instead ([#1177](https://github.com/JarryShaw/PyPCAPKit/issues/1177)). Those were older defects, now fixed: `HIP.alias` read `_info` before `read` had set it, which also broke a standalone HIP parse and `HIP.from_data` ([#1184](https://github.com/JarryShaw/PyPCAPKit/issues/1184)), and the MPL option's maker called `bit_length()` on the `IPv6Address` seed of an S=0 option. MPL in `HOPOPT` and `IPv6_Opts` now takes `S` from the parsed option or a new `seed_type` keyword, and a missing or oversized seed raises `ProtocolError` ([#1185](https://github.com/JarryShaw/PyPCAPKit/issues/1185)). With both fixed, every IPv6 frame of the `options-*.pcap` captures rebuilds byte for byte. - **a breaking change to** the SMF_DPD option's maker in `HOPOPT` and `IPv6_Opts`: it overwrote the parsed DPD mode, so H-DPD rebuilt as I-DPD, and inferred the TaggerID type from the Tagger ID's length, so a 4-octet `DEFAULT` Tagger ID rebuilt as IPv4. Both now come from the parsed option, or from `mode` and a new `tid_type` keyword. A Tagger ID that does not fit its type raises `ProtocolError`, so a 17-octet one does where it raised `FieldValueError`, and `tid=b''` builds a NULL Tagger ID where it raised `ValueError` ([#1190](https://github.com/JarryShaw/PyPCAPKit/issues/1190)). An H-DPD option with an empty Hash Assist Value raises `ProtocolError` where it raised a bare `IndexError` ([#1193](https://github.com/JarryShaw/PyPCAPKit/issues/1193)). - a `ForwardMatchField` that read short at the end of the stream rewound by its declared width, not by the octets it read, so the fields after it re-read the preceding octet. An SMF_DPD option in the last two octets of a `HOPOPT` or `IPv6_Opts` header was re-read from the padding and raised a bare `TypeError`; it now raises `FieldValueError`, as CALIPSO and MPL do. A full read rewinds as before ([#1195](https://github.com/JarryShaw/PyPCAPKit/issues/1195)). - `HIP()` with no parameters wrote Header Length 0, below [RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401) section 5.1's minimum of 4 for the two HITs, so it failed to read back with `ProtocolError`; it now writes 4. `read` names a Header Length below 4 in a new `invalid header length` message. Lengths 0 to 3 already raised `ProtocolError`, so only the message changes ([#1189](https://github.com/JarryShaw/PyPCAPKit/issues/1189)). @@ -292,8 +295,8 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - IPv4 option areas, route slots, timestamps and reserved bits ([#1344](https://github.com/JarryShaw/PyPCAPKit/pull/1344)); the TS option's slots beyond the pointer ([#1349](https://github.com/JarryShaw/PyPCAPKit/issues/1349), [#1390](https://github.com/JarryShaw/PyPCAPKit/pull/1390)); TS and SEC options byte for byte ([#1399](https://github.com/JarryShaw/PyPCAPKit/pull/1399)). - IPv6 core headers ([#1307](https://github.com/JarryShaw/PyPCAPKit/pull/1307)); `HOPOPT` and `IPv6_Opts` option areas ([#1310](https://github.com/JarryShaw/PyPCAPKit/pull/1310)); a CALIPSO option's trailing pad ([#1403](https://github.com/JarryShaw/PyPCAPKit/issues/1403), [#1407](https://github.com/JarryShaw/PyPCAPKit/pull/1407)), and its `make()` writing a corrupt length on overflow ([#1408](https://github.com/JarryShaw/PyPCAPKit/issues/1408), [#1409](https://github.com/JarryShaw/PyPCAPKit/pull/1409)); an extension header parsed with `extension=True` ([#1446](https://github.com/JarryShaw/PyPCAPKit/issues/1446), [#1449](https://github.com/JarryShaw/PyPCAPKit/pull/1449)); RPL source routes ([#1463](https://github.com/JarryShaw/PyPCAPKit/pull/1463)); and a cut extension-header chain ([#1476](https://github.com/JarryShaw/PyPCAPKit/pull/1476)). - octets past IPv4 Total Length and IPv6 Payload Length ([#1209](https://github.com/JarryShaw/PyPCAPKit/issues/1209), [#1431](https://github.com/JarryShaw/PyPCAPKit/pull/1431)), and past the IPX Length field ([#1433](https://github.com/JarryShaw/PyPCAPKit/issues/1433), [#1435](https://github.com/JarryShaw/PyPCAPKit/pull/1435)). IPX `from_data` raised `struct.error`, and AH wrote its reserved field back as zero ([#1338](https://github.com/JarryShaw/PyPCAPKit/pull/1338)). -- HIP: Header Length, an empty parameter list, the PUZZLE lifetime and reserved bits ([#1342](https://github.com/JarryShaw/PyPCAPKit/pull/1342)); Locator flags ([#1365](https://github.com/JarryShaw/PyPCAPKit/issues/1365), [#1367](https://github.com/JarryShaw/PyPCAPKit/pull/1367)); three `BitField` namespaces' reserved bits ([#1417](https://github.com/JarryShaw/PyPCAPKit/issues/1417), [#1419](https://github.com/JarryShaw/PyPCAPKit/pull/1419)); and the two fixed header bits ([#1459](https://github.com/JarryShaw/PyPCAPKit/issues/1459), [#1462](https://github.com/JarryShaw/PyPCAPKit/pull/1462)). -- MH: zero-length PadN, reserved fields and BU/BA flag bits ([#1324](https://github.com/JarryShaw/PyPCAPKit/issues/1324), [#1345](https://github.com/JarryShaw/PyPCAPKit/pull/1345)); the reserved bits of 24 `BitField` flag namespaces ([#1418](https://github.com/JarryShaw/PyPCAPKit/issues/1418), [#1421](https://github.com/JarryShaw/PyPCAPKit/pull/1421)); and the reserved octets `PaddingField` sites capture ([#1442](https://github.com/JarryShaw/PyPCAPKit/pull/1442)). +- HIP: Header Length, an empty parameter list, the PUZZLE lifetime and reserved bits ([#1342](https://github.com/JarryShaw/PyPCAPKit/pull/1342)); Locator flags ([#1365](https://github.com/JarryShaw/PyPCAPKit/issues/1365), [#1367](https://github.com/JarryShaw/PyPCAPKit/pull/1367)); `BitField` namespaces' reserved bits ([#1417](https://github.com/JarryShaw/PyPCAPKit/issues/1417), [#1419](https://github.com/JarryShaw/PyPCAPKit/pull/1419)); and the two fixed header bits ([#1459](https://github.com/JarryShaw/PyPCAPKit/issues/1459), [#1462](https://github.com/JarryShaw/PyPCAPKit/pull/1462)). +- MH: zero-length PadN, reserved fields and BU/BA flag bits ([#1324](https://github.com/JarryShaw/PyPCAPKit/issues/1324), [#1345](https://github.com/JarryShaw/PyPCAPKit/pull/1345)); the reserved bits of the `BitField` flag namespaces ([#1418](https://github.com/JarryShaw/PyPCAPKit/issues/1418), [#1421](https://github.com/JarryShaw/PyPCAPKit/pull/1421)); and the reserved octets `PaddingField` sites capture ([#1442](https://github.com/JarryShaw/PyPCAPKit/pull/1442)). - PCAP-NG: constructed and rebuilt blocks ([#1339](https://github.com/JarryShaw/PyPCAPKit/pull/1339)), lengths and padding ([#1373](https://github.com/JarryShaw/PyPCAPKit/pull/1373)), blocks and options as parsed ([#1378](https://github.com/JarryShaw/PyPCAPKit/pull/1378)), further follow-ups ([#1412](https://github.com/JarryShaw/PyPCAPKit/pull/1412)), malformed journal entries and short packet blocks ([#1420](https://github.com/JarryShaw/PyPCAPKit/pull/1420)), an SHB or IDB shorter than its fixed fields ([#1422](https://github.com/JarryShaw/PyPCAPKit/issues/1422), [#1425](https://github.com/JarryShaw/PyPCAPKit/pull/1425)), and captured padding octets ([#1441](https://github.com/JarryShaw/PyPCAPKit/pull/1441)). - a cut PCAP header or PCAP-NG block ([#1470](https://github.com/JarryShaw/PyPCAPKit/issues/1470), [#1475](https://github.com/JarryShaw/PyPCAPKit/pull/1475)). @@ -305,30 +308,31 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - IPv4: a Total Length of 0 is read as TSO, and one inside the header as bogus ([#1547](https://github.com/JarryShaw/PyPCAPKit/issues/1547), [#1554](https://github.com/JarryShaw/PyPCAPKit/pull/1554)). - IPv6: a jumbogram's payload is dissected up to its Jumbo Payload Length ([#1434](https://github.com/JarryShaw/PyPCAPKit/issues/1434), [#1440](https://github.com/JarryShaw/PyPCAPKit/pull/1440)), a jumbogram [RFC 2675](https://datatracker.ietf.org/doc/html/rfc2675) calls an error is rejected ([#1687](https://github.com/JarryShaw/PyPCAPKit/issues/1687), [#1713](https://github.com/JarryShaw/PyPCAPKit/pull/1713)), and so is a header whose Version is not 6 ([#1680](https://github.com/JarryShaw/PyPCAPKit/issues/1680), [#1723](https://github.com/JarryShaw/PyPCAPKit/pull/1723)). - the warnings of a next layer replaced by `Raw` are dropped ([#1580](https://github.com/JarryShaw/PyPCAPKit/issues/1580), [#1588](https://github.com/JarryShaw/PyPCAPKit/pull/1588)). -- IPv6-Route's three data-type readers name the routing type in their `ProtocolError`, where each message read `[TypeNo ]` ([#442](https://github.com/JarryShaw/PyPCAPKit/issues/442)), and `HTTP(..., version=1)` parses from the frame's data rather than the drained file, where it raised `ValueError` ([#447](https://github.com/JarryShaw/PyPCAPKit/issues/447), [#451](https://github.com/JarryShaw/PyPCAPKit/pull/451)). +- IPv6-Route's data-type readers name the routing type in their `ProtocolError`, where each message read `[TypeNo ]` ([#442](https://github.com/JarryShaw/PyPCAPKit/issues/442)), and `HTTP(..., version=1)` parses from the frame's data rather than the drained file, where it raised `ValueError` ([#447](https://github.com/JarryShaw/PyPCAPKit/issues/447), [#451](https://github.com/JarryShaw/PyPCAPKit/pull/451)). - `Schema.pack` accepts a `tuple` for a `ListField`, as the data models declare, so HIP and MH options that hold one round-trip ([#476](https://github.com/JarryShaw/PyPCAPKit/issues/476), [#480](https://github.com/JarryShaw/PyPCAPKit/pull/480)). - a Mobility Header CGA extension reports its length with its 4-octet header, where every parsed one read 2 octets short ([#512](https://github.com/JarryShaw/PyPCAPKit/issues/512), [#517](https://github.com/JarryShaw/PyPCAPKit/issues/517), [#528](https://github.com/JarryShaw/PyPCAPKit/pull/528)). - TCP's SACK option enforces the `8n + 2` length [RFC 2018](https://datatracker.ietf.org/doc/html/rfc2018) gives it, as its docstring promised ([#519](https://github.com/JarryShaw/PyPCAPKit/issues/519), [#535](https://github.com/JarryShaw/PyPCAPKit/pull/535)). - `IPv4.from_data()` rebuilds a parsed datagram: `Schema.pack` checks a payload against `ProtocolBase`, and option padding ends in an `EOOL` option rather than a bare wire code ([#506](https://github.com/JarryShaw/PyPCAPKit/issues/506), [#536](https://github.com/JarryShaw/PyPCAPKit/pull/536)). - IPv4's SEC option sets the termination bit its own reader requires, sizes the bitmap from the bit count, and SID is 16 bits wide ([#534](https://github.com/JarryShaw/PyPCAPKit/issues/534), [#537](https://github.com/JarryShaw/PyPCAPKit/issues/537), [#543](https://github.com/JarryShaw/PyPCAPKit/pull/543)). - the MPTCP option header reads `length` from bit 8, per [RFC 8684 Section 3](https://datatracker.ietf.org/doc/html/rfc8684#section-3), rather than bit 1 ([#553](https://github.com/JarryShaw/PyPCAPKit/issues/553), [#558](https://github.com/JarryShaw/PyPCAPKit/pull/558)). -- four Mobility Header enumerations defined inline from their RFCs (`FastBindingAcknowledgmentStatus`, `IPv6AddressPrefixCode`, `LocalizedRoutingStatus`, `LMAAddressCode`) raise `EnumValueError` for an unassigned value rather than minting `Unassigned_N` ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#889](https://github.com/JarryShaw/PyPCAPKit/pull/889)), and all four inherit `get` and its `default` from `EnumLookup` rather than overriding it ([#935](https://github.com/JarryShaw/PyPCAPKit/issues/935), [#940](https://github.com/JarryShaw/PyPCAPKit/pull/940)). +- Mobility Header enumerations defined inline from their RFCs (`FastBindingAcknowledgmentStatus`, `IPv6AddressPrefixCode`, `LocalizedRoutingStatus`, `LMAAddressCode`) raise `EnumValueError` for an unassigned value rather than minting `Unassigned_N` ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#889](https://github.com/JarryShaw/PyPCAPKit/pull/889)), and each inherits `get` and its `default` from `EnumLookup` rather than overriding it ([#935](https://github.com/JarryShaw/PyPCAPKit/issues/935), [#940](https://github.com/JarryShaw/PyPCAPKit/pull/940)). - `pcapkit.protocols.protocol.__all__` lists `Protocol` beside `ProtocolBase`, so `from pcapkit.protocols.protocol import *` binds the module's public class ([#1770](https://github.com/JarryShaw/PyPCAPKit/issues/1770), [#1778](https://github.com/JarryShaw/PyPCAPKit/pull/1778)). +- the data and schema packages re-export what their submodules list in `__all__`, under the prefixes their siblings use: `pcapkit.protocols.data.link` exports the L2TP data model as `L2TP` and `L2TP_Flags` ([#1797](https://github.com/JarryShaw/PyPCAPKit/issues/1797), [#1806](https://github.com/JarryShaw/PyPCAPKit/pull/1806)), and the Mobility Header models, the HOPOPT and IPv6-Opts `SMFDPDOption` and `QuickStartOption`, PCAP-NG's `PacketBlock` and `PACK_*` options and HTTP/2's `Settings` reach their packages and the aggregators ([#1807](https://github.com/JarryShaw/PyPCAPKit/issues/1807), [#1815](https://github.com/JarryShaw/PyPCAPKit/pull/1815)). ### pcapkit.toolkit #### Added -- `util/pyshark_encap_map.py`, a generator that regenerates `ENCAP_TYPE_TO_LINKTYPE` (152 entries) and `FILTER_NAME_TO_LINKTYPE` (58) in place inside `pcapkit/toolkit/pyshark.py`, so the two tables [#850](https://github.com/JarryShaw/PyPCAPKit/pull/850) hand-built stop being hand-maintained. It lives under `util/` rather than `pcapkit/vendor/` because `Vendor`'s base class is hardwired to an HTTP fetch and a generated enum-class file, where this generator's source of truth is the local `tshark`/`editcap` binaries. It guards against its worst failure mode: a naive `LinkType(dlt)` cannot detect an unmapped DLT, because `_missing_` mints a placeholder rather than raising, so the generator snapshots every known `LinkType` value first and calls `LinkType(dlt)` only once a value is confirmed a member. The real-`tshark` sweep is gated on a new `HAS_WIRESHARK` flag, registered in `tests/_dependency_gates.py`'s `NON_DISTRIBUTION_FLAGS` next to `HAS_PROC_FD`, for things pip cannot install. The sweep arithmetic is version-pinned (`editcap -T` accepts 226 encapsulations on Wireshark 4.6.9, 224 on 4.2.2, CI's Ubuntu noble), so count assertions run only under exactly the measured version, with a separate `VersionIndependentInvariantTests` class for what holds regardless. One line of `pcapkit/toolkit/pyshark.py` changes: the `IPMB_LINUX` entry becomes `I2C_LINUX`, the canonical name [#848](https://github.com/JarryShaw/PyPCAPKit/pull/848) gave value 209 -- the same member either way ([#853](https://github.com/JarryShaw/PyPCAPKit/pull/853)). +- `util/pyshark_encap_map.py`, a generator that regenerates `ENCAP_TYPE_TO_LINKTYPE` and `FILTER_NAME_TO_LINKTYPE` in place inside `pcapkit/toolkit/pyshark.py`, so the two tables [#850](https://github.com/JarryShaw/PyPCAPKit/pull/850) hand-built stop being hand-maintained. It lives under `util/` rather than `pcapkit/vendor/` because `Vendor`'s base class is hardwired to an HTTP fetch and a generated enum-class file, where this generator's source of truth is the local `tshark`/`editcap` binaries. It guards against its worst failure mode: a naive `LinkType(dlt)` cannot detect an unmapped DLT, because `_missing_` mints a placeholder rather than raising, so the generator snapshots every known `LinkType` value first and calls `LinkType(dlt)` only once a value is confirmed a member. The real-`tshark` sweep is gated on a new `HAS_WIRESHARK` flag, registered in `tests/_dependency_gates.py`'s `NON_DISTRIBUTION_FLAGS` next to `HAS_PROC_FD`, for things pip cannot install. The sweep arithmetic is version-pinned (`editcap -T` accepts 226 encapsulations on Wireshark 4.6.9, 224 on 4.2.2, CI's Ubuntu noble), so count assertions run only under exactly the measured version, with a separate `VersionIndependentInvariantTests` class for what holds regardless. One line of `pcapkit/toolkit/pyshark.py` changes: the `IPMB_LINUX` entry becomes `I2C_LINUX`, the canonical name [#848](https://github.com/JarryShaw/PyPCAPKit/pull/848) gave value 209 -- the same member either way ([#853](https://github.com/JarryShaw/PyPCAPKit/pull/853)). - `tcp_segment` and its result type are public in `pcapkit.toolkit.pcap` ([#1570](https://github.com/JarryShaw/PyPCAPKit/issues/1570), [#1579](https://github.com/JarryShaw/PyPCAPKit/pull/1579)). #### Fixed -- the engine adapters, which were quietly wrong rather than loud. The `dpkt` toolkit split TCP and IPv4 headers at their fixed struct size instead of their real length, so option octets overwrote payload in the sequence-indexed reassembly buffer; it also read an `ipv6_frag.nh` that `dpkt` does not have, and passed fragment offsets unscaled ([#351](https://github.com/JarryShaw/PyPCAPKit/issues/351), [#370](https://github.com/JarryShaw/PyPCAPKit/issues/370), [#385](https://github.com/JarryShaw/PyPCAPKit/pull/385), [#395](https://github.com/JarryShaw/PyPCAPKit/pull/395)). `scapy` never loaded its layer registry, so that engine did not dissect at all ([#409](https://github.com/JarryShaw/PyPCAPKit/pull/409)), and its IPv4 fragment offset reached reassembly unscaled ([#483](https://github.com/JarryShaw/PyPCAPKit/issues/483), [#484](https://github.com/JarryShaw/PyPCAPKit/pull/484)). The four IPv6 adapters disagreed about whether the 8-octet Fragment header belongs to `ihl`, `header` and `tl`; per [RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200) section 4.5 it belongs to none, and all four now agree ([#415](https://github.com/JarryShaw/PyPCAPKit/issues/415), [#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). -- `pypcapfile`'s `IP.src`/`.dst` are dotted-decimal ASCII text in a `ctypes.c_char_p` (per `pypcapfile` 0.12.0's `ip.py`), not packed 4-byte values, so `ipaddress.IPv4Address(ipv4.src)` raised `AddressValueError` and `ipv4_header()`'s `struct.pack('...II', ipv4.src, ...)` raised `struct.error` on every call. A single `_parse_ipv4_address()` chokepoint replaces the four call sites, also accepting packed `bytes` or an `int`. The same engine's `IP.opt`/`.payload` carry hex-ASCII rather than raw bytes once decoding stops short of the transport layer; a new `_maybe_unhex()` covers the three sites that read them. Not `breaking`: nothing on this path worked on 3.10/3.11, so correct behaviour cannot regress a working caller ([#743](https://github.com/JarryShaw/PyPCAPKit/issues/743)). +- the engine adapters, which were quietly wrong rather than loud. The `dpkt` toolkit split TCP and IPv4 headers at their fixed struct size instead of their real length, so option octets overwrote payload in the sequence-indexed reassembly buffer; it also read an `ipv6_frag.nh` that `dpkt` does not have, and passed fragment offsets unscaled ([#351](https://github.com/JarryShaw/PyPCAPKit/issues/351), [#370](https://github.com/JarryShaw/PyPCAPKit/issues/370), [#385](https://github.com/JarryShaw/PyPCAPKit/pull/385), [#395](https://github.com/JarryShaw/PyPCAPKit/pull/395)). `scapy` never loaded its layer registry, so that engine did not dissect at all ([#409](https://github.com/JarryShaw/PyPCAPKit/pull/409)), and its IPv4 fragment offset reached reassembly unscaled ([#483](https://github.com/JarryShaw/PyPCAPKit/issues/483), [#484](https://github.com/JarryShaw/PyPCAPKit/pull/484)). The IPv6 adapters disagreed about whether the 8-octet Fragment header belongs to `ihl`, `header` and `tl`; per [RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200) section 4.5 it belongs to none, and they now agree ([#415](https://github.com/JarryShaw/PyPCAPKit/issues/415), [#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). +- `pypcapfile`'s `IP.src`/`.dst` are dotted-decimal ASCII text in a `ctypes.c_char_p` (per `pypcapfile` 0.12.0's `ip.py`), not packed 4-byte values, so `ipaddress.IPv4Address(ipv4.src)` raised `AddressValueError` and `ipv4_header()`'s `struct.pack('...II', ipv4.src, ...)` raised `struct.error` on every call. A single `_parse_ipv4_address()` chokepoint replaces the separate call sites, also accepting packed `bytes` or an `int`. The same engine's `IP.opt`/`.payload` carry hex-ASCII rather than raw bytes once decoding stops short of the transport layer; a new `_maybe_unhex()` covers the sites that read them. Not `breaking`: nothing on this path worked on 3.10/3.11, so correct behaviour cannot regress a working caller ([#743](https://github.com/JarryShaw/PyPCAPKit/issues/743)). - `pypcapfile`'s `_read_a_packet(layers=0)` hexlifies a whole frame into ASCII text and stops, but `_decode()` fed that into `Ethernet(packet.packet, layers=1)`, whose `__init__` unpacks the header with `struct.unpack`, so every field came out garbage without raising. Fixed with `binascii.unhexlify(packet.packet)` ahead of the decoder call, in preference to decoding eagerly at `layers>0` inside the lazy generator, which would lose `_decode()`'s per-frame `AttributeWarning` fallback. Link-layer dispatch resolves from the header's `ll_type` and is unaffected. Landed with [#743](https://github.com/JarryShaw/PyPCAPKit/issues/743): standalone, this turns a non-crashing extraction into a crashing one, since `Extractor.run()` catches only `EOFError`, `StopIteration` and `KeyboardInterrupt` and the `AddressValueError`/`struct.error` [#743](https://github.com/JarryShaw/PyPCAPKit/issues/743) fixes escape until both land ([#746](https://github.com/JarryShaw/PyPCAPKit/issues/746)). -- **a breaking change to** pyshark link-type resolution in `tcp_traceflow`: an unrecognised encapsulation or filter name now raises `MissingKeyError` instead of substituting a plausible DLT. `pcapkit/toolkit/pyshark.py` resolved a frame's link type from `packet.layers[0].layer_name`, a Wireshark display-filter name, with a blanket `Enum_LinkType.get(name.upper())` fallback. One filter name serves several encapsulations, so that fallback returned *wrong DLTs silently* (`null` for a `DLT_LOOP` capture, `101` (`RAW`) for `rawip6`). Re-keyed onto `frame.encap_type`, which is distinct per encapsulation, and the fallback is gone. The two tables were measured rather than transcribed (Wireshark's source is not on the build host): each entry round-tripped through `editcap -F pcap -T ` and read back with `tshark`. `ENCAP_TYPE_TO_LINKTYPE` grows to 152 entries with no DLT serving two keys, and `FILTER_NAME_TO_LINKTYPE` from [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s two entries to 58, every root filter name single-DLT across the 157 (of 226) encapsulations this host's Wireshark can write. `pcapkit/toolkit/pyshark.py` reaches 100% coverage under new tests that fail 26 ways against the unfixed tree; live pyshark parsing stays unexercised, since `pyshark.FileCapture` does not run on Python 3.14 ([#850](https://github.com/JarryShaw/PyPCAPKit/pull/850)). -- `pypcapfile`'s `tcp_reassembly` set `last` one past the segment. `TCP_Packet.last` is inclusive, and the four other adapters set `seqnum + len(payload) - 1`, as this one now does too ([#1100](https://github.com/JarryShaw/PyPCAPKit/issues/1100)). +- **a breaking change to** pyshark link-type resolution in `tcp_traceflow`: an unrecognised encapsulation or filter name now raises `MissingKeyError` instead of substituting a plausible DLT. `pcapkit/toolkit/pyshark.py` resolved a frame's link type from `packet.layers[0].layer_name`, a Wireshark display-filter name, with a blanket `Enum_LinkType.get(name.upper())` fallback. One filter name serves several encapsulations, so that fallback returned *wrong DLTs silently* (`null` for a `DLT_LOOP` capture, `101` (`RAW`) for `rawip6`). Re-keyed onto `frame.encap_type`, which is distinct per encapsulation, and the fallback is gone. The two tables were measured rather than transcribed (Wireshark's source is not on the build host): each entry round-tripped through `editcap -F pcap -T ` and read back with `tshark`. `ENCAP_TYPE_TO_LINKTYPE` grows to cover every measured encapsulation with no DLT serving two keys, and `FILTER_NAME_TO_LINKTYPE` grows from [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s hand-built entries, every root filter name single-DLT across the encapsulations this host's Wireshark can write. `pcapkit/toolkit/pyshark.py` reaches full coverage under new tests that fail against the unfixed tree; live pyshark parsing stays unexercised, since `pyshark.FileCapture` does not run on Python 3.14 ([#850](https://github.com/JarryShaw/PyPCAPKit/pull/850)). +- `pypcapfile`'s `tcp_reassembly` set `last` one past the segment. `TCP_Packet.last` is inclusive, and the other adapters set `seqnum + len(payload) - 1`, as this one now does too ([#1100](https://github.com/JarryShaw/PyPCAPKit/issues/1100)). - scapy's `packet2dict` dumped nested packets as `{}` and mutated its input ([#1287](https://github.com/JarryShaw/PyPCAPKit/pull/1287)); the dpkt reassembly adapters build their data from the record's own octets rather than dpkt's re-serialised bytes, and leave the packet alone ([#1297](https://github.com/JarryShaw/PyPCAPKit/issues/1297), [#1335](https://github.com/JarryShaw/PyPCAPKit/pull/1335)). - the TCP adapters: pyshark accepts tshark's `True`/`False` TCP flags ([#1514](https://github.com/JarryShaw/PyPCAPKit/issues/1514), [#1527](https://github.com/JarryShaw/PyPCAPKit/pull/1527)); a TCP segment whose header the snapshot cut is traced and reassembled ([#1518](https://github.com/JarryShaw/PyPCAPKit/issues/1518), [#1556](https://github.com/JarryShaw/PyPCAPKit/pull/1556)); dpkt reads SYN, FIN and RST whatever the NS bit ([#1618](https://github.com/JarryShaw/PyPCAPKit/issues/1618), [#1621](https://github.com/JarryShaw/PyPCAPKit/pull/1621)); and the scapy adapters stop at the IP datagram's declared length ([#1584](https://github.com/JarryShaw/PyPCAPKit/issues/1584), [#1589](https://github.com/JarryShaw/PyPCAPKit/pull/1589)). - timestamps: dpkt and pypcapfile hand over the correctly rounded timestamp ([#1668](https://github.com/JarryShaw/PyPCAPKit/issues/1668), [#1690](https://github.com/JarryShaw/PyPCAPKit/pull/1690)), and a copied `DecimalTimestamp` keeps `orig_len` under `_pydecimal` ([#1647](https://github.com/JarryShaw/PyPCAPKit/issues/1647), [#1653](https://github.com/JarryShaw/PyPCAPKit/pull/1653)). @@ -338,17 +342,17 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r #### Added -- `pcapkit.utilities.logging` as a real interface: `get_logger()` for per-module children, `configure()` to set level, handler, stream, format or propagation at runtime, `reset()` to return to library-neutral, and `ensure_output()`. Seventeen modules now log under their own `__name__`, so a consumer can silence `pcapkit.foundation.registry` while keeping `pcapkit.foundation.extraction` ([#384](https://github.com/JarryShaw/PyPCAPKit/pull/384)). -- `pcapkit.utilities.config`, one module for every tunable limit, each read once from a `PCAPKIT_*` environment variable: the frame-layer and extension-header limits, the pypcapfile engine's TCP-over-IPv6 warning limit, the dumpkit fallback, the charset-detection cache, the read chunk and the three zero-padding guards. A bad value warns with `ConfigWarning` and falls back to the default; `-1` lifts a limit. The homes these limits had in the prereleases keep no re-export ([#1726](https://github.com/JarryShaw/PyPCAPKit/issues/1726), [#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). +- `pcapkit.utilities.logging` as a real interface: `get_logger()` for per-module children, `configure()` to set level, handler, stream, format or propagation at runtime, `reset()` to return to library-neutral, and `ensure_output()`. Modules now log under their own `__name__`, so a consumer can silence `pcapkit.foundation.registry` while keeping `pcapkit.foundation.extraction` ([#384](https://github.com/JarryShaw/PyPCAPKit/pull/384)). +- `pcapkit.utilities.config`, one module for every tunable limit, each read once from a `PCAPKIT_*` environment variable: the frame-layer and extension-header limits, the pypcapfile engine's TCP-over-IPv6 warning limit, the dumpkit fallback, the charset-detection cache, the read chunk and the zero-padding guards. A bad value warns with `ConfigWarning` and falls back to the default; `-1` lifts a limit. The homes these limits had in the prereleases keep no re-export ([#1726](https://github.com/JarryShaw/PyPCAPKit/issues/1726), [#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). #### Changed -- `pcapkit` no longer configures logging at import. It installs a `NullHandler` and sets no level, so verbosity is inherited from the application instead of being seized by whichever library was imported second; the old stderr handler stays as the `PCAPKIT_DEVMODE` opt-in. Three consequences: the previous behaviour is `configure(logging.INFO, stream=sys.stderr)`; 38 registry and extractor `info` calls became `debug`, so those messages are invisible even at `INFO`; and the handler is no longer `logger.handlers[0]`. `verbose=` output stays on stdout and is not logging ([#384](https://github.com/JarryShaw/PyPCAPKit/pull/384)). +- `pcapkit` no longer configures logging at import. It installs a `NullHandler` and sets no level, so verbosity is inherited from the application instead of being seized by whichever library was imported second; the old stderr handler stays as the `PCAPKIT_DEVMODE` opt-in. Consequences: the previous behaviour is `configure(logging.INFO, stream=sys.stderr)`; the registry and extractor `info` calls became `debug`, so those messages are invisible even at `INFO`; and the handler is no longer `logger.handlers[0]`. `verbose=` output stays on stdout and is not logging ([#384](https://github.com/JarryShaw/PyPCAPKit/pull/384)). - each warning is reported once per channel, and `pcapkit` no longer inserts a `simplefilter('ignore', ...)` at the front of the process-global `warnings.filters` ([#362](https://github.com/JarryShaw/PyPCAPKit/issues/362)--[#364](https://github.com/JarryShaw/PyPCAPKit/issues/364), [#390](https://github.com/JarryShaw/PyPCAPKit/pull/390)). The application's own filter therefore wins, which is the point and also the sharp edge: under `-W error`, or pytest's `filterwarnings = error`, a pcapkit warning that used to be suppressed will raise. Suppress them deliberately with `warnings.filterwarnings('ignore', category=BaseWarning)`. `quiet=True` now means no record at any level and no longer sets `sys.tracebacklimit`, and the `pcapkit.utilities.warnings.DEVMODE` re-export is gone; its home is `pcapkit.utilities.logging`. #### Fixed -- two more gaps the [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) keyword audit turned up, neither covered by a test: `StreamEOFError`'s docstring did not say that `@prepare` always raises it with `quiet=True` (the end-of-stream convention `StructError` follows via its own `eof=True`), so nothing pinned that silence against regression; and `register_extractor_engine`'s real keyword, `name`, was not itself under test, only its corrected docstring, so a rename could put the two out of step again ([#577](https://github.com/JarryShaw/PyPCAPKit/pull/577)). +- further gaps the [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) keyword audit turned up, none covered by a test: `StreamEOFError`'s docstring did not say that `@prepare` always raises it with `quiet=True` (the end-of-stream convention `StructError` follows via its own `eof=True`), so nothing pinned that silence against regression; and `register_extractor_engine`'s real keyword, `name`, was not itself under test, only its corrected docstring, so a rename could put the two out of step again ([#577](https://github.com/JarryShaw/PyPCAPKit/pull/577)). - `hold_warnings` is in `pcapkit.utilities.warnings.__all__` ([#1602](https://github.com/JarryShaw/PyPCAPKit/issues/1602), [#1603](https://github.com/JarryShaw/PyPCAPKit/pull/1603)). - the default threading excepthook is the fallback before Python 3.10, which has no `threading.__excepthook__` ([#1691](https://github.com/JarryShaw/PyPCAPKit/issues/1691), [#1701](https://github.com/JarryShaw/PyPCAPKit/pull/1701)). - `stacklevel()` returns a level relative to its caller, as `warnings.warn` and `logging` count it, so a warning or a `BaseError` log record names the caller's frame at any stack depth rather than a frame inside `logging` or pcapkit ([#397](https://github.com/JarryShaw/PyPCAPKit/pull/397)). @@ -359,11 +363,11 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r #### Fixed -- thirteen `re.sub` sites under `pcapkit/vendor/` passed `re.MULTILINE` as the fourth *positional* argument, which is `count`, not `flags`: substitution capped at 8 matches, and a `DeprecationWarning` on Python 3.13+ (a future `TypeError`). Found by an AST sweep rather than grep, since 8 of the 13 put the flag on a continuation line. The flag moves to `flags=` at all 13 sites (`default.py` plus one each in `hip/eddsa_curve.py`, `http/frame.py`, `http/method.py`, `http/status_code.py`, `ipv4/router_alert.py`, `ipv6/option.py`, `ipv6/router_alert.py`, `ipv6/tagger_id.py`, `reg/ethertype.py`, `tcp/flags.py`, `tcp/mp_tcp_option.py` and `tcp/option.py`). Behaviour preserving: the shared pattern (`r'\r*\n'`) has no MULTILINE-sensitive anchor, so no `const/` regeneration was needed ([#796](https://github.com/JarryShaw/PyPCAPKit/issues/796)). -- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)): 12 tests across three files, all `RecursionError` (`tests/const/test_const_enum_lookup.py`; two `test_const_enum_no_mint.py` methods, two registries each; 7 of the 12 in `tests/protocols/misc/test_pcapng_unit.py`). `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a two-line `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not the two *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled 5 of 82 regenerations, missing these two. Both crawlers now collapse to the single returning line the other 101 registries use, and the two const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process`'s shape legitimately breaks. The guard sees only 35 of 96 crawlers; the other 61 are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). -- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about failed sub-vendors without failing as a whole -- three pieces land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). -- three follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: one unpinned invariant and two inaccurate pieces of prose/lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: a test asserts that a `KeyboardInterrupt` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose `_dest_path` is an instance-method-only override fails the *class* call `_snapshot_and_restore` makes but not the *instance* call inside `__init__`, so that target runs unprotected and **succeeds silently**. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**: the belief that R1720 never applies to `try`/`except`/`else` was a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does), and under the `Makefile`'s flags, which CI runs, pylint flags the `else:`, which is deliberate because it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, since `copy2` reopens the path) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). -- `TLSKeyLabel` tracked the Mozilla NSS key-log wiki, but `draft-ietf-opsawg-pcapng-06` S4.7 now points at `[I-D.ietf-tls-keylogfile]` (draft-05), published as [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850), "The SSLKEYLOGFILE Format for TLS", whose S4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, `TLSKeyLabel` was missing `ECH_SECRET` and `ECH_CONFIG` ([RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) SS2.3/4.2), so `TLSKeyLabel('ECH_SECRET')` raised `ValueError` on a legitimate Decryption Secrets Block; both are added, `ECH_SECRET` keeping the file's `# nosec B105` bandit convention for `*_SECRET` members. `RSA` has no row in [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850)'s registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since removing a public member is breaking, with a docstring comment marking it the historical exception. `pcapkit/vendor/pcapng/secrets_type.py`'s stale `# NSS Key Log Format` comment is repointed at RFC 9850, and `TLSKeyLabel`'s docstring cites it. A new test pins the member set against RFC 9850 plus the `RSA` exception. Closes [#882](https://github.com/JarryShaw/PyPCAPKit/issues/882) ([#883](https://github.com/JarryShaw/PyPCAPKit/pull/883)). +- `re.sub` sites under `pcapkit/vendor/` passed `re.MULTILINE` as the fourth *positional* argument, which is `count`, not `flags`: substitution capped at 8 matches, and a `DeprecationWarning` on Python 3.13+ (a future `TypeError`). Found by an AST sweep rather than grep, since most put the flag on a continuation line. The flag moves to `flags=` at every site (`default.py` plus one each in `hip/eddsa_curve.py`, `http/frame.py`, `http/method.py`, `http/status_code.py`, `ipv4/router_alert.py`, `ipv6/option.py`, `ipv6/router_alert.py`, `ipv6/tagger_id.py`, `reg/ethertype.py`, `tcp/flags.py`, `tcp/mp_tcp_option.py` and `tcp/option.py`). Behaviour preserving: the shared pattern (`r'\r*\n'`) has no MULTILINE-sensitive anchor, so no `const/` regeneration was needed ([#796](https://github.com/JarryShaw/PyPCAPKit/issues/796)). +- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)): 12 tests across three files, all `RecursionError` (`tests/const/test_const_enum_lookup.py`; two `test_const_enum_no_mint.py` methods, two registries each; 7 of the 12 in `tests/protocols/misc/test_pcapng_unit.py`). `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a two-line `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not the two *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled only some regenerations, missing these two. Both crawlers now collapse to the single returning line the other registries use, and the two const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process`'s shape legitimately breaks. The guard sees only some crawlers; the rest are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). +- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about failed sub-vendors without failing as a whole -- the pieces below land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). +- follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: an unpinned invariant and inaccurate prose and lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: a test asserts that a `KeyboardInterrupt` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose `_dest_path` is an instance-method-only override fails the *class* call `_snapshot_and_restore` makes but not the *instance* call inside `__init__`, so that target runs unprotected and **succeeds silently**. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**: the belief that R1720 never applies to `try`/`except`/`else` was a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does), and under the `Makefile`'s flags, which CI runs, pylint flags the `else:`, which is deliberate because it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, since `copy2` reopens the path) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). +- `TLSKeyLabel` tracked the Mozilla NSS key-log wiki, but `draft-ietf-opsawg-pcapng-06` section 4.7 now points at `[I-D.ietf-tls-keylogfile]` (draft-05), published as [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850), "The SSLKEYLOGFILE Format for TLS", whose section 4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, `TLSKeyLabel` was missing `ECH_SECRET` and `ECH_CONFIG` ([RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) sections 2.3 and 4.2), so `TLSKeyLabel('ECH_SECRET')` raised `ValueError` on a legitimate Decryption Secrets Block; both are added, `ECH_SECRET` keeping the file's `# nosec B105` bandit convention for `*_SECRET` members. `RSA` has no row in [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850)'s registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since removing a public member is breaking, with a docstring comment marking it the historical exception. `pcapkit/vendor/pcapng/secrets_type.py`'s stale `# NSS Key Log Format` comment is repointed at RFC 9850, and `TLSKeyLabel`'s docstring cites it. A new test pins the member set against RFC 9850 plus the `RSA` exception. Closes [#882](https://github.com/JarryShaw/PyPCAPKit/issues/882) ([#883](https://github.com/JarryShaw/PyPCAPKit/pull/883)). - both spellings of an IANA citation render as `:rfc:` roles in the generated `pcapkit.const` docstrings ([#1664](https://github.com/JarryShaw/PyPCAPKit/issues/1664), [#1673](https://github.com/JarryShaw/PyPCAPKit/pull/1673)). - `MAX_RETRY` and `CI_MODE` stay in `pcapkit.vendor.default`, read from `PCAPKIT_VENDOR_RETRY` and `PCAPKIT_CI_MODE`; a bad value warns with `ConfigWarning` and falls back, where 1.4.x crashed on a non-integer and read `0` as `1` ([#1745](https://github.com/JarryShaw/PyPCAPKit/pull/1745)). - the IPX socket crawler no longer scrapes Wikipedia, whose page dropped the socket table, so a regeneration stops failing on it ([#507](https://github.com/JarryShaw/PyPCAPKit/issues/507), [#511](https://github.com/JarryShaw/PyPCAPKit/pull/511)). @@ -380,43 +384,47 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - an end-to-end test tier ([#376](https://github.com/JarryShaw/PyPCAPKit/pull/376)), sample-capture generators so a fresh clone can rebuild every fixture ([#340](https://github.com/JarryShaw/PyPCAPKit/pull/340)), a Dockerised engine benchmark covering every supported Python version ([#410](https://github.com/JarryShaw/PyPCAPKit/pull/410)), and registry round-trip coverage that records the entries which cannot close the cycle rather than skipping them ([#440](https://github.com/JarryShaw/PyPCAPKit/pull/440), [#504](https://github.com/JarryShaw/PyPCAPKit/pull/504)). - coverage for the untested half of the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation: a TCP or IPv4 option whose declared length asks for more data than the capture holds, pinning that it still parses with the short read zero-padded rather than rejected. The one test [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) left covers only an option area with no data behind it; a candidate fix for [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554) turned the untested half into an unwrapped `FieldValueError` while the rest of the suite stayed green ([#571](https://github.com/JarryShaw/PyPCAPKit/pull/571), [#572](https://github.com/JarryShaw/PyPCAPKit/issues/572)). - `CITATION.cff`, citation metadata in Citation File Format 1.2.0, which GitHub renders as the "Cite this repository" button and which citation managers and dependency inventories read directly. It is the machine-readable half of the attribution BSD-3-Clause already asks for. Validated with `cffconvert --validate` and against the published 1.2.0 schema; `doi` and `orcid` are omitted rather than invented, since neither exists for this project and both are checked formats, so a wrong value would still validate. The licence is unchanged (BSD-3-Clause, no `NOTICE`). Alongside it the copyright line moves from `2018-2023` to `2018-2026` (`LICENSE` was its only occurrence; `docs/source/conf.py` derives its own from the current year), and a stray `s` after the closing `DAMAGE.` of the licence text, present since the Mozilla-to-BSD relicence, is removed so the wording matches canonical BSD-3-Clause ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)). -- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository, which is why [#605](https://github.com/JarryShaw/PyPCAPKit/issues/605) survived its own code review: the one-character fix left the corrected path as untested as the broken one. The generator writes `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`), carrying the *same three records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet `snaplen`), so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives all three through `extract()` and derives expectations by walking each file's record chain with `struct`; a unit-tier case in `test_header_frame_unit.py` builds a two-record big-endian capture in memory, so the fixture-free selection CI runs on every push catches it too. All four fail on the unfixed tree (the three fixture-backed by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000`) while the little-endian twin passes on both, showing the records themselves are not the variable ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). +- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository, which is why [#605](https://github.com/JarryShaw/PyPCAPKit/issues/605) survived its own code review: the one-character fix left the corrected path as untested as the broken one. The generator writes `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`), carrying the *same records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet `snaplen`), so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives all three through `extract()` and derives expectations by walking each file's record chain with `struct`; a unit-tier case in `test_header_frame_unit.py` builds a two-record big-endian capture in memory, so the fixture-free selection CI runs on every push catches it too. Each fails on the unfixed tree (the fixture-backed ones by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000`) while the little-endian twin passes on both, showing the records themselves are not the variable ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). #### Changed - renames with no compatibility alias left behind: `examples/sample` and `examples/samples` -- one letter apart, holding different things -- are now `examples/captures` and `examples/generators`. -- the README is a landing page now, and Markdown rather than reStructuredText. `README.rst` (424 lines) became `README.md` (102), keeping what a reader arriving from PyPI or a search result needs (what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, where the documentation lives). **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in `docs/source/index.rst`, so they are linked rather than restated; the two blocks that existed nowhere else moved: **Testing** is now `docs/source/contributing/testing.rst`, and the `pipenv` and `make setup` block joined the Installation section of `docs/source/index.rst`. Requested by the project owner, and a deliberate exception to the reStructuredText convention, for the README only, since its renderers are GitHub and PyPI rather than Sphinx. `setup.py` reads `README.md` and declares `text/markdown`, and `MANIFEST.in` gains `include README.md`, which is belt-and-braces rather than load-bearing: setuptools' `sdist` ships a `README.md` regardless. Of the `include` lines only `CHANGELOG.md` is load-bearing, measured in the [#631](https://github.com/JarryShaw/PyPCAPKit/pull/631) entry below. `examples/benchmark/Dockerfile` copies `README.md`, while the `README.md` that `examples/benchmark/benchmark.py` names is the benchmark suite's own ([#619](https://github.com/JarryShaw/PyPCAPKit/pull/619)). -- `CODE_OF_CONDUCT.md` moves from Contributor Covenant 1.4 to 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than transcribed. Three things needed deciding. 3.0's two `[NOTE` placeholders are filled or removed: the reporting channel names `jarryshaw@icloud.com` (the contact 1.4 carried and `SECURITY.md` uses as email fallback) plus GitHub's report-abuse form for a report about the maintainer, which a single-maintainer project cannot otherwise cover; the enforcement placeholder is an instruction to the adopter and is removed. 3.0's plural "Community Moderators" become the singular maintainer (eight occurrences), this repository having no moderators. The four-rung ladder (Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so the attribution names version 3.0, links the permanent `version/3/0/` URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the share-alike term covers this document only (the code remains BSD-3-Clause, `LICENSE` untouched). The ladder renders as four list items each nesting three, checked against GitHub's Markdown API; [#613](https://github.com/JarryShaw/PyPCAPKit/pull/613) had to repair the 1.4 file when a stray list marker collapsed it into one nested item ([#624](https://github.com/JarryShaw/PyPCAPKit/pull/624)). -- reconciled which private (`_xxx`) attributes and methods the Sphinx build documents, replacing an ad hoc mix with one rule: document the contract, hide the recipe. A member every subclass must implement, or whose shape a caller depends on, stays documented despite its underscore; a private helper that only keeps one method short does not. Seven module-level directives for pure implementation detail (`esp._resolve`, `esp._CRYPTO`, `ngap._convert`, `ngap._revert`, `ngap._PYCRATE`, `ngap._PDU_LOCK`, `pypcapfile._NamedStream`) were dropped, while 39 `autoattribute` directives were added for class-private state that *is* contract, among them `Extractor._flag_f`, the `PyPCAP` and `PCAP_CT` engines' `_backend` (the only two of the six third-party engines with one), `TraceFlow`'s internal fields, `FieldBase`/`Field` internals, the built-in `PCAP` engine's `_gbhdr`, `_vinfo`, `_nnsec` and `_dlink` (shared with `PyPCAP`, `PCAP_CT` and `PyPCAPFile`) and `PCAPNG`'s `_ctx`/`_ctx_list`. The 128 runtime definitions of `_missing_` (121 under `pcapkit.const`, 7 inline in `pcapkit.protocols`) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in `docs/source/pcapkit/const/index.rst` cross-referenced from `registry.rst` (`conf.py` already excludes `_missing_` from every `autoclass`). `CONTRIBUTING.md` gained the rule as a named section. The one new Sphinx warning, a bare `Type` newly rendered by `TraceFlow._foutio`'s directive, is [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709). No line under `pcapkit/` changed ([#684](https://github.com/JarryShaw/PyPCAPKit/issues/684)). -- `examples/captures/out.json`, `out.plist`, `out.txt` and `pcapng.txt` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. `pcapng.txt` is exactly that: it still recorded `packet -> NIL` for the four Enhanced Packet Blocks of `dhcp.pcapng` long after [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) gave those blocks their captured octets back. Rather than regenerate it again and leave the drift free to recur, the four files are removed from the index and folded into `examples/captures/`'s blanket `.gitignore` rule; the genuine inputs `in.pcap` and `dhcp.pcapng` stay tracked. `examples/legacy_smoke/Makefile` and its `README.md` now describe regenerating these reports via `make fixtures`, and `tests/project/test_capture_tracking.py` pins the invariant ([#685](https://github.com/JarryShaw/PyPCAPKit/issues/685)). -- `tests/project/test_isort_clean.py` gated on `isort` with a `try`/`except ImportError` inside `setUpClass`, which `tests/_dependency_gates.py` cannot see (`_gates_of()` walks a decorator list, never a function body), and `isort` was in no `pyproject.toml` extra, so the file reported `OK (skipped=1)` on every CI leg and the dependency-gate guard could not tell. It now gates on a module-level `HAS_ISORT` and a class-level `@unittest.skipUnless` so `gated_scopes()` counts it, `MODULE_PROVIDERS` maps `isort`, and `isort` joins the `test` extra rather than being excluded (`lint.yml` installs none, so unlike `mypy` there is no existing check to defer to). No workflow changed, every pytest job already building on `.[test]`; `gated_scopes()` goes 250 to 251 and `dependency_gate_gaps()` holds at 18. The third instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop ([#766](https://github.com/JarryShaw/PyPCAPKit/issues/766)). -- `tests/vendor/test_vendor_reg_apptype_generator_unit.py` gated on `mypy` with no `MODULE_PROVIDERS` entry, so `_top_level_providers()` died on a bare `KeyError: 'mypy'` once the gate became visible -- a further instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop, like [#766](https://github.com/JarryShaw/PyPCAPKit/issues/766) above. `MODULE_PROVIDERS['mypy'] = ('mypy',)` fixes the lookup, and the three bare `MODULE_PROVIDERS[...]` subscripts route through that function, which raises an `AssertionError` naming the module, key and table on any future gap. `mypy` is deliberately not added to a `pyproject.toml` extra: no extra carries it, it is a `Pipfile` dev-dependency that `lint.yml` installs and already runs whole-package, and `lint.yml` runs no pytest job for `pytest_jobs()` to see, so the fix is a `DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']` entry on `test`, `engine-tests` and `gate`. `dependency_gate_gaps()` goes 15 to 18 `(flag, job)` pairs, all three new and expected; ([#779](https://github.com/JarryShaw/PyPCAPKit/issues/779)). +- the README is a landing page now, and Markdown rather than reStructuredText. `README.rst` became a much shorter `README.md`, keeping what a reader arriving from PyPI or a search result needs (what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, where the documentation lives). **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in `docs/source/index.rst`, so they are linked rather than restated; the blocks that existed nowhere else moved: **Testing** is now `docs/source/contributing/testing.rst`, and the `pipenv` and `make setup` block joined the Installation section of `docs/source/index.rst`. Requested by the project owner, and a deliberate exception to the reStructuredText convention, for the README only, since its renderers are GitHub and PyPI rather than Sphinx. `setup.py` reads `README.md` and declares `text/markdown`, and `MANIFEST.in` gains `include README.md`, which is belt-and-braces rather than load-bearing: setuptools' `sdist` ships a `README.md` regardless. Of the `include` lines only `CHANGELOG.md` is load-bearing, measured in the [#631](https://github.com/JarryShaw/PyPCAPKit/pull/631) entry below. `examples/benchmark/Dockerfile` copies `README.md`, while the `README.md` that `examples/benchmark/benchmark.py` names is the benchmark suite's own ([#619](https://github.com/JarryShaw/PyPCAPKit/pull/619)). +- `CODE_OF_CONDUCT.md` moves from Contributor Covenant 1.4 to 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than transcribed. Some things needed deciding. 3.0's `[NOTE` placeholders are filled or removed: the reporting channel names `jarryshaw@icloud.com` (the contact 1.4 carried and `SECURITY.md` uses as email fallback) plus GitHub's report-abuse form for a report about the maintainer, which a single-maintainer project cannot otherwise cover; the enforcement placeholder is an instruction to the adopter and is removed. 3.0's plural "Community Moderators" become the singular maintainer, this repository having no moderators. The enforcement ladder (Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so the attribution names version 3.0, links the permanent `version/3/0/` URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the share-alike term covers this document only (the code remains BSD-3-Clause, `LICENSE` untouched). The ladder renders as nested list items, checked against GitHub's Markdown API; [#613](https://github.com/JarryShaw/PyPCAPKit/pull/613) had to repair the 1.4 file when a stray list marker collapsed it into one nested item ([#624](https://github.com/JarryShaw/PyPCAPKit/pull/624)). +- reconciled which private (`_xxx`) attributes and methods the Sphinx build documents, replacing an ad hoc mix with one rule: document the contract, hide the recipe. A member every subclass must implement, or whose shape a caller depends on, stays documented despite its underscore; a private helper that only keeps one method short does not. Module-level directives for pure implementation detail (`esp._resolve`, `esp._CRYPTO`, `ngap._convert`, `ngap._revert`, `ngap._PYCRATE`, `ngap._PDU_LOCK`, `pypcapfile._NamedStream`) were dropped, while `autoattribute` directives were added for class-private state that *is* contract, among them `Extractor._flag_f`, the `PyPCAP` and `PCAP_CT` engines' `_backend` (the only third-party engines with one), `TraceFlow`'s internal fields, `FieldBase`/`Field` internals, the built-in `PCAP` engine's `_gbhdr`, `_vinfo`, `_nnsec` and `_dlink` (shared with `PyPCAP`, `PCAP_CT` and `PyPCAPFile`) and `PCAPNG`'s `_ctx`/`_ctx_list`. The runtime definitions of `_missing_` (under `pcapkit.const`, and inline in `pcapkit.protocols`) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in `docs/source/pcapkit/const/index.rst` cross-referenced from `registry.rst` (`conf.py` already excludes `_missing_` from every `autoclass`). `CONTRIBUTING.md` gained the rule as a named section. The new Sphinx warning, a bare `Type` newly rendered by `TraceFlow._foutio`'s directive, is [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709). No line under `pcapkit/` changed ([#684](https://github.com/JarryShaw/PyPCAPKit/issues/684)). +- `examples/captures/out.json`, `out.plist`, `out.txt` and `pcapng.txt` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. `pcapng.txt` is exactly that: it still recorded `packet -> NIL` for the Enhanced Packet Blocks of `dhcp.pcapng` long after [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) gave those blocks their captured octets back. Rather than regenerate it again and leave the drift free to recur, these files are removed from the index and folded into `examples/captures/`'s blanket `.gitignore` rule; the genuine inputs `in.pcap` and `dhcp.pcapng` stay tracked. `examples/legacy_smoke/Makefile` and its `README.md` now describe regenerating these reports via `make fixtures`, and `tests/project/test_capture_tracking.py` pins the invariant ([#685](https://github.com/JarryShaw/PyPCAPKit/issues/685)). +- `tests/project/test_isort_clean.py` gated on `isort` with a `try`/`except ImportError` inside `setUpClass`, which `tests/_dependency_gates.py` cannot see (`_gates_of()` walks a decorator list, never a function body), and `isort` was in no `pyproject.toml` extra, so the file reported `OK (skipped=1)` on every CI leg and the dependency-gate guard could not tell. It now gates on a module-level `HAS_ISORT` and a class-level `@unittest.skipUnless` so `gated_scopes()` counts it, `MODULE_PROVIDERS` maps `isort`, and `isort` joins the `test` extra rather than being excluded (`lint.yml` installs none, so unlike `mypy` there is no existing check to defer to). No workflow changed, every pytest job already building on `.[test]`; `gated_scopes()` gains this scope and `dependency_gate_gaps()` is unchanged. Another instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop ([#766](https://github.com/JarryShaw/PyPCAPKit/issues/766)). +- `tests/vendor/test_vendor_reg_apptype_generator_unit.py` gated on `mypy` with no `MODULE_PROVIDERS` entry, so `_top_level_providers()` died on a bare `KeyError: 'mypy'` once the gate became visible -- a further instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop, like [#766](https://github.com/JarryShaw/PyPCAPKit/issues/766) above. `MODULE_PROVIDERS['mypy'] = ('mypy',)` fixes the lookup, and the bare `MODULE_PROVIDERS[...]` subscripts route through that function, which raises an `AssertionError` naming the module, key and table on any future gap. `mypy` is deliberately not added to a `pyproject.toml` extra: no extra carries it, it is a `Pipfile` dev-dependency that `lint.yml` installs and already runs whole-package, and `lint.yml` runs no pytest job for `pytest_jobs()` to see, so the fix is a `DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']` entry on `test`, `engine-tests` and `gate`. `dependency_gate_gaps()` gains the new `(flag, job)` pairs, each expected; ([#779](https://github.com/JarryShaw/PyPCAPKit/issues/779)). - `engine='pyshark'` had no CI job driving it through a real `tshark`-parsed capture. `pypcap-parity` installs `pyshark` but its `apt-get install` named no `tshark` package, so `PyShark.unsupported_reason()` declined the engine, the extractor fell back to the built-in parser, and the one real-extraction assertion, `assertGreater(extractor.length, 0)`, passed against the fallback's output: a vacuous pass invisible in every skip count. `pypcap-parity` now installs `tshark` with the debconf pre-seed `engine-tests` carries, and the test proves which engine ran (no `EngineWarning`, `extractor._exnam == 'pyshark'`, a `PyShark` instance), as the PyPCAP parity test does. Only `tests/integration/test_engine_runtime.py` and `test_engine_parity.py` change; the length assertion is kept ([#846](https://github.com/JarryShaw/PyPCAPKit/pull/846)). - `engine-tests` ran one Python-version leg per interpreter regardless of which engines that interpreter could install, so `pyproject.toml`'s `PyPCAPFile = ["pypcapfile; python_version < '3.12'"]` marker resolved to *nothing* on 3.12--3.14, yet the `Engines Python 3.12/3.13/3.14` legs passed and, once required, gated merges while exercising an engine that was never installed. Rebuilt as a Python x engine matrix (6 interpreters x 6 engines, 36 cells after 10 `include` overrides) with three outcomes: **26** `supported` cells that install through the real extra and must pass; **6** `unsupported` cells (`PyPCAPFile` on 3.12--3.15, `PyShark` on 3.14/3.15) installed by raw `pip install` to bypass the marker and assert the engine's own `PYTHON_CEILING` decline; and **4** `not-installable` cells (`PyPCAP` on 3.12--3.15) where a genuine C-extension build failure is logged and the test step skipped. The install step never passes silently in either direction. 3.15 is `continue-on-error` throughout. Job names change shape (`Engines Python X` becomes `Engines Python X (engine)`), so the branch-protection ruleset's required-check list was updated in the same window, lest a required check that stopped existing block every merge ([#849](https://github.com/JarryShaw/PyPCAPKit/pull/849)). - Ruleset `23497679`'s `required_status_checks` names 22 exact check contexts, and GitHub rulesets match literally with no wildcard, so the list needs hand-editing on every matrix change. Five of the 22, the old `Engines Python ` names, are dead since [#849](https://github.com/JarryShaw/PyPCAPKit/pull/849); the five `Compat Python 3.10`-`3.14` names stay live, emitted by `python-compatibility.yml`, and are untouched. Adds one job, `required-checks` (context `Required checks passed`), depending on `[test, integration, engine-tests, pypcap-parity]` with `if: always() && inputs.gate-only != true` and an explicit per-dependency check that fails loudly, naming the job, on anything but `success`: a bare `needs:` list would leave it *skipped* rather than failed when a dependency failed, and GitHub's docs list a skipped required check as passing. That one context is meant to replace 17 of the ruleset's 22 entries (`test`, `integration`, `Engines`, `pypcap-parity`) once the ruleset is updated separately, as a repository setting outside the change; the five `Compat` names are excluded from the replacement. One file, `.github/workflows/unit-tests.yml`, 167 lines added; no library or test code ([#856](https://github.com/JarryShaw/PyPCAPKit/pull/856)). -- the prose the post-[#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) defect wave added or touched, checked against the code; no code changes. `MesgIDOption.seconds` in the Mobility Header schema said seconds since 1970 where it holds an [RFC 1305](https://datatracker.ietf.org/doc/html/rfc1305) NTP count from 1900; two `HTTP._guess_version` comments predated the HTTP/2 gate; and the `_parse_next_layer` note miscounted the captured octets. `Extractor._open_output` now says only the PCAP engine passes header fields, `Extractor.run` and the PCAP-NG engine's `run` list the `FormatError` for PCAP output from a PCAP-NG input, and the extraction and engine pages document the PCAP-output path ([#1179](https://github.com/JarryShaw/PyPCAPKit/pull/1179), [#1180](https://github.com/JarryShaw/PyPCAPKit/pull/1180), [#1186](https://github.com/JarryShaw/PyPCAPKit/pull/1186), [#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103)). -- **a breaking change to** the `all` extra: it carries the core addons only -- `cli`, `crypto` and `NGAP`. The three third-party engines 1.4.x bundled in it (`dpkt`, `scapy` and `pyshark`) and the vendor crawler's `requests` and `beautifulsoup4` are no longer in it; the engines keep their own `DPKT`, `Scapy` and `PyShark` extras and the crawler its `vendor` extra, and `pypcapfile` has a new `PyPCAPFile` extra of its own. A new `dev` extra carries what CI needs ([#910](https://github.com/JarryShaw/PyPCAPKit/issues/910), [#914](https://github.com/JarryShaw/PyPCAPKit/pull/914)). +- the prose the post-[#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) defect wave added or touched, checked against the code; no code changes. `MesgIDOption.seconds` in the Mobility Header schema said seconds since 1970 where it holds an [RFC 1305](https://datatracker.ietf.org/doc/html/rfc1305) NTP count from 1900; `HTTP._guess_version` comments predated the HTTP/2 gate; and the `_parse_next_layer` note miscounted the captured octets. `Extractor._open_output` now says only the PCAP engine passes header fields, `Extractor.run` and the PCAP-NG engine's `run` list the `FormatError` for PCAP output from a PCAP-NG input, and the extraction and engine pages document the PCAP-output path ([#1179](https://github.com/JarryShaw/PyPCAPKit/pull/1179), [#1180](https://github.com/JarryShaw/PyPCAPKit/pull/1180), [#1186](https://github.com/JarryShaw/PyPCAPKit/pull/1186), [#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103)). +- **a breaking change to** the `all` extra: it carries the core addons only -- `cli`, `crypto` and `NGAP`. The third-party engines 1.4.x bundled in it (`dpkt`, `scapy` and `pyshark`) and the vendor crawler's `requests` and `beautifulsoup4` are no longer in it; the engines keep their own `DPKT`, `Scapy` and `PyShark` extras and the crawler its `vendor` extra, and `pypcapfile` has a new `PyPCAPFile` extra of its own. A new `dev` extra carries what CI needs ([#910](https://github.com/JarryShaw/PyPCAPKit/issues/910), [#914](https://github.com/JarryShaw/PyPCAPKit/pull/914)). - the `Scapy` and `dev` extras require `scapy>=2.6.0` ([#1572](https://github.com/JarryShaw/PyPCAPKit/issues/1572), [#1578](https://github.com/JarryShaw/PyPCAPKit/pull/1578)), and the `docs` extra `Sphinx>=7.3`, the first release with `versionremoved` ([#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). -- the READMEs of `examples/benchmark` and `examples/legacy_smoke` are Markdown, `README.md` rather than `README.rst`, and `MANIFEST.in` ships them through `recursive-include examples *.md`. Stale build leftovers go: `setup.py`'s `bdist_wheel` override, since `build_py` and `install` already convert the tree the wheel archives, and `MANIFEST.in`'s `prune test`, which named no directory; `make profile` runs in `examples/legacy_smoke` ([#1773](https://github.com/JarryShaw/PyPCAPKit/issues/1773), [#1780](https://github.com/JarryShaw/PyPCAPKit/pull/1780)). +- the READMEs of `examples/benchmark` and `examples/legacy_smoke` are Markdown, `README.md` rather than `README.rst`, and `MANIFEST.in` ships them through `recursive-include examples *.md`. Stale build leftovers go: `setup.py`'s `bdist_wheel` override, since `build_py` and `install` already convert the tree the wheel archives, and `MANIFEST.in`'s `prune test`, which named no directory; `make profile` runs in `examples/legacy_smoke` ([#1773](https://github.com/JarryShaw/PyPCAPKit/issues/1773), [#1780](https://github.com/JarryShaw/PyPCAPKit/pull/1780)). `MANIFEST.in` also drops the lines that matched nothing -- `global-include *.rst`, `include pcapkit/_extern` and `exclude pcapkit/protocols/*/NotImplemented` -- and the source distribution and wheel are unchanged ([#1794](https://github.com/JarryShaw/PyPCAPKit/issues/1794), [#1813](https://github.com/JarryShaw/PyPCAPKit/pull/1813)). +- the contributor pages state no counts that a test or the Python version updater pinned, and each pin checks the fact it stood for instead ([#1811](https://github.com/JarryShaw/PyPCAPKit/issues/1811), [#1817](https://github.com/JarryShaw/PyPCAPKit/pull/1817)). #### Fixed -- 45 places where a documentation page contradicted the code ([#413](https://github.com/JarryShaw/PyPCAPKit/pull/413)), ambiguous cross-references and five autodoc signature failures ([#416](https://github.com/JarryShaw/PyPCAPKit/pull/416)), and `Extractor`'s documented exception plus 40 phantom or stale `Args:` labels ([#501](https://github.com/JarryShaw/PyPCAPKit/pull/501)). -- both halves of what the `Changelog drift` gate told an author, in `util/changelog_md.py`. `ResidualMarkupError` named the entry file but numbered its lines against the *converted* body, which rule 6 joins onto one line per block (55 lines against the file's 580 on this entry), so a reported number could not reach most of the file, and four roles on two source lines were all reported as `line 46`. The conversion now carries a source map, so every complaint cites a line of the file the message names, one per construct, in the entry's order ([#588](https://github.com/JarryShaw/PyPCAPKit/issues/588)). Rule 2 accepted only the bare-number spelling of Sphinx's `:rfc:` role, so [RFC 6554 Section 3](https://datatracker.ietf.org/doc/html/rfc6554#section-3) fell past it and was reported as a role the rules do not cover; both spellings now convert, the anchor carried into the link target and the link text taken from Sphinx's own ([#592](https://github.com/JarryShaw/PyPCAPKit/issues/592)). Regenerating `CHANGELOG.md` is byte-identical over all 37 committed entries: the fix changes what the gate *says*, not what it emits. -- `examples/generators/options.py`'s `TCP_BASE` spelled three header fields with names `TCP.make` does not declare, so every option fixture in `examples/captures/options-tcp.pcap` was built from values the generator had not asked for. `make` ends in `**kwargs` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. `'seq': 1` was `seq_no`, so all 25 captured frames carried sequence number 0; `'urgent_pointer'` was `urgent` (requested value and default both 0); and `'ack_flag': False` was `ack` while `'ack': 0` bound to `ack` itself, the acknowledgement *flag* rather than the number (`ack_no`), so that pair was wrong both ways. Each is renamed to the parameter `make` declares, keeping the intended values. `TCP(**TCP_BASE).info.seq` was 0 against a declared 1 and is now 1, and `options-tcp.pcap` changes in exactly 25 bytes (the low octet of each frame's sequence number), with no case status or `EXPECTED_FAILURES` entry moving. The module's six other base mappings were audited: five pass nothing their `make` does not declare, and HIP's `extension` is declared by `HIP.read`, this library's documented way of forwarding a read-path keyword, not a sixth instance ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). -- `util/bump_version.py` left `CITATION.cff` naming the previous release. Nothing else maintains that file, so every bump since [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) would have stranded the `version` and `date-released` that GitHub's "Cite this repository" button, citation managers, Zenodo and dependency inventories read. Both fields now move with `__version__`, together, since the file's header says both describe the newest *published* release and moving only one would assert that 1.5.0b5 was released on the day 1.5.0b4 was. The date is UTC, because seven of the thirty most recent bumps were made late evening US-Eastern, where a local date is a day behind the publish; the bump triggers `create-release.yml`, the median gap to the PyPI upload is three minutes, and the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment header, key ordering and each field's quoting survive (`cff-version` and a `references` entry's own `version` are anchored out at column zero), and the result is checked with `cffconvert --validate`. An absent file is reported on stderr and skipped rather than failing the vendor cron before its `git commit`, which would discard a whole registry crawl for a documentation file; a file with no `version` field raises before anything is written, since reporting success while rewriting nothing is the staleness being fixed. The script gains a `main()` guard (it ran the whole bump at import, hence no testable surface), and the `import pcapkit` fallback in its version reader, which returned `"1.5.0b4'\n"` (closing quote and newline, which `packaging` rejects), is fixed; it never worked and went unnoticed because the only caller installs the package first. A new gate asserts the committed file still names the packaged version, covering hand-made version changes that never run this script (40 of the 159 commits that moved `__version__` on `main`, 11 of the latest 25) ([#625](https://github.com/JarryShaw/PyPCAPKit/pull/625)). -- `LICENSE`'s copyright notice began the term at 2018, a year after the work it covers. The repository's first commit is `c57f7d0b7`, dated 2017-11-07, so the notice understated the term and contradicted `docs/source/conf.py`, whose Sphinx footer is `f'2017-{datetime.date.today().year}, Jarry Shaw'`. The wrong start year survived every maintenance pass because each looked only at the other end of the range: `bc836cfa2` (2020-05-31) introduced `2018-2020` when the BSD-3-Clause text replaced MPL 2.0, and hand bumps to `2018-2022`, `2018-2023` and `2018-2026` ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)) each copied `2018` forward. 2017 is **restored** rather than newly asserted: the initial MIT `LICENSE` read `Copyright (c) 2017 Jarry Shaw`, and the GPL v3 era carried `Copyright (C) 2017 Jarry Shaw` in its appendix; it was lost on 2018-12-08, when `4f3e00d5f` relicensed to Apache 2.0 under `Copyright 2018 Jarry Shaw` and `1c69341dc` replaced that with MPL 2.0 text with no author notice. The end year is **dropped** rather than automated: the notice ships inside the sdist and wheel, so its text is fixed at build time in every copy already downloaded and cannot be computed like the docs footer; copyright subsists from creation whether or not a notice names the current year; and BSD-3-Clause's canonical form is a singular `Copyright (c) `. A range was six years of hand maintenance buying nothing, and automation was considered and rejected, since automating a value that need not be current is worse than not carrying it. `conf.py` is untouched (a self-maintaining footer showing a range is conventional and correct). The licence body is unaltered: one hunk at line 3, and the `DAMAGE.` that [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) repaired from `DAMAGE.s` is still clean ([#630](https://github.com/JarryShaw/PyPCAPKit/pull/630)). -- `CITATION.cff` shipped in no source distribution. `MANIFEST.in` has an `include` each for `README.md`, `LICENSE` and `CHANGELOG.md` but none for the citation file, and its two `global-include` patterns (`*.rst`, `*.py`) cannot match a `.cff`. Removing all three `include` lines and rebuilding shows `README.md` and `LICENSE` shipping anyway (setuptools adds the latter from `license_files`, recorded as `License-File: LICENSE` in `PKG-INFO`) while `CHANGELOG.md` vanishes, so only `CHANGELOG.md` is load-bearing, and a citation file, which no packaging default covers, is in the same position. The gap is invisible from the web UI, because GitHub renders "Cite this repository" from the repository; citation managers, Zenodo and dependency inventories read the published artifact, the surface that was missing it. [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) flagged the omission and [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) did not address it. It matters because [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) taught `util/bump_version.py` to keep the file's `version` and `date-released` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets `RepositoryCitationTests` in `tests/project/test_bump_version.py` (the gate [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) added for hand-authored bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is not shipped in the source distribution". `python -m build --sdist` gives 860 archive entries before and 861 after (`pypcapkit-1.5.0b4/CITATION.cff` the only difference), the shipped copy is byte-identical, and `twine check --strict` passes on both ([#631](https://github.com/JarryShaw/PyPCAPKit/pull/631)). -- two inaccurate claims in the [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) entry above, caught in review after [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) had merged; both concern the history of `LICENSE`, which is untouched here. 2017 was and remains the right answer, so this is a changelog-prose correction. The entry said the file "until then was stock MPL text carrying no author notice at all, so 2018 is the first start year the project ever asserted and it was already a year late when it was written". That was true of the single blob `bc836cfa2` replaced and wrong as a description of the era: the pre-BSD file was MIT, then GPL v3, then Apache 2.0, then MPL 2.0; the MIT and GPL texts both carried an author notice naming **2017**; and `2018` first appeared in the Apache notice of 2018-12-08 as a single year, current when written. The claim also contradicted the entry's own dating of the first commit to 2017-11-07. The history supports a stronger argument than the one written, which is why this is corrected rather than deleted: 2017 is the year asserted from the first commit, and [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) restores it. The `BSD 3-Clause License` title line is reattributed from opensource.org's template to choosealicense.com's, whose first line is that title and which carries the `Copyright (c) [year], [fullname]` comma this file uses. The error came from generalising the pre-BSD era off two blobs that were the same file (`bc836cfa2^:LICENSE` and `1c69341dc:LICENSE`); enumerating all 14 commits that touched `LICENSE` caught it ([#638](https://github.com/JarryShaw/PyPCAPKit/pull/638)). -- the release workflow published to PyPI and to Anaconda with nothing a human had to approve. `create-release.yml`'s `pypi` job had `environment: release` commented out while the `id-token: write` from the same upstream snippet was re-added live below it, so the block read as disabled as a unit when only its gate still was; the `conda` job had no `environment:` at all. Neither needed a tag: the workflow also fires on completion of `Vendor Update`, so a scheduled registry crawl that bumped the version was enough to publish to two public indexes, a median three minutes from bump to upload. Four jobs reach outside the run and all four are now gated: `github` on `github-release` (the GitHub Release and the `v*` tag), `tag` on `conda-tag` (the commit pushed to `main`), `pypi` on `pypi`, `conda` on `anaconda`. One environment per credential rather than a shared `release`, because approval is granted to an environment and not a job, so one `release` approved once would release both indexes together. PyPI has to be answerable alone: an accepted version number cannot be reused, and `skip-existing: true` makes the re-upload *succeed* having published nothing, so an unintended publish permanently consumes the number the intended release wanted. The workflow half is only half the fix: an `environment:` naming one that does not exist is created implicitly with no protection rules and the job proceeds unapproved, so this is inert until each of the four exists in repository settings with a required reviewer, and each one's "Deployment branches and tags" must stay unrestricted or the `tags: v*` trigger fails outright instead of pausing. `github-pages` is the standing example, carrying no protection rule since 2021. The new test asserts the general rule, that no job running a publishing action or a `git push` may omit an `environment:`, so a publishing job added later without a gate fails rather than ships ([#641](https://github.com/JarryShaw/PyPCAPKit/issues/641)). -- three test comments stated an exact count of committed captures under `examples/captures/`, a number that drifts whenever the tracked set changes and in one case was wrong when written: `tests/_tiers.py` said "six," `tests/test_tier_guard.py` "the moment a seventh capture is committed," and `tests/integration/_helpers.py` "four," which undercounted on the day it was written. All three now describe the invariant (`_tiers.py`: "the moment somebody commits another capture... or stops committing one"; `_helpers.py`: "...are fixtures -- some of them committed --..."; `test_tier_guard.py` drops its ordinal). Prose-only, verified with `coverage.parser.PythonParser` that no executable statement moved ([#700](https://github.com/JarryShaw/PyPCAPKit/issues/700)). +- places where a documentation page contradicted the code ([#413](https://github.com/JarryShaw/PyPCAPKit/pull/413)), ambiguous cross-references and autodoc signature failures ([#416](https://github.com/JarryShaw/PyPCAPKit/pull/416)), and `Extractor`'s documented exception plus phantom or stale `Args:` labels ([#501](https://github.com/JarryShaw/PyPCAPKit/pull/501)). +- both halves of what the `Changelog drift` gate told an author, in `util/changelog_md.py`. `ResidualMarkupError` named the entry file but numbered its lines against the *converted* body, which rule 6 joins onto one line per block, so a reported number could not reach most of the file, and roles on separate source lines were all reported as one line. The conversion now carries a source map, so every complaint cites a line of the file the message names, one per construct, in the entry's order ([#588](https://github.com/JarryShaw/PyPCAPKit/issues/588)). Rule 2 accepted only the bare-number spelling of Sphinx's `:rfc:` role, so [RFC 6554 Section 3](https://datatracker.ietf.org/doc/html/rfc6554#section-3) fell past it and was reported as a role the rules do not cover; both spellings now convert, the anchor carried into the link target and the link text taken from Sphinx's own ([#592](https://github.com/JarryShaw/PyPCAPKit/issues/592)). Regenerating `CHANGELOG.md` is byte-identical over every committed entry: the fix changes what the gate *says*, not what it emits. +- `examples/generators/options.py`'s `TCP_BASE` spelled header fields with names `TCP.make` does not declare, so every option fixture in `examples/captures/options-tcp.pcap` was built from values the generator had not asked for. `make` ends in `**kwargs` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. `'seq': 1` was `seq_no`, so every captured frame carried sequence number 0; `'urgent_pointer'` was `urgent` (requested value and default both 0); and `'ack_flag': False` was `ack` while `'ack': 0` bound to `ack` itself, the acknowledgement *flag* rather than the number (`ack_no`), so that pair was wrong both ways. Each is renamed to the parameter `make` declares, keeping the intended values. `TCP(**TCP_BASE).info.seq` was 0 against a declared 1 and is now 1, and `options-tcp.pcap` changes only in the low octet of each frame's sequence number, with no case status or `EXPECTED_FAILURES` entry moving. The module's other base mappings were audited: the rest pass nothing their `make` does not declare, and HIP's `extension` is declared by `HIP.read`, this library's documented way of forwarding a read-path keyword, not another instance ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). +- `util/bump_version.py` left `CITATION.cff` naming the previous release. Nothing else maintains that file, so every bump since [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) would have stranded the `version` and `date-released` that GitHub's "Cite this repository" button, citation managers, Zenodo and dependency inventories read. Both fields now move with `__version__`, together, since the file's header says both describe the newest *published* release and moving only one would assert that 1.5.0b5 was released on the day 1.5.0b4 was. The date is UTC, because seven of the thirty most recent bumps were made late evening US-Eastern, where a local date is a day behind the publish; the bump triggers `create-release.yml`, the median gap to the PyPI upload is three minutes, and the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment header, key ordering and each field's quoting survive (`cff-version` and a `references` entry's own `version` are anchored out at column zero), and the result is checked with `cffconvert --validate`. An absent file is reported on stderr and skipped rather than failing the vendor cron before its `git commit`, which would discard a whole registry crawl for a documentation file; a file with no `version` field raises before anything is written, since reporting success while rewriting nothing is the staleness being fixed. The script gains a `main()` guard (it ran the whole bump at import, hence no testable surface), and the `import pcapkit` fallback in its version reader, which returned `"1.5.0b4'\n"` (closing quote and newline, which `packaging` rejects), is fixed; it never worked and went unnoticed because the only caller installs the package first. A new gate asserts the committed file still names the packaged version, covering hand-made version changes that never run this script ([#625](https://github.com/JarryShaw/PyPCAPKit/pull/625)). +- `LICENSE`'s copyright notice began the term at 2018, a year after the work it covers. The repository's first commit is `c57f7d0b7`, dated 2017-11-07, so the notice understated the term and contradicted `docs/source/conf.py`, whose Sphinx footer is `f'2017-{datetime.date.today().year}, Jarry Shaw'`. The wrong start year survived every maintenance pass because each looked only at the other end of the range: `bc836cfa2` (2020-05-31) introduced `2018-2020` when the BSD-3-Clause text replaced MPL 2.0, and hand bumps to `2018-2022`, `2018-2023` and `2018-2026` ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)) each copied `2018` forward. 2017 is **restored** rather than newly asserted: the initial MIT `LICENSE` read `Copyright (c) 2017 Jarry Shaw`, and the GPL v3 era carried `Copyright (C) 2017 Jarry Shaw` in its appendix; it was lost on 2018-12-08, when `4f3e00d5f` relicensed to Apache 2.0 under `Copyright 2018 Jarry Shaw` and `1c69341dc` replaced that with MPL 2.0 text with no author notice. The end year is **dropped** rather than automated: the notice ships inside the sdist and wheel, so its text is fixed at build time in every copy already downloaded and cannot be computed like the docs footer; copyright subsists from creation whether or not a notice names the current year; and BSD-3-Clause's canonical form is a singular `Copyright (c) `. A range was six years of hand maintenance buying nothing, and automation was considered and rejected, since automating a value that need not be current is worse than not carrying it. `conf.py` is untouched (a self-maintaining footer showing a range is conventional and correct). The licence body is unaltered: one hunk, on the copyright line, and the `DAMAGE.` that [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) repaired from `DAMAGE.s` is still clean ([#630](https://github.com/JarryShaw/PyPCAPKit/pull/630)). +- `CITATION.cff` shipped in no source distribution. `MANIFEST.in` has an `include` each for `README.md`, `LICENSE` and `CHANGELOG.md` but none for the citation file, and its `global-include` patterns (`*.rst`, `*.py`) cannot match a `.cff`. Removing those `include` lines and rebuilding shows `README.md` and `LICENSE` shipping anyway (setuptools adds the latter from `license_files`, recorded as `License-File: LICENSE` in `PKG-INFO`) while `CHANGELOG.md` vanishes, so only `CHANGELOG.md` is load-bearing, and a citation file, which no packaging default covers, is in the same position. The gap is invisible from the web UI, because GitHub renders "Cite this repository" from the repository; citation managers, Zenodo and dependency inventories read the published artifact, the surface that was missing it. [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) flagged the omission and [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) did not address it. It matters because [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) taught `util/bump_version.py` to keep the file's `version` and `date-released` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets `RepositoryCitationTests` in `tests/project/test_bump_version.py` (the gate [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) added for hand-authored bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is not shipped in the source distribution". `python -m build --sdist` gives one more archive entry after (`pypcapkit-1.5.0b4/CITATION.cff` the only difference), the shipped copy is byte-identical, and `twine check --strict` passes on both ([#631](https://github.com/JarryShaw/PyPCAPKit/pull/631)). +- inaccurate claims in the [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) entry above, caught in review after [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) had merged; both concern the history of `LICENSE`, which is untouched here. 2017 was and remains the right answer, so this is a changelog-prose correction. The entry said the file "until then was stock MPL text carrying no author notice at all, so 2018 is the first start year the project ever asserted and it was already a year late when it was written". That was true of the single blob `bc836cfa2` replaced and wrong as a description of the era: the pre-BSD file was MIT, then GPL v3, then Apache 2.0, then MPL 2.0; the MIT and GPL texts both carried an author notice naming **2017**; and `2018` first appeared in the Apache notice of 2018-12-08 as a single year, current when written. The claim also contradicted the entry's own dating of the first commit to 2017-11-07. The history supports a stronger argument than the one written, which is why this is corrected rather than deleted: 2017 is the year asserted from the first commit, and [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) restores it. The `BSD 3-Clause License` title line is reattributed from opensource.org's template to choosealicense.com's, whose first line is that title and which carries the `Copyright (c) [year], [fullname]` comma this file uses. The error came from generalising the pre-BSD era off two blobs that were the same file (`bc836cfa2^:LICENSE` and `1c69341dc:LICENSE`); enumerating every commit that touched `LICENSE` caught it ([#638](https://github.com/JarryShaw/PyPCAPKit/pull/638)). +- the release workflow published to PyPI and to Anaconda with nothing a human had to approve. `create-release.yml`'s `pypi` job had `environment: release` commented out while the `id-token: write` from the same upstream snippet was re-added live below it, so the block read as disabled as a unit when only its gate still was; the `conda` job had no `environment:` at all. Neither needed a tag: the workflow also fires on completion of `Vendor Update`, so a scheduled registry crawl that bumped the version was enough to publish to two public indexes, a median three minutes from bump to upload. Every job that reaches outside the run is now gated: `github` on `github-release` (the GitHub Release and the `v*` tag), `tag` on `conda-tag` (the commit pushed to `main`), `pypi` on `pypi`, `conda` on `anaconda`. One environment per credential rather than a shared `release`, because approval is granted to an environment and not a job, so one `release` approved once would release both indexes together. PyPI has to be answerable alone: an accepted version number cannot be reused, and `skip-existing: true` makes the re-upload *succeed* having published nothing, so an unintended publish permanently consumes the number the intended release wanted. The workflow half is only half the fix: an `environment:` naming one that does not exist is created implicitly with no protection rules and the job proceeds unapproved, so this is inert until each of these environments exists in repository settings with a required reviewer, and each one's "Deployment branches and tags" must stay unrestricted or the `tags: v*` trigger fails outright instead of pausing. `github-pages` is the standing example, carrying no protection rule since 2021. The new test asserts the general rule, that no job running a publishing action or a `git push` may omit an `environment:`, so a publishing job added later without a gate fails rather than ships ([#641](https://github.com/JarryShaw/PyPCAPKit/issues/641)). +- test comments stated an exact count of committed captures under `examples/captures/`, a number that drifts whenever the tracked set changes and in one case was wrong when written: `tests/_tiers.py` said "six," `tests/test_tier_guard.py` "the moment a seventh capture is committed," and `tests/integration/_helpers.py` "four," which undercounted on the day it was written. Each now describes the invariant (`_tiers.py`: "the moment somebody commits another capture... or stops committing one"; `_helpers.py`: "...are fixtures -- some of them committed --..."; `test_tier_guard.py` drops its ordinal). Prose-only, verified with `coverage.parser.PythonParser` that no executable statement moved ([#700](https://github.com/JarryShaw/PyPCAPKit/issues/700)). - `test_every_tracked_name_exists_and_matches_git` in `tests/test_tier_guard.py` checked neither the git index nor the filesystem despite its name: it asserted only that the tracked-capture list was non-empty and no name contained a `/`, so a hardcoded list passed whether or not it matched reality. Sibling `test_capture_suggestions_are_captures` had the same gap. Fixed in [#703](https://github.com/JarryShaw/PyPCAPKit/pull/703); [#703](https://github.com/JarryShaw/PyPCAPKit/pull/703)'s description names both [#700](https://github.com/JarryShaw/PyPCAPKit/issues/700) and [#708](https://github.com/JarryShaw/PyPCAPKit/issues/708). The first test now re-derives the expected set from `git ls-files -z` under `examples/captures/`, asserts equality against it, and checks each name with `Path.is_file()`; its sibling re-implements `committed_capture_names()`'s filter inline, which catches a wrong filter but still trusts `committed_captures()` for the tracked set. A hardcoded literal that happens to match the tracked names still passes; the test catches a set that is wrong when it runs ([#708](https://github.com/JarryShaw/PyPCAPKit/issues/708)). -- `pcapkit.all.__all__` listed `OSPF`, `RARP` and `DRARP` on its Link Layer line after [#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) moved them ([#1124](https://github.com/JarryShaw/PyPCAPKit/issues/1124)), and omitted 18 names its packages export, among them `ESP`, `HTTPv1`, `HTTPv2`, `PCAPNG` and `EnumRegistry`, so `from pcapkit.all import *` did not bind them. `pcapkit.utilities` stays unlisted deliberately, so generic helpers such as `warn` do not arrive through `import *` ([#1136](https://github.com/JarryShaw/PyPCAPKit/issues/1136)). +- `pcapkit.all.__all__` listed `OSPF`, `RARP` and `DRARP` on its Link Layer line after [#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) moved them ([#1124](https://github.com/JarryShaw/PyPCAPKit/issues/1124)), and omitted names its packages export, among them `ESP`, `HTTPv1`, `HTTPv2`, `PCAPNG` and `EnumRegistry`, so `from pcapkit.all import *` did not bind them. `pcapkit.utilities` stays unlisted deliberately, so generic helpers such as `warn` do not arrive through `import *` ([#1136](https://github.com/JarryShaw/PyPCAPKit/issues/1136)). - `examples/generators/options.py` wrote every HTTP/2 sample frame on stream 0, which [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) forbids for `DATA`, `HEADERS` and the other stream-level types, so six of the ten frames in `options-transport.pcap` decoded as `TCP:Raw`. `_httpv2_build` now puts `SETTINGS`, `PING` and `GOAWAY` on stream 0 and every other type on stream 1, with `PUSH_PROMISE` promising stream 2, and all ten decode as `TCP:HTTP/2`; no other generated capture changes ([#1156](https://github.com/JarryShaw/PyPCAPKit/issues/1156)). - `make_samples.py` builds fixtures from the tree it runs in, not the shared checkout ([#1343](https://github.com/JarryShaw/PyPCAPKit/issues/1343), [#1388](https://github.com/JarryShaw/PyPCAPKit/pull/1388)), and retries the Wireshark sample downloads on 429, 5xx and timeouts ([#1658](https://github.com/JarryShaw/PyPCAPKit/issues/1658), [#1659](https://github.com/JarryShaw/PyPCAPKit/pull/1659)). - the Conda Update workflow tags and uploads each bump exactly once ([#1632](https://github.com/JarryShaw/PyPCAPKit/issues/1632), [#1641](https://github.com/JarryShaw/PyPCAPKit/pull/1641)), stays green when a merge races its push ([#1649](https://github.com/JarryShaw/PyPCAPKit/issues/1649), [#1669](https://github.com/JarryShaw/PyPCAPKit/pull/1669)), and keeps the release's name and notes when it uploads ([#1657](https://github.com/JarryShaw/PyPCAPKit/issues/1657), [#1678](https://github.com/JarryShaw/PyPCAPKit/pull/1678)). Vendor Update commits the new modules a crawl adds ([#1685](https://github.com/JarryShaw/PyPCAPKit/issues/1685), [#1688](https://github.com/JarryShaw/PyPCAPKit/pull/1688)). -- package `__all__` lists match what each package holds: six disagreed with it, and `from pcapkit.protocols.data import *` raised `AttributeError` on the unbound `HOPOPT_QuickStartOption` ([#515](https://github.com/JarryShaw/PyPCAPKit/issues/515), [#527](https://github.com/JarryShaw/PyPCAPKit/pull/527)); the public attributes several packages left out are listed ([#533](https://github.com/JarryShaw/PyPCAPKit/issues/533), [#544](https://github.com/JarryShaw/PyPCAPKit/pull/544)). -- the documentation build's 36 "more than one target found" warnings, fixed where they arise: an implicit reference to `dict`, `type` or `set` no longer matches a registry member such as `TCP.dict`, an enum member's docstring holding a URL no longer gains a spurious type field, and the hand-written `TypeVar` directives take the bare name under their `.. module::` ([#1775](https://github.com/JarryShaw/PyPCAPKit/issues/1775), [#1790](https://github.com/JarryShaw/PyPCAPKit/pull/1790)). +- package `__all__` lists match what each package holds: several disagreed with it, and `from pcapkit.protocols.data import *` raised `AttributeError` on the unbound `HOPOPT_QuickStartOption` ([#515](https://github.com/JarryShaw/PyPCAPKit/issues/515), [#527](https://github.com/JarryShaw/PyPCAPKit/pull/527)); the public attributes several packages left out are listed ([#533](https://github.com/JarryShaw/PyPCAPKit/issues/533), [#544](https://github.com/JarryShaw/PyPCAPKit/pull/544)). +- the documentation build's "more than one target found" warnings, fixed where they arise: an implicit reference to `dict`, `type` or `set` no longer matches a registry member such as `TCP.dict`, an enum member's docstring holding a URL no longer gains a spurious type field, and the hand-written `TypeVar` directives take the bare name under their `.. module::` ([#1775](https://github.com/JarryShaw/PyPCAPKit/issues/1775), [#1790](https://github.com/JarryShaw/PyPCAPKit/pull/1790)). +- the Mermaid diagrams render at their natural size, shrunk only to the page width, where each was fitted into a fixed box, and their `click` targets are relative to the page, so they resolve under the site's `/PyPCAPKit/` prefix ([#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103), [#1819](https://github.com/JarryShaw/PyPCAPKit/pull/1819)). +- `pcapkit-cli`'s help strings say what each option does: `-o` describes the output rather than the input, an extension is appended only under `--auto-extension`, and each format flag names the flags that take precedence over it ([#1796](https://github.com/JarryShaw/PyPCAPKit/issues/1796), [#1810](https://github.com/JarryShaw/PyPCAPKit/pull/1810)). +- the Conda package: its vendored dependencies are the package's `dependencies` and its `all` extra, with their requirements, shipped with their data files rather than as `.py` files only ([#1814](https://github.com/JarryShaw/PyPCAPKit/issues/1814), [#1821](https://github.com/JarryShaw/PyPCAPKit/pull/1821)); the `osx-64` package is built on an Intel runner, where converting the arm64 build shipped arm64 binaries ([#1822](https://github.com/JarryShaw/PyPCAPKit/issues/1822), [#1825](https://github.com/JarryShaw/PyPCAPKit/pull/1825)); and on Intel macOS it pins `cryptography` 48.0.1, which has a wheel for that platform where the newer pin has none ([#1827](https://github.com/JarryShaw/PyPCAPKit/issues/1827), [#1828](https://github.com/JarryShaw/PyPCAPKit/pull/1828)). --- diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 3939825dc..eb6021cfc 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -37,11 +37,11 @@ All notable changes to PyPCAPKit are recorded here. carry something real, it is called out. Releases are listed newest first. The repository root's :file:`CHANGELOG.md` -carries only the version being released; this page is the whole history. +carries only the version being released; this page is the whole recorded history. * **1.5.0** (unreleased) -- The largest release since 1.0, and the first - recorded here as it happened rather than reconstructed: three more - extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, + recorded here as it happened rather than reconstructed: more extraction + engines, ESP with payload decryption, SCTP and NGAP over SCTP, and a logger that no longer hijacks the consumer's. :doc:`Full changelog `. * **1.4.1** (2026-08-22) -- Release engineering only -- no library behaviour @@ -78,12 +78,11 @@ carries only the version being released; this page is the whole history. day earlier in 1.2.0. :doc:`Full changelog `. * **1.2.0** (2023-08-26) -- Added streaming input through ``SeekableReader``, so a capture can be read from ``stdin`` or a pipe - (:issue:`156`), and fixed reassembly, broken since 0.16.3 (:issue:`155`). :doc:`Full - changelog `. -* **1.1.1** (2023-08-15) -- Fixed ``AppType`` enumeration lookups, - ``typing.TypeAlias`` compatibility on older interpreters, the flow-tracing - output file name, and ``Schema`` dumping through a custom dumper. :doc:`Full - changelog `. + (:issue:`156`). :doc:`Full changelog `. +* **1.1.1** (2023-08-15) -- Fixed reassembly, broken since 0.16.3 (:issue:`155`), + ``AppType`` enumeration lookups, ``typing.TypeAlias`` compatibility on older + interpreters, the flow-tracing output file name, and ``Schema`` dumping through + a custom dumper. :doc:`Full changelog `. * **1.1.0** (2023-07-09) -- Rebuilt the schema and field classes on metaclasses, with ``EnumSchema`` for enum-dispatched schemas, and revised the IPv4, HOPOPT, IPv6-Opts, IPv6-Route, HIP, MH, TCP, HTTP/2 and PCAP-NG @@ -101,7 +100,8 @@ carries only the version being released; this page is the whole history. * **1.0.0** (2023-05-09) -- The 1.0 rewrite: parsing and construction became one declarative schema and field definition per protocol, the extraction backend became pluggable (with PCAP-NG support and the DPKT, Scapy and - PyShark engines), and roughly 55,000 lines changed across 617 files. + PyShark engines), and roughly 55,000 lines added and 15,000 removed across + 617 files. :doc:`Full changelog `. * **0.16.3** (2022-10-31) -- Fixed import crashes on some interpreters (:issue:`114`, :issue:`116`), the reassembly property caches, and README rendering for PyPI; the diff --git a/docs/source/changelog/0.16.0.rst b/docs/source/changelog/0.16.0.rst index 9ae00bf9b..74cd90c0b 100644 --- a/docs/source/changelog/0.16.0.rst +++ b/docs/source/changelog/0.16.0.rst @@ -12,8 +12,9 @@ A project-wide revision, and the last release of the pre-1.0 design. * **Added** -- ``pcapkit.foundation.registry``, so protocols can be subscribed by name; ``Protocol.analyze`` for analysing a payload directly, used by the reassembly classes; ``pcapkit.corekit.multidict``, adapted from Werkzeug. -* **Changed** -- reassembly became a package (IP, IPv4, IPv6, TCP) with revised - data models; PCAP and the auxiliary protocols moved under ``pcapkit.misc``. +* **Changed** -- reassembly moved from ``pcapkit.reassembly`` into + ``pcapkit.foundation.reassembly``, with revised data models; PCAP and the + auxiliary protocols moved under ``pcapkit.protocols.misc``. * **Removed** -- the ``validators`` module, several decorators from ``pcapkit.utilities``, and multiprocessing support in ``Extractor``. * **Fixed** -- the DPKT toolkit (:issue:`101`), IP reassembly, and missing ``id`` diff --git a/docs/source/changelog/1.0.0.rst b/docs/source/changelog/1.0.0.rst index e4562011d..710b5d4dd 100644 --- a/docs/source/changelog/1.0.0.rst +++ b/docs/source/changelog/1.0.0.rst @@ -3,7 +3,7 @@ The 1.0 rewrite: parsing and construction are one declarative definition per protocol, and the extraction backend became pluggable. Roughly 55,000 lines -changed across 617 files, released after 21 beta builds and a release candidate. +added and 15,000 removed across 617 files, released after 21 beta builds and a release candidate. * **Added** -- the schema and field layer, ``pcapkit.protocols.schema`` and ``pcapkit.corekit.fields``. A protocol declares its wire format once, as diff --git a/docs/source/changelog/1.1.1.rst b/docs/source/changelog/1.1.1.rst index a405ccfac..795d48596 100644 --- a/docs/source/changelog/1.1.1.rst +++ b/docs/source/changelog/1.1.1.rst @@ -3,6 +3,12 @@ **Fixed** -- ``AppType`` enumeration lookups; ``typing.TypeAlias`` compatibility on older interpreters; the output file name chosen by flow tracing; dumping of -``Schema`` objects through a custom dumper. +``Schema`` objects through a custom dumper; reassembly, broken since 0.16.3 +(:issue:`155`). ``Extractor`` now raises an explicit unsupported-call error, and +logs a notice, when reassembly or flow tracing is requested where it cannot be +provided. -*Also released:* ``1.1.1.post1``, ``1.1.1.post2`` (through 2023-08-23). +*Also released:* ``1.1.1.post1``, ``1.1.1.post2`` (through 2023-08-23); +``post2`` renamed the ``strict`` argument of ``Extractor`` and ``extract()`` to +``reasm_strict``, inserting ``reasm_store`` after it, and ``TraceFlow.register`` +to ``register_dumper``. diff --git a/docs/source/changelog/1.2.0.rst b/docs/source/changelog/1.2.0.rst index 467d026bc..64f56f930 100644 --- a/docs/source/changelog/1.2.0.rst +++ b/docs/source/changelog/1.2.0.rst @@ -4,6 +4,3 @@ * **Added** -- streaming input. ``pcapkit.corekit.io.SeekableReader`` buffers a non-seekable stream so a capture can be read from ``stdin`` or a pipe, which is what live ``tcpdump`` output needs (:issue:`156`). -* **Fixed** -- reassembly, broken since 0.16.3 (:issue:`155`). ``Extractor`` now raises - an explicit unsupported-call error, and logs a notice, when reassembly or flow - tracing is requested where it cannot be provided. diff --git a/docs/source/changelog/1.3.0.rst b/docs/source/changelog/1.3.0.rst index d6e2b1261..09d140e87 100644 --- a/docs/source/changelog/1.3.0.rst +++ b/docs/source/changelog/1.3.0.rst @@ -10,5 +10,7 @@ A redesign of the extension points, on metaclasses. class that are imported lazily on first use rather than at registration; the registry functions accept it throughout. ``VersionInfo`` gained a ``version`` property. -* **Changed** -- ``_Engine`` renamed ``EngineBase``; the ``beholder`` decorator - logs the error it swallows under devmode and verbose mode. +* **Changed** -- what ``Engine`` defined moves into a new base, ``EngineBase``, + and ``Engine`` stays the class to subclass, registering an engine under its + ``name=`` class keyword; the ``beholder`` decorator logs the error it swallows + under devmode and verbose mode. diff --git a/docs/source/changelog/1.4.0.rst b/docs/source/changelog/1.4.0.rst index bf4b76334..06db29694 100644 --- a/docs/source/changelog/1.4.0.rst +++ b/docs/source/changelog/1.4.0.rst @@ -8,7 +8,10 @@ a test suite. the extraction engines, reassembly and flow tracing, the protocol schemas, the CLI and the utility modules. * **Fixed** -- PCAP-NG option dispatch regressions; the fallback name generated - by the ``linktype`` vendor crawler. + by the ``linktype`` vendor crawler; TCP option construction, where building a + Window Scale or any Multipath TCP option, or passing a prebuilt option schema, + raised, and the TCP Fast Open cookie's length check accepted lengths + :rfc:`7413` rules out. * **Changed** -- CI tests current Python versions again; every GitHub Actions job carries an explicit permissions block; the conda and cron release tooling was repaired, and Anaconda uploads use the official upload action. diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index 038afc553..467ea0db4 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -2,10 +2,10 @@ =================== The largest release since 1.0, and the first recorded here as it happened rather -than reconstructed. Three more extraction engines, ESP with payload decryption, +than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, -and a defect programme run through the issue tracker, citing 685 issues and pull -requests from :pr:`340` to :pr:`1791`. +and a defect programme run through the issue tracker, citing issues and pull +requests from :pr:`340` to :pr:`1828`. Preceded by ``1.5.0a1`` (2026-09-15), ``1.5.0b1`` to ``1.5.0b3`` (all 2026-09-19) and ``1.5.0b4`` to ``1.5.0b12`` (2026-09-20 to 2026-10-13), dated in UTC and all @@ -29,14 +29,14 @@ Changed ``SCTP`` and ``DCCP`` -- because ``aenum`` refuses to subclass an enumeration that has members. ``isinstance(TCP.http, AppType)`` still holds and ``AppType.get(port, proto=...)`` proxies to the registry, so nothing under - ``pcapkit.protocols`` changes. **The 1,004 rows IANA assigns no port, and the 704 it + ``pcapkit.protocols`` changes. **The rows IANA assigns no port, and those it assigns no transport protocol, stop being members**; they become list-table docstrings on the registry that owns them. ``__members_proto__`` is now a - per-registry ``MultiDict`` keyed on port, so all three services IANA registers on - port 80 survive. ``get()`` answers with the canonical service, curated from + per-registry ``MultiDict`` keyed on port, so every service IANA registers on + port 80 survives. ``get()`` answers with the canonical service, curated from ``/etc/services`` because IANA names no precedence, and ``get_all()`` reaches the aliases (:pr:`754`). -* the six ``%``-formatted ``raise ValueError(...)`` calls in the generated +* the ``%``-formatted ``raise ValueError(...)`` calls in the generated ``pcapkit.const.reg.apptype.apptype``, left by :issue:`759`, are f-strings (:issue:`792`). The remainder followed: three ``_missing_`` guards (``pcapkit.vendor.tcp.flags``, ``.ftp.command``, ``.http.method``), ``AppType``'s @@ -46,16 +46,16 @@ Changed ``http.method``, whose ``__repr__`` still used ``%``. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows (:issue:`798`). -* the three ``__repr__`` methods :issue:`798` left on ``%`` - (``pcapkit/const/ftp/command.py:40,119``, ``pcapkit/const/http/method.py:42``) are +* the ``__repr__`` methods :issue:`798` left on ``%`` + (in ``pcapkit/const/ftp/command.py`` and ``pcapkit/const/http/method.py``) are f-strings, so the disable drops from both files and both ``pcapkit/vendor/`` templates. The other bespoke templates (``{const,vendor}/ftp/return_code.py``, ``{const,vendor}/http/status_code.py``, ``vendor/http/frame.py``) and ``vendor/default.py`` still carry it, so :issue:`804`'s claim that nothing in ``pcapkit/{const,vendor}/{ftp,http}/`` needs it holds for this pair only; ``repr()`` output is unchanged (:issue:`804`). -* ``test_const_enum_no_mint.py``'s ``is_manufactured()`` exemption for :pr:`878`'s - (above) 53 preserved hex-suffixed names moves from a per-*file* to a +* ``test_const_enum_no_mint.py``'s ``is_manufactured()`` exemption for + :pr:`878`'s (above) preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file ``HEX_SUFFIXED_NAME_EXEMPT_PATHS`` was shape-blind: rewriting a branch to ``'Xyplex_%d' % value`` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check @@ -93,8 +93,9 @@ Fixed the only value :rfc:`2113` defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPX ``Socket(0)`` is that protocol's own default, so ``bytes(IPX(...))`` - crashed on its own defaults; and two FTP ``_missing_`` overrides were plain methods - rather than classmethods, so every unregistered value raised ``TypeError`` instead + crashed on its own defaults; and the FTP ``_missing_`` overrides were plain + methods rather than classmethods, so every unregistered value raised ``TypeError`` + instead of extending the enumeration (:issue:`492`, :pr:`503`). * the string-keyed ``get()`` in ``pcapkit.const.ftp.command`` and ``pcapkit.const.http.method`` tested membership with the raw key but @@ -115,12 +116,13 @@ Fixed ``0``. The integer path now consults ``default`` before letting the error escape. ``-1``, the placeholder the generated signature already carried, still separates "no default" from "a default"; :pr:`859` replaced it with an identity sentinel once the - shared base :pr:`858` (below) put ``get()`` behind one call site. The sweep covers - 110 of the 118 integer registries, not the three :issue:`584` named; ``pcapng`` + shared base :pr:`858` (below) put ``get()`` behind one call site. The sweep covers the + integer registries generated from the shared template, not only those :issue:`584` + named; ``pcapng`` ``OptionType`` and ``reg`` ``AppType`` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix (:issue:`584`). -* ``_missing_`` in the four Mobility Header flag enumerations (``BindingACKFlag``, +* ``_missing_`` in the Mobility Header flag enumerations (``BindingACKFlag``, ``BindingUpdateFlag``, ``HandoverACKFlag``, ``HandoverInitiateFlag``) ended in ``return cls(value)``, the constructor that had just failed, so every in-range value that is not already a member re-entered ``_missing_`` unbounded and raised @@ -128,39 +130,39 @@ Fixed ``BindingACKFlag(0x06)``. Defining ``_missing_`` at all caused it, by shadowing the ``aenum`` ``Flag`` machinery that resolves exactly those values; ``pcapkit/const/tcp/flags.py`` had none and so never had the defect (it has one now, - from :issue:`647` below, with the same tail). All four now end in + from :issue:`647` below, with the same tail). Each now ends in ``return super()._missing_(value)``, as ``pcapkit/vendor/default.py`` emits and most - const modules (:issue:`647`'s two new ones included) already did, so ``F(0)`` is an + const modules (:issue:`647`'s new ones included) already did, so ``F(0)`` is an empty flag and ``BindingACKFlag(0x06)`` is ``S|D``. Minting through ``extend_enum`` was rejected for a flag type: naming ``0x06`` ``Unassigned_0x06`` would hide the composite and add a member per bit pattern. The range guard is untouched (:issue:`623`). -* six of the 123 constant registries rejected an invalid value differently from the - built-in ``enum``, and three did not reject it at all. ``Flags``, +* constant registries that rejected an invalid value differently from + the built-in ``enum``, or did not reject it at all. ``Flags``, ``ftp.command.CommandType`` and ``reg.apptype.TransportProtocol`` are ``IntFlag`` types with no ``_missing_``, so ``aenum`` composed a pseudo-member for any integer: ``Flags(-1)`` returned ``65520`` (every TCP flag set, from a value no 16-bit field can hold) and ``Flags(-65536)`` none. ``ftp.command.Command``, ``FEATCode`` and ``http.method.Method`` are ``StrEnum`` types whose ``_missing_`` called ``value.upper()`` before checking the type, so an integer raised ``AttributeError``. - All six now raise a bare ``ValueError``, as ``enum.IntEnum`` does and 113 of the 117 - modules already did. :issue:`647`'s proposal to replace those 113 with + These now raise a bare ``ValueError``, as ``enum.IntEnum`` does and the other + modules already did. :issue:`647`'s proposal to replace those with ``pcapkit.utilities.exceptions.EnumError`` was deliberately rejected: ``EnumError`` is ``(BaseError, TypeError)``, not a ``ValueError``, so it would diverge from the - built-in, walk past the ``except ValueError`` in all 113 generated ``get()`` bodies + built-in, walk past the ``except ValueError`` in every generated ``get()`` body (silently undoing :issue:`584`), and log CRITICAL from ``BaseError.__init__`` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and ``extend_enum`` rather than raising, so ``Method('FROBNICATE')``, ``ProtectionAuthority(1 << 70)`` and ``AppType.get(65000, proto=tcp)`` still register; the :issue:`775` and - :issue:`860` entries below record how all three came to return an unregistered - member instead. The three flag guards end in + :issue:`860` entries below record how each came to return an + unregistered member instead. The flag guards end in ``return super()._missing_(value)``, so in-range composites still decompose and :issue:`623` stays fixed; ``Flags`` accepts ``0 <= value <= 0xFFFF``. A new - ``tests/const/test_const_enum_builtin_parity.py`` asserts that every one of the 123 - registries raises the same exception type as a stdlib ``enum.IntEnum``; it keys on + ``tests/const/test_const_enum_builtin_parity.py`` asserts that every + registry raises the same exception type as a stdlib ``enum.IntEnum``; it keys on ``aenum.Enum`` because an ``IntFlag`` is not an ``IntEnum`` subclass, which is how - three of the six escaped the existing sweeps (:issue:`647`). + the ``IntFlag`` ones escaped the existing sweeps (:issue:`647`). * **a breaking change to** ``AppType.get``: an out-of-range port is refused rather than minted. Its ``except ValueError`` caught the range rejection from ``_missing_`` and minted regardless, so ``AppType.get(-1, proto='tcp')`` @@ -170,12 +172,12 @@ Fixed an unregistered member. ``Transport._make_port`` **passes an unvalidated** ``int``, **so** ``TCP.make(srcport=99999)`` **now raises where it minted junk.** Separately, the crawler rendered one ``_missing_`` branch per - registry row, so the two spans IANA registers on more than one transport + registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: ``AppType.get(6010, proto='udp')`` gave a member whose ``proto`` read ``tcp``, ``6666/udp`` answered ``ircu`` rather than ``reserved``, and ``proto='sctp'`` minted ``x11`` where IANA assigns nothing. A branch naming a transport now - tests ``cls.__transport__``; the 762 naming none are unchanged (:pr:`764`). + tests ``cls.__transport__``; the branches naming none are unchanged (:pr:`764`). * **a breaking change to** ``AppType._dispatch``: it resolved a ``proto`` naming several transports by its lowest set bit, and ``tcp`` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a @@ -183,31 +185,32 @@ Fixed ``TransportProtocol`` was still an ``aenum.IntFlag`` and a member's ``proto`` carried the whole set IANA assigned the service, so ``tcp | udp`` was an ordinary value to pass back in; the ``AppType.proto`` and ``TransportProtocol`` entries above - record how :issue:`806` and :issue:`808` later retired both. Over every - multi-transport member this misrouted two: port 888 (``accessbuilder`` - resolved to TCP's ``cddbp``) and port 999 (``puprouter`` resolved to TCP's - ``garcon``). It now raises ``ProtocolError``, a ``ValueError`` subclass, so the + record how :issue:`806` and :issue:`808` later retired both. Over + every multi-transport member this misrouted port 888 (``accessbuilder`` resolved to + TCP's ``cddbp``) and port 999 (``puprouter`` resolved to TCP's ``garcon``). It now + raises ``ProtocolError``, a ``ValueError`` subclass, so the documented contract holds. No library caller passes a composite (:issue:`759`). * ``get()``'s string-key miss and every generated registry's ``_missing_`` bounded-range branch called ``aenum.extend_enum()`` on an unrecognised value, so - e.g. ``Hardware(40)`` minted a permanent member. Fixed at the two shared sites in + e.g. ``Hardware(40)`` minted a permanent member. Fixed at the shared sites in ``pcapkit/vendor/default.py``, plus a new ``register(value, name)`` base classmethod - for the caller-named path that still mints. All 105 generated registries are - updated; the non-minting ``get()`` miss applies to all 105, but the ``_missing_`` - fix changes behaviour only for the **21** with a bounded-range ``extend_enum`` - branch. ``pcapkit/toolkit/scapy.py`` and ``pcapkit/toolkit/pyshark.py`` fed + for the caller-named path that still mints. Every generated registry is + updated; the non-minting ``get()`` miss applies to all of them, but the + ``_missing_`` fix changes behaviour only for **those** with a bounded-range + ``extend_enum`` branch. ``pcapkit/toolkit/scapy.py`` and + ``pcapkit/toolkit/pyshark.py`` fed ``get()`` a string key with no default and now raise ``MissingKeyError`` instead of - receiving a placeholder; four reader sites in ``hopopt.py``/``ipv6_opts.py`` convert + receiving a placeholder; the reader sites in ``hopopt.py``/``ipv6_opts.py`` convert the ``KeyError`` into ``ProtocolError``. Riding along: - ``pcapkit/toolkit/pyshark.py`` gains a two-entry ``FILTER_NAME_TO_LINKTYPE`` + ``pcapkit/toolkit/pyshark.py`` gains a ``FILTER_NAME_TO_LINKTYPE`` map (``eth`` to ``ETHERNET``, ``tr`` to ``IEEE802_5``), consulted ahead of ``LinkType.get`` since pyshark reports a filter name, not a DLT name. The other minting call sites in ``_missing_``, ``get`` and the hand-written ``mh.py`` and - ``ngap.py`` were deferred; the 106 ``register``/``register_alias`` sites are not + ``ngap.py`` were deferred; the ``register``/``register_alias`` sites are not deferred but the by-design caller-named path (:pr:`838`). -* **a breaking change to** ``pcapkit.const.ipx.socket.Socket``'s ``_missing_``: three - of its five archived range branches were strict subsets of a range tested earlier -- +* **a breaking change to** ``pcapkit.const.ipx.socket.Socket``'s ``_missing_``: archived + range branches that were strict subsets of a range tested earlier -- ``(0x0020, 0x003F)`` after ``(0x0001, 0x0BB8)``, and ``(0x4000, 0x4FFF)`` and ``(0x8000, 0xFFFF)`` after ``(0x0BB9, 0xFFFF)`` -- so they never fired. The generator ``pcapkit/vendor/ipx/socket.py`` now lists each subset range ahead of the @@ -227,17 +230,17 @@ Fixed (:pr:`848`). The rule sinks only a row whose value another row also holds, counting values inside ranges, so a current row that merely mentions a legacy name stays first (:issue:`852`, :pr:`854`). -* **a breaking change to** 82 more const registries' ``_missing_``: applies the +* **a breaking change to** more const registries' ``_missing_``: applies the owner's mint/unmint ruling (settled on :pr:`847`, confirmed on :issue:`775`: a final, concrete assigned name mints, and a notation for readers does not) across - the 89 ``_missing_`` files still calling ``extend_enum``. 80 registries convert - wholly (164 branches), each minting a bare status word (``Unassigned``, + the ``_missing_`` files still calling ``extend_enum``. Most convert wholly, each + minting a bare status word (``Unassigned``, ``Reserved [...]``, ``Reserved for Private/Experimental Use``, ``Unspecified in the IANA registry``, ``Deprecated``), which the ruling treats as a - notation rather than an assignment. Two are mixed: ``EtherType``'s - ``DEC Unassigned`` (3 branches) and "Old Xerox Experimental values. Invalid as an - Ethertype since 1983." (1) convert, while 46 vendor-company names across 50 range - blocks (Xyplex, Motorola, Walker Richer & Quinn and 43 more) keep minting, since a + notation rather than an assignment. Some are mixed: + ``EtherType``'s ``DEC Unassigned`` and "Old Xerox Experimental values. Invalid as an + Ethertype since 1983." branches convert, while vendor-company names + (Xyplex, Motorola, Walker Richer & Quinn and others) keep minting, since a proprietary protocol has no public name beyond the company's; ``IEEE802_3_Length_Field`` and ``Berkeley_Trailer_encap_IP`` also keep minting but are not company names. ``Socket`` (ipx)'s allocation-policy names (``Experimental``, @@ -262,7 +265,7 @@ Fixed fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only containment among the 56 ranges. Closes :issue:`862` (:pr:`865`). * **a breaking change to** ``Method``: ``__new__`` called ``str.__new__(cls)`` with no - argument, so all 40 registered members' ``str`` payload was empty regardless of + argument, so every registered member's ``str`` payload was empty regardless of value -- ``str(Method.GET) == ''`` and ``Method.GET == 'GET'`` was ``False``; ``.value`` was always correct. :pr:`869` (above) had fixed the *unregistered* path only, leaving ``str(Method('frob')) == 'frob'`` beside ``str(Method.GET) == ''``. @@ -272,20 +275,20 @@ Fixed store a display string). Consequences, all corrections: ``Method.GET == 'GET'`` is ``True``; ``bool(Method.GET)`` flips ``False`` to ``True``; ``json.dumps(Method.GET)`` emits ``"GET"`` not ``""``; ``sorted()`` is lexicographic - rather than input-order-stable; and ``{m: ... for m in Method}`` gains 40 entries - where it had 1, because all 40 members previously hashed as ``''`` and compared + rather than input-order-stable; and ``{m: ... for m in Method}`` gains an entry per + member where it had one, because every member previously hashed as ``''`` and compared equal. Nothing under ``pcapkit/protocols`` relied on the old behaviour. Closes :issue:`870` (:pr:`871`). -* **a breaking change to** ``TransportProtocol`` and ``AppType``: step 2 of - :issue:`860`, PR 2 of 2 (the 8 non-``AppType`` bespoke registries were :pr:`869`, - above). ``AppType`` and its four transport subclasses move onto ``EnumRegistry`` and - both mint sites -- ``_missing_``'s 766 range branches and ``get()``'s second mint +* **a breaking change to** ``TransportProtocol`` and ``AppType``: step 2 + of :issue:`860`, the ``AppType`` half (the other bespoke registries were :pr:`869`, + above). ``AppType`` and its transport subclasses move onto ``EnumRegistry`` and + both mint sites -- ``_missing_``'s range branches and ``get()``'s second mint (``PORT_{port}_{transport}`` -> ``'unknown'``) -- now build through a new ``_unregistered_member`` override that reconstructs ``svc``/``port``/``proto``, per the owner's ruling that only IANA-registered values are legitimate and that - ``get()`` lacks the information to create new ones. All 766+1 branches convert, - including 8 naming a real service assigned to a whole port span (7 distinct names: - ``x11`` twice, ``active-net``, ``satvid-datalnk``, ``vrml-multi-use``, ``ircu``, + ``get()`` lacks the information to create new ones. Every branch converts, + including those naming a real service assigned to a whole port span (``x11``, + ``active-net``, ``satvid-datalnk``, ``vrml-multi-use``, ``ircu``, ``swx``, ``flex-lm``); unlike ``FEATCode``'s fix (:pr:`869`, above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. ``AppType`` gains a working ``register()``; the @@ -300,20 +303,20 @@ Fixed ``8`` used to mint ``DCCP.PORT_80_dccp`` and now raises ``ValueError``. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path (:pr:`874`). -* closes the remaining scope of :issue:`775`: ``EtherType`` (52 branches) and - ``Socket`` (1), the last registries whose ``_missing_`` minted a permanent member, +* closes the remaining scope of :issue:`775`: ``EtherType`` and + ``Socket``, the last registries whose ``_missing_`` minted a permanent member, now return a non-registering ``cls._unregistered_member(value, name)``. The issue's - AST census of ``extend_enum`` in ``_missing_``/``get`` goes from 54 sites to the - single ``CGAType`` carve-out (its mint is not an IANA-style range), from the - original 1,169. ``EtherType(0x0888)`` used to grow ``__members__`` from 160 to 161 + AST census of ``extend_enum`` in ``_missing_``/``get`` is down to the + single ``CGAType`` carve-out (its mint is not an IANA-style range). + ``EtherType(0x0888)`` used to add a member to ``__members__`` and return the identical object on a repeat; it now returns ```` with ``__members__`` unchanged, absent from ``_value2member_map_``, equal but not identical across lookups -- likewise ``Socket(0x0010)``. The regenerated ``EtherType`` was not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape ``test_const_enum_no_mint.py``'s - ``is_manufactured()`` sweep forbids; rather than weaken the predicate, a narrow - exemption covers exactly these two files, since the collision the check guards + ``is_manufactured()`` sweep forbids; rather than weaken the predicate, a + narrow exemption covers exactly these files, since the collision the check guards against cannot occur once ``_unregistered_member`` never registers (:pr:`878`). * ``OptionType.register()`` and ``Command.register()`` built members that ``get()`` could not find again, and ``OptionType.register_alias()`` raised @@ -335,6 +338,18 @@ Fixed * ``FEATCode.get`` matches case-insensitively, as :rfc:`5797#section-2` defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold (:issue:`903`, :pr:`913`). +* ``register`` refuses what ``get`` refuses. ``TCP.register(70000, 'x')`` and + ``Hardware.register(70000, 'x')`` minted a member that ``get`` then returned, + because no registry overrode the ``_validate_value`` hook ``register`` calls. Each + generated registry, ``AppType`` included, now overrides it with the check its + ``_missing_`` makes, raising ``EnumValueError``. ``get(value, default)`` on a value + the hook refuses still returns ``default``, but now also logs the + ``EnumValueError`` at ``CRITICAL``, as any loud pcapkit error is (:issue:`1792`, + :pr:`1818`, :issue:`1816`, :pr:`1823`). +* ``Command.get`` refuses a key that is not a ``str`` with ``EnumValueError``, as + ``Method.get`` does. ``get(b'\x01')`` minted a member keyed by ``bytes``, and + ``get(-1)`` raised ``AttributeError`` whatever its ``default`` (:issue:`1824`, + :pr:`1826`). pcapkit.corekit --------------- @@ -346,7 +361,7 @@ Added boundary in its own method, since the defect is a pattern and a single case would pass against a fix that special-cased the reported width (:issue:`599`). * ``FieldBaseShortReadPaddingSideTests``, covering both byte orders at 2, 4 and 8 - octets and every truncation point, including the two figures :issue:`604` reports, + octets and every truncation point, including the figures :issue:`604` reports, plus guard rails for a full read and for a read against an empty buffer, the :issue:`431` behaviour the option and list loops depend on (:issue:`604`). @@ -467,7 +482,7 @@ Fixed * which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised ``ProtocolError`` as documented, but a process that had popped ``pcapkit.corekit.fields.misc`` from ``sys.modules`` - (as the ``#439`` ABC-cache regression tests do in every ``setUp``/``tearDown``) + (as the :issue:`439` ABC-cache regression tests do in every ``setUp``/``tearDown``) raised ``FieldValueError`` from a different layer, before the documented check (:issue:`525`). ``ListField.unpack`` resolved ``SchemaField`` through a function-local import re-run on every call; a module reimported mid-process @@ -501,7 +516,7 @@ Fixed :issue:`591`'s suite, whose values all had byte-aligned bit lengths, missed it; :issue:`591`'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so - an unresolved signed field still cannot pack ``128``. The same typo at the two ILNP + an unresolved signed field still cannot pack ``128``. The same typo in the ILNP nonce option builders is a separate change (:issue:`599`). * ``FieldBase.unpack`` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated @@ -513,7 +528,7 @@ Fixed read is untouched (:issue:`604`). * ``SeekableReader.truncate`` put its padding where the reader's bookkeeping says the content is, and ``read`` then returned one octet fewer than the buffer held. Same - family as :issue:`604`, but two defects. The buffer keeps content at + family as :issue:`604`, but separate defects. The buffer keeps content at ``[0:_buffer_cur]`` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; ``temp[-size:]`` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and @@ -522,7 +537,7 @@ Fixed padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: ``read(4)``, ``truncate(8)``, ``seek(0)``, ``read(8)`` over ``b'abcde'`` returned ``b'\x00\x00\x00e'`` and now returns - ``b'abcde'``. Three more contract faults are fixed with it: the position was reset + ``b'abcde'``. Further contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted ``size`` resized to ``0`` rather than the current position; and ``_buffer_cur`` was left addressing octets a reduced buffer no longer had, so the next read raised @@ -542,7 +557,8 @@ Fixed length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now - synthesises none, with every frame and option still parsed. The bound is taken one layer + synthesises none, with every frame and option still parsed. The bound is taken one + layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, ``captured_len`` and its padding, and an option @@ -550,14 +566,14 @@ Fixed capture says so through ``captured_len`` and leaves its options whole, so it never trips this -- which the :pr:`571` ``len(buffer) < length`` rejection could not distinguish, and why it was declined. A new ``bounded_option`` clamps the payload to - the octets the area has left and warns with ``SchemaWarning``; it is applied to all - fifteen variable-width option and record payloads in the PCAP-NG schema, and + the octets the area has left and warns with ``SchemaWarning``; it is applied to + every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A second, ``bounded_area``, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no - warning. Both are no-ops on a well-formed block. The five non-packet blocks' option + warning. Both are no-ops on a well-formed block. The non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is :issue:`678`. Clamping rather than refusing keeps the :issue:`431` accommodation, since a PCAP-NG block read has no catch point above @@ -589,26 +605,27 @@ Fixed converts the first six registries with no bespoke ``__new__`` (``ipv6.extension_header.ExtensionHeader``, ``tcp.flags.Flags`` and four ``mh.*_flag`` modules), and serves ``IntEnum``, ``IntFlag`` and ``StrEnum`` alike. - The guard reaches only those six: the other 105 const registries still carry the old - generator's bare ``extend_enum(cls, name, value)``, so ``pcapkit.const`` ships two - contradictory ``register`` contracts until :pr:`858` (below). The six also gain the + The guard reaches only those: the other const registries still carry the old + generator's bare ``extend_enum(cls, name, value)``, so ``pcapkit.const`` ships + contradictory ``register`` contracts until :pr:`858` (below). These also gain the generated template's ``KeyError`` catching on a missing name, so - ``Flags.get('NOPE', 2048)`` returns ``Flags.ACK`` rather than raising; and a key that is neither - ``int`` nor ``str``, which raised ``TypeError`` on the five ``IntFlag`` registries + ``Flags.get('NOPE', 2048)`` returns ``Flags.ACK`` rather than raising; and a key + that is neither + ``int`` nor ``str``, which raised ``TypeError`` on the ``IntFlag`` registries (``Flags.get(3.5)``), now raises ``ValueError``, tried as a value. On - ``ExtensionHeader`` it raised ``KeyError``, the shift :pr:`858`'s entry attributes - to the other 105 (:pr:`855`). -* **a breaking change to** ``EnumRegistry.register()``, extended: the 105 const + ``ExtensionHeader`` it raised ``KeyError``, the shift :pr:`858`'s entry attributes to + the other registries (:pr:`855`). +* **a breaking change to** ``EnumRegistry.register()``, extended: the const registries still on the generated template's bare ``extend_enum`` now inherit the guard :pr:`855` (above) added, so ``TransType.register(6, 'TOTALLY_NEW_NAME')`` raises ``ValueError`` naming the value and the member that holds it and pointing at ``register_alias()``, where it used to mint and raise nothing. ``pcapkit/vendor/default.py``'s template now emits ``class {NAME}(EnumRegistry, IntEnum)`` and drops its own - ``get``/``register``/``_unregistered_member``. The 10 modules with a bespoke + ``get``/``register``/``_unregistered_member``. The modules with a bespoke ``__new__`` (``ftp/command``, ``ftp/return_code``, ``http/method``, - ``http/status_code``, ``pcapng/option_type`` and ``reg/apptype``) are left alone, so - **111 of the 127 classes** inherit ``EnumRegistry``, up from 6 (:pr:`855`). + ``http/status_code``, ``pcapng/option_type`` and ``reg/apptype`` among them) are + left alone, so **every other class** inherits ``EnumRegistry`` (:pr:`855`). ``get()``'s dispatch shifts only for a key that is neither ``int`` nor ``str`` (``KeyError``, tried as a name; now ``ValueError``, tried as a value), a key type nothing in the library passes. @@ -631,12 +648,13 @@ Fixed raised ``KeyError`` now resolves, and no caller that got a correct result gets a different one (:pr:`863`). * **a breaking change to** ``EnumRegistry.get()``: its docstring claimed "It never - mints", but both ``cls(default)`` call sites reached ``_missing_`` for a ``default`` + mints", but each ``cls(default)`` call site reached ``_missing_`` for a ``default`` inside a still-minting registry's range, growing the registry as a side effect of resolving ``default`` rather than ``key``. Per the owner's ruling that only ``register()`` mints, and ``get()`` only through the ranges ``_missing_`` mints, - both sites now resolve ``default`` through a plain ``_value2member_map_`` - lookup; ``key`` resolution is unchanged, so ``EtherType.get(0x0888)`` still mints + each site now resolves ``default`` through a plain + ``_value2member_map_`` lookup; ``key`` resolution is unchanged, so + ``EtherType.get(0x0888)`` still mints ``Xyplex_0x0888`` through its own ``_missing_``, as :issue:`775`'s deliberate exception permits. A ``default`` naming no registered member now falls through to the error ``key`` itself would have raised, rather than a fresh error about the @@ -652,8 +670,8 @@ Fixed and ``+=`` follows, since it falls back to ``__add__`` (:issue:`1099`). * a ``SchemaField`` with no declared length handed its ``-1`` placeholder to the nested schema as that schema's length, so every item of such a list warned - ``packet length < 0``: 12 ``SchemaWarning`` for a two-locator HIP ``LOCATOR_SET``, 6 - for one MH CGA Parameters item. ``SchemaField.unpack`` now passes what is left in the + ``packet length < 0``: a HIP ``LOCATOR_SET`` and an MH CGA + Parameters item among them. ``SchemaField.unpack`` now passes what is left in the buffer as a declared length -- not ``None``, which would turn a truncated item's ``FieldValueError`` into ``prepare()``'s quiet ``StreamEOFError``. Fixed-length items are unchanged; the HTTP/2 ``SETTINGS`` pair, whose size is fixed, is given it in its @@ -695,7 +713,7 @@ Fixed ``length`` and a negative ``bit_length`` (:issue:`828`, :pr:`829`, :issue:`831`, :pr:`834`). * ``ModuleDescriptor.klass`` raises ``ProtocolError`` for a class name that resolves - to nothing, or one never given, rather than a bare ``AttributeError``; the two + to nothing, or one never given, rather than a bare ``AttributeError``; the registry modules share one ``NULL`` sentinel compared by identity, where each had its own string (:issue:`832`, :issue:`833`, :pr:`835`). * ``PortEnumField`` given a ``proto`` with no ``.name`` -- a string such as @@ -726,12 +744,12 @@ Fixed (``Flags(1)`` and ``Flags(9)`` are equally nameless). Surfaced by :pr:`634`, which made a flagless TCP segment reach this path (:issue:`648`). * ``tests/dumpkit/test_nameless_enum_rendering_unit.py`` (added by :pr:`670`) carried - two probes assuming every flag-enum registry is 16 bits wide: + probes assuming every flag-enum registry is 16 bits wide: ``test_scalar_return_renders_a_nameless_member_as_its_value`` used one fixed tuple ending in ``65536``, and ``test_no_flag_registry_renders_the_literal_none`` swept every registry but only against ``registry(0)``. Widths differ: ``ftp.command.CommandType`` 3 bits, ``reg.apptype.TransportProtocol`` 4 (computed at - runtime), the three ``mh`` ack/initiate flags 8, and + runtime), the ``mh`` ack/initiate flags 8, and ``mh.binding_update_flag.BindingUpdateFlag`` and ``tcp.flags.Flags`` 16. ``tcp.flags.Flags(65536)`` correctly raises (``0 <= value <= 0xFFFF``), so the sweep was failing on correct behaviour. The probes now derive each registry's bound from @@ -753,8 +771,10 @@ Fixed not parse (``Expecting ':' delimiter``): its TLS key log is keyed by a ``bytes`` client random whose repr holds ``"`` and ``\``. ``escape_key`` in ``pcapkit.dumpkit.common`` now JSON-escapes keys for JSON output, as :issue:`772` made it XML-escape them for - plist, and the JSON reports of all 23 ``examples/captures/`` files now parse. The - upstream defect, ``JarryShaw/DictDumper#121``, stands (:issue:`1152`). + plist, and the JSON report of every ``examples/captures/`` file now parses. The + upstream defect, ``JarryShaw/DictDumper#121``, stands (:issue:`1152`). The PCAP-NG + end-to-end test of that report no longer skips; test-only (:issue:`1795`, + :pr:`1809`). * **a breaking change to** the dump schema: the output writers dropped or corrupted what the packets carried. An ``OrderedMultiDict``, which every option list is, now dumps as a list of single-key entries in wire order @@ -776,7 +796,7 @@ pcapkit.foundation Added ~~~~~ -* three extraction engines: ``engine='pypcap'`` and ``engine='pcap_ct'``, two +* the ``engine='pypcap'`` and ``engine='pcap_ct'`` extraction engines, independent distributions of the same ``libpcap`` interface, and ``engine='pypcapfile'`` (:pr:`386`, :pr:`405`). They buy speed by doing less -- neither ``pypcap`` nor ``pcap_ct`` dissects at all, so they offer no reassembly @@ -785,8 +805,8 @@ Added both present ``pcap-ct`` wins the import and the other becomes unselectable. The missing interface constants ``PyPCAP``, ``PCAP_CT`` and ``PyPCAPFile`` are now exported alongside ``DPKT``, ``Scapy``, ``PyShark`` and ``PCAPKit`` (:pr:`412`). - That makes seven built-in engines; 3.11 is the last interpreter on which every - one can run, and even there two cannot coexist. + 3.11 is the last interpreter on which every built-in engine can run, and + even there ``pypcap`` and ``pcap-ct`` cannot coexist. * ``EngineBase.unsupported_reason``, a preflight every engine answers and ``Extractor.run`` consults before anything is imported. An engine that cannot run in the current environment now gives one warning naming the real cause (a @@ -808,8 +828,8 @@ Changed can be resolved from its enclosing packet (:pr:`404`). ``follow_tcp_stream`` dispatches on the engine type, where both branches of the old test were dead and the native adapter ran against every engine's frames (:pr:`402`). -* two reassembly and flow-tracing defaults moved (:pr:`435`), both visible to a - caller. ``Datagram.completed`` widened from ``bool`` to a ``Completion`` enumeration +* reassembly and flow-tracing defaults moved (:pr:`435`), visible to + a caller. ``Datagram.completed`` widened from ``bool`` to a ``Completion`` enumeration (``COMPLETE``, ``PARTIAL``, ``TIMEOUT``); only ``COMPLETE`` is truthy, so ``if datagram.completed:`` is unaffected but ``datagram.completed == True`` no longer holds. TCP flow tracing is **bidirectional by default**, which merges each @@ -835,11 +855,12 @@ Changed ``Reassembly`` and ``TraceFlow``, ``fmt=`` for ``Dumper``. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel ``*Base`` class under - an alias, which every built-in does (hence the public classes had **0** subclasses - against the ``*Base`` classes' 9, 5, 2 and 3). **This breaks out-of-tree code that - subclasses one of the four and relies on the derived key**; pass the keyword, or + an alias, which every built-in does (hence no public class had a subclass, while + every built-in derives from its ``*Base`` class). **This breaks out-of-tree code + that subclasses one of these classes and relies on the derived key**; pass the + keyword, or call the matching ``register_*`` function. Nothing the library ships is affected, - and the ``*Base`` classes remain importable. Two silent failures are now loud: an + and the ``*Base`` classes remain importable. Silent failures are now loud: an unrecognised class keyword raises ``UnsupportedCall`` instead of being swallowed by ``**kwargs`` (passing ``name=`` to a ``Reassembly`` subclass silently ignored the key, since ``protocol=`` is the real one), and so does ``Dumper``'s ``ext=`` without @@ -857,13 +878,15 @@ Changed now accepts only a subclass of the public ``Engine``, ``Reassembly`` or ``TraceFlow`` (:issue:`1016`). They had accepted a subclass of the matching ``*Base`` class since :issue:`513`, which was a tolerance for pcapkit's own - built-ins (all 13 derive from the base, none from the public class) rather than + built-ins (every built-in derives from the base, none from the public class) rather + than a contract: ``*Base`` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses ``EngineBase``, ``ReassemblyBase`` or ``TraceFlowBase`` directly and registers through these functions now gets ``RegistryError`` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a - registrar; ``Extractor._register_internal_engine`` and its two siblings check the + registrar; ``Extractor._register_internal_engine`` and its sibling internal + registrars check the base for internal registration, and are what the :issue:`513` regression test now exercises. ``register_protocol`` is untouched (:pr:`1020`). * extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical @@ -878,7 +901,7 @@ Changed are now declared to return ``EngineBase[_P]`` rather than ``Engine``, and the private ``_exeng`` attribute follows. The built-in engines subclass ``EngineBase`` directly, so none was ever an ``Engine``, which adds only the ``__init_subclass__`` - registration hook. Two things are visible to a type checker. Code passing the + registration hook. A type checker sees the change: Code passing the result to an ``Engine``-typed parameter must now accept ``EngineBase``; and because the old declarations were *unparameterised*, ``engine.read_frame()`` and ``record_header().read_frame()`` @@ -894,7 +917,7 @@ Changed at ``EngineBase``, which stays out of ``__all__``, as :issue:`1016` settled (:issue:`1024`). * **a breaking change to** ``register_mh_extension``: ``schema`` is keyword-only, as - in the other fourteen schema-taking registrars (:issue:`516`, :pr:`526`). + in the other schema-taking registrars (:issue:`516`, :pr:`526`). Fixed ~~~~~ @@ -916,12 +939,12 @@ Fixed :pr:`577`, :pr:`596` and :pr:`600` (:issue:`606` had only made the test immune). Both conditions now follow ownership via a single ``Extractor._owns_input``, and a finaliser releases the handle of an extraction *abandoned* before end of file - (``auto=False``, iterated part way and dropped); the two files :issue:`610` names + (``auto=False``, iterated part way and dropped); the files :issue:`610` names emit no such warning. ``__exit__`` deliberately still closes unconditionally, since a caller who scopes an ``Extractor`` with ``with`` asked for that (:issue:`610`). * ``extract(..., no_eof=True)`` never returned. End of stream was detected and - ``ExtractionWarning: EOF reached`` fired, but all three loops that handle it + ``ExtractionWarning: EOF reached`` fired, but every loop that handles it (``record_frames``, ``__next__``, ``__call__``) read ``if self._flag_n: continue``, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- @@ -941,14 +964,14 @@ Fixed end-to-end tests are bounded by a **child process** rather than ``tests._support.time_limit``, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than - a red test. Two guesses at the cause are ruled out: the parse path's two + a red test. Guesses at the cause are ruled out: the parse path's ``except Exception`` handlers are never entered during the spin, and the memory is the retry loop's ``EOF reached`` records, which the runner retains (:issue:`620`). * ``register_protocol`` displaced one protocol class with another and said nothing. The registry behind ``pcapkit.protocols.__proto__`` is keyed on - ``cls.__name__.upper()``, and three dispatchable classes are all named ``HTTP`` (the - base ``pcapkit.protocols.application.http.HTTP`` and the ``httpv1`` and ``httpv2`` - implementations), so a bare assignment decided the winner: + ``cls.__name__.upper()``, and several dispatchable classes are named ``HTTP`` + (the base ``pcapkit.protocols.application.http.HTTP`` and the ``httpv1`` and + ``httpv2`` implementations), so a bare assignment decided the winner: ``register_protocol(httpv2.HTTP)`` moved ``__proto__['HTTP']`` to the ``httpv2`` class with no exception, log line or ``RegistryWarning``, and ``ProtocolBase.expand_comp('HTTP')`` then resolved to whichever registered last. The @@ -956,27 +979,27 @@ Fixed class. The sharpest path is not a call to the registrar: ``Protocol.__init_subclass__`` registers unconditionally, so merely *defining* a subclass of the public ``Protocol`` under a name a built-in holds displaced that - built-in. A plain ``import pcapkit`` emits zero new warnings: the three built-in + built-in. A plain ``import pcapkit`` emits zero new warnings: the built-in ``HTTP`` classes derive from ``ProtocolBase``, so ``__init_subclass__`` never fires for them, and import-time seeding keys off the distinct names ``HTTP``, ``HTTPv1`` and ``HTTPv2``. The guard deliberately reads "key present and the incumbent is a - different class", not mere presence (:pr:`726` later gave all seven siblings the - same identity guard): this function is the funnel all nine wrapper registrars end - in, so registering one class under two codes (``register_tcp`` then + different class", not mere presence (:pr:`726` later gave its sibling + registrars the same identity guard): this function is the funnel every wrapper + registrar ends in, so registering one class under two codes (``register_tcp`` then ``register_udp``, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or ``Raw`` on a miss), so it belongs to the registry redesign in :issue:`514`. The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod (:issue:`675`). -* two ``#:`` autodoc comments in ``pcapkit/foundation/traceflow/traceflow.py`` named a - bare ``Type``, which Sphinx resolves against every class named ``Type`` in the - project (five) rather than ``typing.Type``, and silently linked to +* the ``#:`` autodoc comments in ``pcapkit/foundation/traceflow/traceflow.py`` that + named a bare ``Type``, which Sphinx resolves against every class named ``Type`` in + the project rather than ``typing.Type``, and silently linked to ``pcapkit.const.l2tp.type.Type``, an L2TP field-type enum; once :issue:`684` rendered ``TraceFlow._foutio``, the built docs pointed at the wrong class with no - warning. Both sites now spell it ``~typing.Type[Dumper]``, one pair of the sites - :issue:`709` lists (:issue:`709`). Four more - bare ``Type`` sites, in hand-written ``:type:`` fields under + warning. These sites now spell it ``~typing.Type[Dumper]``, among the sites + :issue:`709` lists (:issue:`709`). The other bare ``Type`` sites, in hand-written + ``:type:`` fields under ``docs/source/pcapkit/foundation/``, are fixed separately by :pr:`714` (:issue:`709`). * ``register_protocol``'s overwrite warning could claim a protocol was replaced with @@ -989,46 +1012,50 @@ Fixed reprs and, only when they coincide, appends each object's ``id()``; ``__module__``/``__qualname__`` would not disambiguate, being what the repr already renders (:issue:`710`). -* five more registrars warned on mere key presence rather than an actual overwrite, +* more registrars warned on mere key presence rather than an actual overwrite, outside the wording of :issue:`718`'s identity guard - (``incumbent is not None and incumbent is not new``, landed by :pr:`726`), whose - issue named only code-keyed registrars: ``register_engine``, ``register_reassembly`` - and ``register_traceflow`` on ``Extractor``, and ``register_dumper`` on both - ``Extractor`` and ``TraceFlow``. All five now compare the incumbent by identity + (``incumbent is not None and incumbent is not new``, landed by :pr:`726`), whose issue named only code-keyed + registrars: ``register_engine``, ``register_reassembly`` and ``register_traceflow`` + on ``Extractor``, and ``register_dumper`` on both ``Extractor`` and ``TraceFlow``. + Each now compares the incumbent by identity before warning; both ``register_dumper`` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings (:issue:`739`). -* **six registrars** now raise ``RegistryError`` for a non-class argument, where a bare - ``TypeError`` ("issubclass() arg 1 must be a class") used to escape -- from the guard - itself for the two ``register_dumper`` methods, and from inside ``abc`` for the other - four. They are ``register_dumper``, ``register_engine``, ``register_reassembly`` and +* **the class registrars listed below** now raise ``RegistryError`` for a non-class + argument, where a bare ``TypeError`` ("issubclass() arg 1 must be a class") used to + escape -- from the guard itself for the ``register_dumper`` methods, and from inside + ``abc`` for the others. They are ``register_dumper``, ``register_engine``, + ``register_reassembly`` and ``register_traceflow`` on ``Extractor``, ``TraceFlow.register_dumper`` (reached through ``register_traceflow_dumper``), and ``pcapkit.foundation.registry.protocols.register_protocol``. Each guard gains an - explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in the five that - accept a ``ModuleDescriptor`` it runs after the descriptor is unwrapped, so a descriptor + explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that + accept a ``ModuleDescriptor`` it runs after the descriptor is unwrapped, so a + descriptor naming a non-class attribute is rejected too, while ``register_protocol`` takes no descriptor. No guard's target class changes, and a wrong class still raises ``RegistryError`` as before. ``RegistryError`` subclasses ``TypeError``, so ``except TypeError`` still catches it; a caller matching the exact type, or the old - message, does not. The same guard in the seven protocol ``register`` classmethods is + message, does not. The same guard in the protocol ``register`` classmethods is fixed separately by :issue:`1026` (:issue:`1021`). -* a missing extraction engine package now gives one ``EngineWarning`` rather than two +* a missing extraction engine package now gives a single ``EngineWarning`` rather than + a duplicate (:issue:`1045`, :pr:`1046`). ``Extractor.import_test`` warned that the package was absent, then ``Extractor.run`` warned again when it fell back to the default engine; ``run`` now stays silent there, and ``import_test`` says what ``run`` used to: ``engine () is not installed; using default engine instead``, the module name in backticks. Only - code that matched ``import_test``'s old wording, or that expects two warnings, - breaks. -* three ``Extractor`` defects: ``Extractor.output``'s ``UnsupportedCall`` named + code that matched ``import_test``'s old wording, or that expects the duplicate + warning, breaks. +* ``Extractor`` defects: ``Extractor.output``'s ``UnsupportedCall`` named ``format`` rather than ``output``; ``make_name`` looked an unknown format up in the ``__output__`` ``defaultdict`` and so inserted it; and the Scapy engine never set ``_offmt``, so ``Extractor.format`` raised ``AttributeError`` under ``engine='scapy'`` (:issue:`1095`). * the DPKT, Scapy and PyShark engines call ``Extractor.record_header()`` at setup again, as they did until 2023, so the global-header step is shared with the default - engine and sets ``_offmt`` for all four. **Their dumped output changes**: JSON, plist and tree + engine and sets ``_offmt`` for each of them. **Their dumped output changes**: JSON, + plist and tree reports gain the leading ``Global Header`` record (``Section Header 1`` for PCAP-NG) the default engine already writes, identical to its, and ``files=True`` writes that record's file too; frame records are unchanged. ``Extractor.format`` now also answers @@ -1043,9 +1070,9 @@ Fixed byte. The third-party engines hand the writer no frame octets, so they fall back to JSON with a ``FormatWarning``, as their flow tracing already does, and a PCAP-NG input raises ``FormatError`` (:issue:`1127`). The written global header also - dropped the input's ``thiszone``, ``sigfigs`` and ``snaplen``, so only 1 of 17 - sample PCAP files came back byte for byte; the PCAP engine now passes all three - through ``_open_output`` to ``PCAPIO``, and all 17 do (:issue:`1165`). + dropped the input's ``thiszone``, ``sigfigs`` and ``snaplen``, so most sample + PCAP files did not come back byte for byte; the PCAP engine now passes all three + through ``_open_output`` to ``PCAPIO``, and every one does (:issue:`1165`). * IP reassembly reports only the octets it holds: a truncated fragment, a strict run and a zero-length datagram no longer report octets from the declared Total Length (:pr:`1368`). Only the final fragment's partial block counts, and a datagram too @@ -1057,7 +1084,7 @@ Fixed the last block (:pr:`1564`). * a second TCP flow with the same label truncated the first flow's file; a reused label now gets the next free suffix (:issue:`1360`, :pr:`1371`). -* the engine, reassembly, traceflow and dumper registrars accept the built-in a name +* the engine, reassembly, traceflow and dumper registrars accept the built-in each name shipped with, which restores it (:issue:`1363`, :pr:`1376`). * PCAP-NG blocks are numbered across the whole file, not per section (:issue:`1256`, :pr:`1426`). @@ -1104,15 +1131,15 @@ Added check is recorded on the parsed result the same way. Extended Sequence Numbers, TFC padding and anti-replay are not implemented, and an unsupported cipher or MAC is refused with a clear error rather than half-processed. -* SCTP as a transport protocol [:rfc:`9260`]: all 13 chunk types, 8 chunk - parameters and 13 error causes, with the CRC32c both recorded and verifiable (it +* SCTP as a transport protocol [:rfc:`9260`]: every chunk type, chunk + parameter and error cause it defines, with the CRC32c both recorded and verifiable (it covers the SCTP packet alone, with no IP pseudo-header). Upper layers register on the DATA chunk's Payload Protocol Identifier through ``register_sctp``, not on a port (:pr:`379`). * NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through ``pycrate`` (``pip install pypcapkit[NGAP]``) (:discussion:`251`, :pr:`417`). Decoding is - generic by ASN.1 shape rather than per-procedure, so all 81 elementary - procedures and 438 protocol IEs work and a new 3GPP release needs no code + generic by ASN.1 shape rather than per-procedure, so every + elementary procedure and protocol IE works and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. ``pycrate`` was at first left out of the ``all`` extra -- it is LGPL-2.1+ where this package @@ -1120,12 +1147,12 @@ Added counted it among ``all``'s core addons, so ``all`` installs it (the ``all`` entry under *Project infrastructure*). * the Mobility Header registry, completed (:pr:`383`, :pr:`437`): the :rfc:`5568` - fast-handover messages and options, then all 24 registered message data types, all - 71 registered options (with nested sub-option registries for the flow + fast-handover messages and options, then every registered message data type, + every registered option (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG - parameter families) and all 4 CGA extensions. The CGA Parameters option, the only + parameter families) and every CGA extension. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the - generic handler, which is what let the four extensions round-trip end to end. + generic handler, which is what let the CGA extensions round-trip end to end. * dispatch entries for dissectors that existed but were reachable from no registry (:pr:`436`): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, ``L2TPv2`` on UDP 1701 and OSPF at ``TransType`` 89. ``VLAN`` became an abstract base with @@ -1150,7 +1177,7 @@ Added mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises ``RegistryError``, and an unrecognised class keyword raises ``UnsupportedCall``, - matching the other four families. Backed by + matching the other registrar families. Backed by ``pcapkit.foundation.registry.protocols.register_protocol_code``, also callable directly for a class that declined at definition time. It was written up as the mechanism :issue:`548` needs; :issue:`548` turned out to need a missing class @@ -1177,14 +1204,15 @@ Added * ILNP nonce sizing coverage in ``tests/protocols/internet/test_ipv6_extension_unit.py``, one test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse - cycle over ten nonces, most with a bit length that is *not* a multiple of eight, + cycle over a spread of nonces, most with a bit length that is *not* a multiple of + eight, where floor division and the ceiling disagree (:issue:`601`). * ``SOLUTION`` parameter width coverage in - ``tests/protocols/internet/test_hip_unit.py``: eight widths asserting the declared + ``tests/protocols/internet/test_hip_unit.py``: widths asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length :rfc:`5201#section-5.2.5` requires of HIPv1. - Five widths are deliberately *not* multiples of eight, where the defective - ``ceil(bits / 4)`` coincides with the correct width (:issue:`608`). + Some widths are deliberately *not* multiples of eight, where the defective + ``ceil(bits / 4)`` differs from the correct width (:issue:`608`). * the BSD loopback layer for ``LINKTYPE_NULL`` and ``LINKTYPE_LOOP``, and dissection of ``LINKTYPE_RAW`` by the version nibble (:pr:`1594`). * one home for each RFC- or format-fixed constant spelled out in more than one module @@ -1295,8 +1323,8 @@ Changed frame cut at the chain bound warns once (:issue:`1619`, :pr:`1630`). * **a breaking change to** IPv6 extension headers: an unrecognised or unparsable one is parsed generically, per :rfc:`6564`, rather than collapsing the packet to ``Raw`` - (:issue:`891`, :pr:`904`). ``IPv6_Ext`` does it, and is also the shared base of all - eight extension headers (:issue:`917`, :pr:`924`). + (:issue:`891`, :pr:`904`). ``IPv6_Ext`` does it, and is also the shared base of + every extension header (:issue:`917`, :pr:`924`). * **a breaking change to** ``pack()``: it no longer keeps the header it builds on the instance it is called on, which in 1.4.1 too left ``schema`` disagreeing with ``info``: ``UDP(srcport=1234).pack()`` reset ``schema.srcport`` to 0 while @@ -1307,6 +1335,11 @@ Changed an option allowed only once no longer raises. Code that relied on ``pack()`` to re-sync ``schema`` calls ``make()`` or constructs a new instance (:issue:`1769`, :pr:`1791`). +* **a breaking change to** the HOPOPT, IPv6-Opts and Mobility Header padding: two + octets are one ``PadN`` with no data (``01 00``) rather than two ``Pad1`` options, + as :rfc:`8200#section-4.2` and :rfc:`6275#section-6.2.2` ask. Only a freshly built + header with exactly two octets of padding changes; a parsed one that carried two + ``Pad1`` options still rebuilds byte for byte (:issue:`1793`, :pr:`1812`). Fixed ~~~~~ @@ -1350,8 +1383,9 @@ Fixed IPv6-Opts, MH, HIP and SCTP; each iteration must now advance the stream by at least one octet, and the error names the option, the offset and the octets remaining (:issue:`431`, :pr:`432`). Separately, wire-derived lengths in - ``ipv6_opts``, CALIPSO, MPL, REG_INFO and four HIP list callbacks underflowed - below zero, which ``ListField``'s own ``while length > 0`` turned into a silent + ``ipv6_opts``, CALIPSO, MPL, REG_INFO and HIP list callbacks + underflowed below zero, which ``ListField``'s own ``while length > 0`` turned into a + silent empty list; they are floored and raise instead (:pr:`449`, :pr:`456`, :pr:`460`, :issue:`463`). * field widths and units, each measured against the specification: HIP's @@ -1363,24 +1397,24 @@ Fixed (:issue:`487`, :pr:`489`); and the Fast Binding Update and Acknowledgment Lifetimes are plain seconds [:rfc:`5568`], not :rfc:`6275`'s four-second units (:pr:`502`). -* the last four call sites where a ``_make_*`` method converts an address itself, +* the remaining call sites where a ``_make_*`` method converts an address itself, ahead of the schema, so the same ``bool``-as-``int`` mistake reached them, unguarded by the schema-layer fix above: ``ARP._make_proto_resolve``, ``IPv6_Route._make_data_type_rpl`` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the ``dst`` - parameter of ``IPv6_Route.make``, and the latent ``OSPF._make_id_numbers``. All - four now go through ``parse_ip_address``, the helper :pr:`539` and :issue:`552` + parameter of ``IPv6_Route.make``, and the latent ``OSPF._make_id_numbers``. Each + now goes through ``parse_ip_address``, the helper :pr:`539` and :issue:`552` route their own sites through, completing the programme :pr:`481` and - :issue:`508` started. Pinning ``version=6`` on the two ``IPv6_Route`` sites is a + :issue:`508` started. Pinning ``version=6`` on the ``IPv6_Route`` sites is a second, smaller behaviour change: both converted a plain integer through the family-inferring ``ipaddress.ip_address``, so ``ip=[258]`` packed a 4-octet IPv4 address (``0.0.1.2``) inside an IPv6-only header; it now packs the 16-octet ``::102`` (:issue:`540`). * PCAP and PCAP-NG output and parsing: ``bytes(frame)`` returned the *next* frame's octets, ``files=True`` wrote names like ``Frame 1..json``, and a PCAP-NG - ``timestamp_epoch`` was shifted by the reading host's timezone (:pr:`403`); seven - further parser defects (:issue:`341`--:issue:`347`, :pr:`371`) and, on the write - path, four more block-parsing ones (:pr:`388`); ``BitField`` packed every named + ``timestamp_epoch`` was shifted by the reading host's timezone (:pr:`403`); further + parser defects (:issue:`341`--:issue:`347`, :pr:`371`) and, on the write path, more + block-parsing ones (:pr:`388`); ``BitField`` packed every named bit as set (:issue:`359`, :pr:`374`); the extension-header walk failed to advance past the last IPv6 extension header (:issue:`348`, :pr:`373`); and IPv6 fragment offsets went unscaled, with reassembly keyed on the flow label -- optional, and @@ -1399,14 +1433,14 @@ Fixed always right). Spec-correct ``ADD_ADDR`` and ``MP_PRIO`` options failed with ``FieldError: TCP: [OptNo 30] 3 invalid IP version`` and ``KeyError: 'length'``; both parse now. -* two dropped-keyword/wrong-cast defects flagged in review during this release and +* dropped-keyword/wrong-cast defects flagged in review during this release and filed only as :issue:`556`: HIP's ``_make_param_encrypted`` passed ``cipher=`` to a schema with no such field, so an AES-cipher ``ENCRYPTED`` parameter built through ``make`` packed without its IV; and IPv6-Route's ``RPL.post_process``, which runs on every ``Schema.pack``, assumed ``self.addresses`` was still the concatenated ``bytes`` a parse leaves, and raised slicing the ``list[bytes]`` a ``make``-built multi-address header holds (:issue:`556`). -* two more defects :issue:`541` exposed by letting construction reach code that had +* further defects :issue:`541` exposed by letting construction reach code that had never run. ``MPTCP.subtype`` was still ``typing.TYPE_CHECKING``-only, so ``TCP(options=[(Enum_Option.Multipath_TCP, ...)])`` raised ``AttributeError: ... has no attribute 'subtype'`` for every subtype but @@ -1429,18 +1463,20 @@ Fixed * TCP's MP_CAPABLE now parses and builds all five lengths of :rfc:`8684` section 3.1 -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where ``_read_mptcp_capable`` raised - ``ProtocolError: TCP: [OptNo 30] invalid format`` for anything but 12 and 20. In the schema ``skey`` is now present - only for ``length > 4`` and ``rkey`` for ``length > 12``, and two fields are new: - ``dl_len`` (16 bits, ``length > 20``) and ``checksum`` (2 octets, ``length > 22``). + ``ProtocolError: TCP: [OptNo 30] invalid format`` for anything but 12 and 20. In the + schema ``skey`` is now present + only for ``length > 4`` and ``rkey`` for ``length > 12``, and the fields + ``dl_len`` (16 bits, ``length > 20``) and ``checksum`` (2 octets, ``length > 22``) + are new. ``_make_mptcp_capable`` takes ``dl_len`` and ``checksum`` too, where it used to drop them silently through ``**kwargs``, and picks the length from how many of ``skey``, ``rkey``, ``dl_len`` and ``checksum`` are not ``None``. A field given without the one before it raises ``ProtocolError``. The data model ``MPTCPCapable`` gains ``dl_len`` and ``checksum``, and ``skey`` becomes ``Optional[int]``: code that constructs it directly must now pass both new fields, - and ``to_dict()`` carries two more keys (:issue:`1042`). + and ``to_dict()`` carries the new keys (:issue:`1042`). * IPv6-Route's RPL routing data was five octets wide at the front where :rfc:`6554` - section 3 gives four, with three further defects behind it. ``CmprI``, ``CmprE`` and + section 3 gives four, with further defects behind it. ``CmprI``, ``CmprE`` and ``Pad`` are each 4 bits sharing a 32-bit word with a 20-bit ``Reserved``, but the schema declared ``cmpr_i`` and ``cmpr_e`` as whole octets, so a constructed two-address header packed to 41 octets while its ``Hdr Ext Len`` of 5 declared 48. @@ -1516,8 +1552,8 @@ Fixed not a multiple of eight was affected, and **small values were the worst case rather than boundary values**: any nonce below 256 was declared as *zero* octets and dropped (``nonce=9`` packed to ``b'\x8b\x00'`` and parsed back as ``0``). Fixed to - ``max(1, math.ceil(nonce.bit_length() / 8))``, the form used at five other sizing - sites in ``hip.py`` and ``mh.py``. The one-octet floor is the ``mh.py`` + ``max(1, math.ceil(nonce.bit_length() / 8))``, the form used at the other + sizing sites in ``hip.py`` and ``mh.py``. The one-octet floor is the ``mh.py`` convention and is load-bearing because ``nonce`` defaults to ``0``, whose bit length is ``0``: without it the default builds an option with no Nonce Value field at all, collapsing "the nonce is 0" into "there is no nonce" when @@ -1545,10 +1581,11 @@ Fixed :rfc:`9112#section-4` gives ``status-code = 3DIGIT``, exactly three, so the anchor is what the grammar said (the production lives in HTTP/1.1 because ``status-code`` is part of its ``status-line``; :rfc:`9110#section-15` covers code semantics and the - registry). ``_RE_VERSION`` was audited and is safe, since both call sites read the - captured group (:issue:`583`). -* as a consequence of the above, the unhandled ``MemoryError`` at - ``pcapkit/protocols/protocol.py:1411`` on a truncated PCAP-NG capture. + registry). ``_RE_VERSION`` was audited and is safe, since every call site reads + the captured group (:issue:`583`). +* as a consequence of the above, the unhandled ``MemoryError`` in + ``pcapkit/protocols/protocol.py``'s ``self._file.read()`` call on a truncated + PCAP-NG capture. ``dhcp_little_endian.pcapng`` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which ``rjust()`` turned into ``0x78000000`` or ``0x84000000`` (1.88 to 2.06 GiB) and passed straight to @@ -1567,7 +1604,7 @@ Fixed ``74``. ``incl_len`` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising ``ValueError: read length must be non-negative or -1`` -- the reported crash, and - the *second* symptom. The sibling ``Frame.pack`` eleven lines earlier spelled the + the *second* symptom. The sibling ``Frame.pack`` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; ``byteorder_callback`` now records that it defines the key and why the fallback hides a typo (:issue:`605`). @@ -1649,9 +1686,11 @@ Fixed :issue:`554`, :issue:`573` and :issue:`594` on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until :pr:`614` (:issue:`618`). -* the HIP ``PUZZLE`` and ``SOLUTION`` builders derived three wire-format values from - the payload value instead of the data model, and all three were wrong. **This - changes two public data models and the octets both parameters emit.** The width of +* the HIP ``PUZZLE`` and ``SOLUTION`` builders derived wire-format values from the + payload + value instead of the data model, and each was wrong. **This changes the** + ``PuzzleParameter`` **and** ``SolutionParameter`` **data models and the octets both + parameters emit.** The width of ``Random #I`` and ``Puzzle solution #J`` came from ``int.bit_length()`` alone, so every leading zero octet was dropped on re-serialisation: a ``SOLUTION`` read with ``Length = 20`` rebuilt as ``Length = 6``, a ``PUZZLE`` with ``Length = 12`` as @@ -1678,23 +1717,23 @@ Fixed HIPv1 each builder accepted only a ``bit_length()`` of 57..64 and built, for anything narrower, a parameter this library's reader rejects -- the shape of :issue:`608`, in the ``PUZZLE`` builder that :pr:`629` never opened (:issue:`655`). - The two remaining ``math.log2`` sites, both in ``PUZZLE`` where a lifetime is real, + The remaining ``math.log2`` sites, in ``PUZZLE`` where a lifetime is real, raise ``ProtocolError`` (``(BaseError, ValueError)``, so a caller written around the bare exception still catches it). Migration: ``SolutionParameter.lifetime`` is now ``reserved`` and an ``int`` rather than a ``timedelta``; ``PuzzleParameter`` and ``SolutionParameter`` gained a required ``rhash_len``; and ``_make_param_solution`` - no longer takes ``lifetime=``. All three were reachable end to end only once + no longer takes ``lifetime=``. Each was reachable end to end only once :issue:`608` was fixed in :pr:`629`, which removed the parity guard that had failed these rebuilds loudly; the round trip then stopped raising and started quietly emitting a different parameter, which is why they were fixed together (:issue:`653`, :issue:`654`, :issue:`655`). * every HIP parameter pcapkit emitted was ``4 (mod 8)`` octets long, because the - padding aligned the *contents* rather than the record. Corrected for **45 of the - 46** HIP parameters; ``LOCATOR_SET`` is excluded, below. **This changes the octets - those 45 parameters write, and the** ``length`` **their data models report.** + padding aligned the *contents* rather than the record. Corrected for **every + HIP parameter but** ``LOCATOR_SET``, excluded below. **This changes the octets those + parameters write, and the** ``length`` **their data models report.** :rfc:`7401#section-5.2.1` requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives - ``Total Length = 11 + Length - (Length + 3) % 8``; all 95 padding sites computed + ``Total Length = 11 + Length - (Length + 3) % 8``; every padding site computed ``(8 - (Length % 8)) % 8``, which aligns the contents alone. For every ``Length`` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at ``Length = 4`` (a whole ``SEQ``, which @@ -1702,12 +1741,13 @@ Fixed that must not be there, and at ``Length = 8`` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test - could see it and why the fix is asserted against the RFC's arithmetic. 94 of the 95 - sites (45 of 46 ``PaddingField`` callbacks in - ``pcapkit/protocols/schema/internet/hip.py`` and 49 of 49 record lengths in - ``pcapkit/protocols/internet/hip.py``) are now one ``parameter_total_len`` and one - ``parameter_padding_len`` (:issue:`651`). ``LOCATOR_SET`` kept the old expression - because two defects there cancelled exactly: its padding callback never received the + could see it and why the fix is asserted against the RFC's arithmetic. Every site + but ``LOCATOR_SET``'s (the ``PaddingField`` callbacks in + ``pcapkit/protocols/schema/internet/hip.py`` and the record lengths in + ``pcapkit/protocols/internet/hip.py``) now goes through one ``parameter_total_len`` + and one ``parameter_padding_len`` (:issue:`651`). ``LOCATOR_SET`` kept the old + expression + because defects there cancelled exactly: its padding callback never received the parameter's ``len`` (the nested ``Locator`` schemas share a packet context whose own ``len`` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's ``len`` was written in 4-octet units where the RFC's ``Length`` is a byte count; the result, @@ -1720,7 +1760,8 @@ Fixed the defect: :rfc:`7401#section-5.1.3` defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a ``SEQ`` parameter's ``length`` is now 8 where it was - 12, and the other 44 move likewise, so code comparing stored output byte for byte or + 12, and the other parameters move likewise, so code comparing stored output byte for + byte or asserting on ``Data_*Parameter.length`` sees the RFC's values. ``LOCATOR_SET`` is unchanged in both respects until :issue:`679` (below). ``examples/generators/options.py``'s ``HIP_COPIES`` stayed at two for @@ -1728,13 +1769,13 @@ Fixed eight, which this defect had masked, until :issue:`672` widened ``counter`` and :issue:`679` fixed ``LOCATOR_SET``'s ``Length`` unit, after which :issue:`689` dropped ``HIP_COPIES`` to one (:issue:`651`). -* two HTTP/2 flag defects, one on each side of a round trip. **This changes the octets +* HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's - flags.** ``_make_http_data`` was the only one of the six ``_make_http_*`` methods + flags.** ``_make_http_data`` was the only ``_make_http_*`` method that never read ``frame.flags``, so a DATA frame parsed with ``END_STREAM`` set rebuilt with the bit clear (flags octet ``0x01`` to ``0x00``): a well-formed frame saying the stream continues where the capture said it ended - [:rfc:`9113#section-6.1`]. ``PADDED`` was re-derived from ``pad_len``, and the five + [:rfc:`9113#section-6.1`]. ``PADDED`` was re-derived from ``pad_len``, and the sibling builders already restored theirs, so this was a missing read-back (:issue:`652`). Separately, ``FrameType.post_process`` seeded its flag accumulator with a bare ``0``, which ``|=`` promotes only as a side effect, so a frame with @@ -1743,17 +1784,17 @@ Fixed ``TypeError: argument of type 'int' is not a container or iterable`` rather than answering ``False``. That is the common case: a ``0x00`` flags octet is every opening ``SETTINGS``, every non-final ``DATA`` and every non-ACK ``PING``. It now - seeds ``self.Flags(0)``, as the construct path already did at all six of its sites. + seeds ``self.Flags(0)``, as the construct path already did at each of its sites. In the dump, ``__value__`` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, ``"Flags::None [0]"`` where it read ``0``, the same change :issue:`616` made to TCP's ``connection``. The seed is guarded because ``FrameType.Flags`` declares no members and, from Python 3.11, a memberless ``enum.Flag`` subclass refuses - ``Flags(0)``, so the five frame schemas that inherit it (``UnassignedFrame``, + ``Flags(0)``, so the frame schemas that inherit it (``UnassignedFrame``, ``PriorityFrame``, ``RSTStreamFrame``, ``GoawayFrame``, ``WindowUpdateFrame``) keep - the plain ``int``, which nothing observes since all five pass ``flags=None``. A DATA - round trip is **still** lossy for an unrelated reason found while measuring: three - payload length callbacks are mis-parenthesised, so an unpadded ``DATA``, ``HEADERS`` + the plain ``int``, which nothing observes since each passes ``flags=None``. A DATA + round trip is **still** lossy for an unrelated reason found while measuring: payload + length callbacks are mis-parenthesised, so an unpadded ``DATA``, ``HEADERS`` or ``PUSH_PROMISE`` frame parses with its whole payload dropped (:issue:`668`). (:issue:`652`, :issue:`650`). * four MPTCP error messages carried a doubled separator, rendering as @@ -1765,13 +1806,13 @@ Fixed file spelled correctly; now 32 correct against 0 doubled. Pre-existing since 2023-04-10. The new test asserts each message in full, since the existing ``assertIn`` on the body passes either side of the change (:issue:`649`). -* three names appeared in string annotations their own module never imported, so a +* names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: ``Any`` in ``pcapkit/utilities/logging.py``, and in ``pcapkit/protocols/schema/internet/ipv6_route.py`` both ``Protocol`` in a - ``payload:`` stub and ``Optional`` inside a ``typing.cast``. All three are now in - their module's ``TYPE_CHECKING`` block, ``Protocol`` spelled - ``ProtocolBase as Protocol`` as twenty-one sibling schema modules already do. The + ``payload:`` stub and ``Optional`` inside a ``typing.cast``. Each is now in its + module's ``TYPE_CHECKING`` block, ``Protocol`` spelled ``ProtocolBase as Protocol`` + as the sibling schema modules already do. The ``cast`` case is the one worth naming: ``typing.cast`` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (``TYPE_CHECKING`` is ``False``); making these @@ -1779,8 +1820,8 @@ Fixed ``tests/project/test_annotation_names.py`` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as ``'list["Nested"]'`` - and treating ``Literal`` members and ``Annotated`` metadata as values, and reports - exactly those three findings on the unfixed tree. The same change corrects the + and treating ``Literal`` members and ``Annotated`` metadata as values, and + reports exactly those findings on the unfixed tree. The same change corrects the ``MANIFEST.in`` comment asserting that ``include README.md`` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the :pr:`619` entry above carried the same @@ -1800,8 +1841,8 @@ Fixed no ``length`` could make it right. The fix is at the computation rather than the injection: ``PCAPNG.packet`` is overridden to take the payload from the block schema's ``__payload__`` field and the header from the octets ahead of it, summed - out of the schema buffers so the three types' payload offsets (28, 12, 28) are not - hard-coded. Teaching ``ProtocolBase.__init__`` to skip a protocol that had set + out of the schema buffers so the packet types' payload offsets (28, 12, 28) are + not hard-coded. Teaching ``ProtocolBase.__init__`` to skip a protocol that had set ``packet`` was rejected: it heals ``frame.info.packet`` while ``frame.packet`` still reports the whole block as header. The override is a plain ``property``, not the ``cached_property`` it replaces: the inherited one caches because it *reads the @@ -1817,9 +1858,9 @@ Fixed :issue:`685` (below) later removed the stale ``examples/captures/pcapng.txt`` from the index (:issue:`646`). * an unpadded HTTP/2 ``DATA``, ``HEADERS`` or ``PUSH_PROMISE`` frame parsed with its - whole payload silently discarded. Three payload length callbacks in - ``pcapkit/protocols/schema/application/httpv2.py`` put the conditional in the wrong - place: a conditional binds looser than ``-``, so + whole payload silently discarded. The payload length callbacks for + those frames in ``pcapkit/protocols/schema/application/httpv2.py`` put the + conditional in the wrong place: a conditional binds looser than ``-``, so ``pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0`` groups as ``(pkt['__length__'] - pkt['pad_len']) if ... else 0``, and the ``else`` arm returned ``0`` ("read no octets") where it meant to subtract ``0``. Padding is rare @@ -1846,15 +1887,16 @@ Fixed (:issue:`668`). * the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. ``EnumSchema.register`` was a bare - ``cls.__enum__[code] = schema``, and is the schema half of 14 public registrars, so + ``cls.__enum__[code] = schema``, and is the schema half of the public option + registrars, so one ``register_ipv4_option`` call replacing a built-in named the parser it displaced and said nothing about the schema. ``EnumSchema.__init_subclass__`` reaches the same registry without calling ``register``, so a subclass declared with a ``code=`` keyword (the documented way to add a schema) stayed silent too; both are guarded, - and the declaration hook's two branches are folded into one loop. Presence is a + and the declaration hook's branches are folded into one loop. Presence is a faithful test only because ``_EnumRegistry.__missing__`` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate - registration warn. The seven code-keyed parser registrars (on ``ProtocolBase``, + registration warn. The code-keyed parser registrars (on ``ProtocolBase``, ``Link``, ``Internet``, ``Frame``, ``PCAPNG``, ``Transport`` and ``SCTP``) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard :pr:`681` @@ -1866,9 +1908,9 @@ Fixed (``Internet.register(TransType.TCP, TCP)``) still warns, a false positive the harmonised guard does not close. ``ContextRegistry.register`` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so - none of the three takes a guard. ``import pcapkit`` emits no ``RegistryWarning``: - none of its 327 registry writes (one being ``R1CounterParameter``'s second code, from - :issue:`690`) lands on a key already present (:issue:`692`). + none of them takes a guard. ``import pcapkit`` emits no ``RegistryWarning``: + none of its registry writes (``R1CounterParameter``'s second code, from + :issue:`690`, among them) lands on a key already present (:issue:`692`). * every EOF-truncated PCAP-NG file raised a bare ``ValueError`` out of ``Extractor``, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last @@ -1880,11 +1922,11 @@ Fixed ``Extractor.record_frames`` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). A second route to the same shape is closed by a new ``nonnegative()`` helper, composed into ``bounded_option`` and - ``bounded_area`` and applied to the eight unwrapped ``length - N`` spans and eight + ``bounded_area`` and applied to the unwrapped ``length - N`` spans and the ``__option_padding__`` fields: ``_TextField.__call__`` builds ``f'{length}s'`` unconditionally, so a negative became ``'-8s'`` and ``struct.calcsize`` raised an - exception caught nowhere a frame loop looks. Both of :issue:`678`'s failure families - are gone; two others remain: + exception caught nowhere a frame loop looks. Both of :issue:`678`'s failure + families are gone; others remain: ``ValueError: N is not a valid BlockType`` (filed as :issue:`701`) and ``MemoryError`` from :pr:`593`'s 32-bit band. Three bare exceptions in ``SystemdJournalExportBlock.post_process``, reachable from valid input, are fixed @@ -1893,25 +1935,25 @@ Fixed length was unbounded against its entry (``OverflowError``; now clamped and reported); and a non-UTF-8 name, key or value raised ``UnicodeDecodeError`` (now ``errors='replace'`` and reported). The silent loss of every field after a binary - one is filed as :issue:`704`. The five unclamped non-packet option areas keep the - per-block framing assumption :pr:`676`'s note describes. Also here, the fifth and - last registrar with no overwrite guard: ``Option.register`` now reports a displaced + one is filed as :issue:`704`. The unclamped non-packet option areas keep the + per-block framing assumption :pr:`676`'s note describes. Also here, the last + registrar with no overwrite guard: ``Option.register`` now reports a displaced option schema as a ``RegistryWarning`` naming every namespace the ``ns='opt'`` fan-out displaced something in, tested by membership (subscripting the per-namespace ``defaultdict`` would insert ``UnknownOption`` for a code nobody registered). The - condition is identity-based like the seven parser registrars and :pr:`681`'s guard + condition is identity-based like the parser registrars and :pr:`681`'s guard for ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the call itself - creates is exempt. Flooring the - three decryption-secrets payloads that read ``__length__`` *whole* was tried and + creates is exempt. Flooring + the decryption-secrets payloads that read ``__length__`` *whole* was tried and reverted, since ``Schema.pack`` leaves that key at ``-1`` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may - carry zero-padded octets, and eight short-file depths move from - ``ProtocolError: unknown byteorder magic`` to ``StreamEOFError``. The :pr:`683` + carry zero-padded octets, and some short-file depths move + from ``ProtocolError: unknown byteorder magic`` to ``StreamEOFError``. The :pr:`683` drift :issue:`685` tracks is unchanged (:issue:`678`). * HIP's ``R1_COUNTER``/``R1_Counter`` parameter packed 12 octets where :rfc:`7401#section-5.2.3` requires 16, and ``LOCATOR_SET`` declared its ``Length`` - in 4-octet units where :rfc:`7401#section-5.2.1`'s ``Length`` is a byte count -- two + in 4-octet units where :rfc:`7401#section-5.2.1`'s ``Length`` is a byte count -- independent defects, fixed together because the evidence for either needed the other out of the way. ``R1CounterParameter``'s ``counter`` was a ``UInt32Field`` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" @@ -1934,8 +1976,8 @@ Fixed ``counter=0xaabbccdd``; new tests are ``test_hip_r1_counter_width_unit.py`` and ``test_hip_locator_set_length_unit.py``. The ``hip-parameter/R1_Counter`` ``EXPECTED_FAILURES`` entry (code 128 registered no schema of its own, since - ``R1CounterParameter`` declared only ``code=129``) was deleted once that registry - defect was fixed as :issue:`690`, leaving 43 entries. ``HIP_COPIES`` dropped to one + ``R1CounterParameter`` declared only ``code=129``) was deleted once that + registry defect was fixed as :issue:`690`. ``HIP_COPIES`` dropped to one as :issue:`689`, once :issue:`672` and :issue:`679` left it routing around nothing. * ``httpv1.HTTP`` raised a bare exception for any payload that is not an HTTP/1 message, at four sites: the header/body split in ``read``, the header's CRLF split @@ -1988,7 +2030,7 @@ Fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare ``struct.error``; that root cause is in the generic ``Schema.unpack``/ - ``FieldBase.length`` machinery shared by ten schema modules, out of scope and filed + ``FieldBase.length`` machinery the schema modules share, out of scope and filed as :issue:`805`, which is why ``_guess_version``'s last arm keeps suppressing ``struct.error`` alongside ``ProtocolError`` (:pr:`802`). * ``SystemdJournalExportBlock.post_process`` skipped a binary field's one trailing @@ -2016,13 +2058,13 @@ Fixed its storage base's ``__new__``, so ``isinstance`` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under - ``opt``, ``OptionType.get`` merged ``opt`` over the requested namespace and 6 of the - other 7 namespaces answered ``opt_unknown`` for it; each now gets its own + ``opt``, ``OptionType.get`` merged ``opt`` over the requested namespace and most + other namespaces answered ``opt_unknown`` for it; each now gets its own ``_unknown``. **Breaking** because :issue:`575` asked for byte-identical output across the sample captures, as :pr:`420` and :pr:`427` did, and this deliberately - misses that bar: 14 of 23 captures change, entirely from the new ```` - rendering plus the PLIST escaping above; normalising both leaves all 14 - byte-identical. So that a resolved unassigned member still pickles, + misses that bar: some sample captures change, entirely from the new + ```` rendering plus the PLIST escaping above; normalising both leaves + them byte-identical. So that a resolved unassigned member still pickles, ``_unregistered_member`` installs a ``__reduce_ex__`` rebuilding an equivalent unregistered member (:issue:`575`). * ``httpv2._guess_version`` tried to parse a stream as HTTP/2 and read failure as "not @@ -2046,14 +2088,14 @@ Fixed positive only once :issue:`800` and :pr:`814` landed, which is why this waited. ``udp.py`` already bound the proxy for both ports; this removes the asymmetry its docstring and ``docs/source/contributing/pep.rst`` documented as an open request. - Over the sample captures every HTTP/1.1 frame keeps its chain, and nine genuine + Over the sample captures every HTTP/1.1 frame keeps its chain, and the genuine HTTP/2 frames in ``options-transport.pcap`` change ``Ethernet:IPv4:TCP:Raw`` to ``Ethernet:IPv4:TCP:HTTP/2``. Not labelled breaking, but not free: - TCP:80/8080 traffic that is neither valid HTTP/1 nor preface-carrying is now exposed - to the fall-through arm, where the direct ``httpv1`` binding left it ``Raw`` + ``TCP:80/8080`` traffic that is neither valid HTTP/1 nor preface-carrying is now + exposed to the fall-through arm, where the direct ``httpv1`` binding left it ``Raw`` regardless; a caller depending on that is the one who would notice (:issue:`682`). -* **a breaking change to** 8 more bespoke registries: step 2 of :issue:`860`, PR 1 of 2 - (``AppType`` is PR 2, :pr:`874` below), bringing ``StatusCode``, ``ReturnCode``, +* **a breaking change to** more bespoke registries: step 2 of :issue:`860`, + ahead of ``AppType`` (:pr:`874`, below), bringing ``StatusCode``, ``ReturnCode``, ``ResponseKind``, ``GroupingInformation``, ``OptionType``, ``FEATCode``, ``Command`` and ``Method`` onto ``EnumRegistry`` and applying :issue:`775`'s mint/unmint ruling to their ``get()`` as well as ``_missing_``. The first five convert 12 unambiguous @@ -2073,17 +2115,17 @@ Fixed ``feat``/``desc``/``type``/``conf`` and ``Method`` needs ``safe``/``idempotent``, which a bare wire string lacks), both ``_missing_`` and each class's own ``get()`` (a second, independent mint site) now build an unregistered member. ``FEATCode``'s - crawler now declares all 15 real FEAT-code values from the live IANA table instead of - minting 10 of them as a side effect of evaluating ``Command``'s rows at import time, + crawler now declares every real FEAT-code value from the live IANA table instead of + minting some of them as a side effect of evaluating ``Command``'s rows at import time, the import-time-mutation shape :pr:`861` removed from ``FilterType``; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: ``CommandType`` stays ``IntFlag`` (real ``A|P`` composites in the generated data), and ``TransportProtocol``'s ``auto()`` renumbering and - ``AppType`` are PR 2's. ``pcapkit/protocols/schema/misc/pcapng.py``'s + ``AppType`` are :pr:`874`'s. ``pcapkit/protocols/schema/misc/pcapng.py``'s ``OptionEnumField.post_process`` docstring is corrected: it still bypasses ``OptionType.get()`` directly, now for ``pickle`` round-trip safety (:issue:`860` - already stops the minting that used to be the reason), since neither of - ``OptionType``'s two paths gets both pickling and correct rendering right at once + already stops the minting that used to be the reason), since neither + of ``OptionType``'s paths gets both pickling and correct rendering right at once (:pr:`869`). * ``register`` on ``ProtocolBase``, ``Frame``, ``PCAPNG``, ``SCTP``, ``Link``, ``Internet`` and the concrete ``Transport`` subclasses (``TCP``, ``UDP``) now raises @@ -2091,7 +2133,7 @@ Fixed ("issubclass() arg 1 must be a class") from inside ``abc``. Each guard gains an explicit ``isinstance(protocol, type)`` test ahead of its ``issubclass``, after the ``ModuleDescriptor`` is unwrapped, so a descriptor naming a non-class attribute is - rejected too. All seven ``Raises:`` clauses already promised ``RegistryError`` if + rejected too. Every such ``Raises:`` clause already promised ``RegistryError`` if ``protocol`` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises ``RegistryError`` as before. @@ -2099,8 +2141,8 @@ Fixed first; only its subclasses reach the guard. Not breaking for exception handling: ``RegistryError`` subclasses ``TypeError`` through ``BaseError``, so ``except TypeError`` still catches it, and the type and message text are the only - things a handler can key on that change. Two observable side effects do come with - it, because ``BaseError`` is loud by default: the non-class path now emits + things a handler can key on that change. Observable side effects do come + with it, because ``BaseError`` is loud by default: the non-class path now emits pcapkit's one ``CRITICAL`` log record, and outside devmode it installs ``sys.excepthook`` and ``threading.excepthook``, neither of which the bare ``TypeError`` from ``abc`` did. That is the wrong-class path's existing behaviour, @@ -2114,10 +2156,11 @@ Fixed ``RegistryWarning`` likewise now says "name resolution record already registered". The make-side ``ns_dnsIP4addr``/``ns_dnsIP6addr`` messages also drop the lowercase ``ns_dnsip4addr``/``ns_dnsip6addr`` spelling for the ``OptionType`` member name. - Two more guards named the wrong option: ``_make_option_if_rxspeed``'s scope and + Other guards named the wrong option too: ``_make_option_if_rxspeed``'s scope and duplicate guards now say ``[if_rxspeed]`` rather than ``[if_txspeed]``, and ``_read_option_epb_queue``'s length guard says ``[epb_queue]`` rather than - ``[epb_packetid]``. Only message text changes; the exception and warning types are unchanged + ``[epb_packetid]``. Only message text changes; the exception and warning types are + unchanged (:issue:`1038`). * ``httpv1.HTTP`` refused every HTTP/1 message with no header fields, which [:rfc:`9112#section-2.1`] allows (``*( field-line CRLF )``): @@ -2146,7 +2189,7 @@ Fixed writes ``round(frac * 2**32)``, clamped to 32 bits, and the reader adds ``fraction / 2**32`` seconds, so ``timestamp`` keeps sub-second precision (:issue:`1093`). -* three HIP parameter defects. ``HOST_ID`` packed DI-Type and DI-Length as a 4-octet +* HIP parameter defects. ``HOST_ID`` packed DI-Type and DI-Length as a 4-octet ``BitField`` where :rfc:`7401` section 5.2.9 has one 16-bit word, so every built record carried two surplus octets and read back the wrong algorithm (:issue:`1118`); with it, ``make_samples.py`` builds every HIP parameter. ``HIP_CIPHER`` warned at six @@ -2158,7 +2201,7 @@ Fixed ``HIP_TRANSFORM`` already do on a version mismatch (:issue:`1133`). * the PCAP ``Header`` could not rebuild itself. ``_make_magic``'s explicit-endian branch ignored ``nanosecond`` (:issue:`1096`), and ``_make_data`` returned a - ``magic_number`` that ``make()`` does not take -- one of the three keys the + ``magic_number`` that ``make()`` does not take -- one of the keys the :issue:`617` entry above reports -- so ``from_data`` dropped it with an ``UnknownFieldWarning`` and rebuilt every header with the host-default ``d4c3b2a1`` magic. ``_make_data`` now returns ``byteorder`` and ``nanosecond``, and every byte @@ -2173,7 +2216,7 @@ Fixed ruling it is exposed rather than removed, so ``info.flags.ns`` exists and ``from_data`` keeps the bit (:issue:`1098`). * ``from_data`` round trips. ``Frame._make_data`` returned ``ts_src`` and - ``L2TPv2._make_data`` ``prio``, the other two keys the :issue:`617` entry reports, so + ``L2TPv2._make_data`` ``prio``, the other keys the :issue:`617` entry reports, so ``from_data`` discarded both with an ``UnknownFieldWarning`` and an L2TPv2 priority bit was lost; they are now ``ts_sec`` and ``priority`` (:issue:`1097`). And ``ProtocolBase.from_data`` took no keywords, so ``Frame.from_data`` and @@ -2185,8 +2228,8 @@ Fixed class has; it now dispatches on the data's class. HTTP/2's ``_make_data`` also returned the payload ``length``, which ``ProtocolBase`` takes as the parse length, so even a direct call - failed as shorter than the 9-octet frame header; ``make`` now computes it, and all - 231 HTTP frames in three sample captures rebuild byte for byte (:issue:`1154`). + failed as shorter than the 9-octet frame header; ``make`` now computes it, and + every HTTP frame in the sample captures rebuilds byte for byte (:issue:`1154`). * Quick-Start options did not round-trip. The IPv4, HOPOPT and IPv6-Opts readers took the function from ``schema.func``, which only a parsed schema carries, so a built option raised ``AttributeError``; HOPOPT and IPv6-Opts also sized the 8-octet option @@ -2207,7 +2250,7 @@ Fixed ``00000e`` for a 5-octet PRIORITY frame and ``read`` rejected RFC-conformant frames -- and, since ``_read_http_priority`` checked for 9, pcapkit's own PRIORITY frame. The length and every per-type check now exclude the header. Captures written to the - old convention no longer parse; nine frames packed by Scapy's ``contrib.http2`` now + old convention no longer parse; frames packed by Scapy's ``contrib.http2`` now parse and rebuild byte for byte (:issue:`1121`). * **a breaking change to** ``ARP.make()``: ``hlen`` and ``plen`` are derived from the packed ``sha`` and ``spa`` when omitted, and an address that disagrees with its length @@ -2227,11 +2270,12 @@ Fixed (:issue:`1141`). * **a breaking change to** ``TCP.make()``'s options: ``_make_tcp_options`` dropped the ``NOP`` and ``EOOL`` options it was given and padded each other option to 4 octets - separately, so ``from_data`` rebuilt none of the 1,117 segments in ``http.pcap`` + separately, so ``from_data`` rebuilt none of the segments in ``http.pcap`` with their original header octets. Options are now emitted as given and padded once at the end -- zero octets after a trailing ``EOOL``, otherwise ``NOP`` options and an - ``EOOL`` -- so a built header can be shorter than before; every segment in six - sample captures rebuilds, and ``options-*.pcap`` regenerate unchanged (:issue:`1151`). + ``EOOL`` -- so a built header can be shorter than before; every segment in the + sample captures measured rebuilds, and ``options-*.pcap`` regenerate unchanged + (:issue:`1151`). * **a breaking change to** ``from_data`` on a capture truncated by snaplen: ``IPv4``, ``IPv6`` and ``UDP`` rebuilt their length field from the payload at hand, writing the recorded length, so ``big_endian.pcap`` frame 3 came back as ``45000052`` where @@ -2266,16 +2310,16 @@ Fixed * **a breaking change to** ``IPv6.from_data``: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden ``__exthdr__`` key, absent from ``to_dict`` - and dumped output, and re-emitted ahead of the upper layer. Of 1,602 sample frames, - 130 that rebuilt wrong now rebuild exactly, and 50 that rebuilt with wrong bytes - raised ``AttributeError`` instead (:issue:`1177`). Those were two older defects, now fixed: + and dumped output, and re-emitted ahead of the upper layer. Sample frames that + rebuilt wrong now rebuild exactly, and those that rebuilt with wrong bytes raised + ``AttributeError`` instead (:issue:`1177`). Those were older defects, now fixed: ``HIP.alias`` read ``_info`` before ``read`` had set it, which also broke a standalone HIP parse and ``HIP.from_data`` (:issue:`1184`), and the MPL option's maker called ``bit_length()`` on the ``IPv6Address`` seed of an S=0 option. MPL in ``HOPOPT`` and ``IPv6_Opts`` now takes ``S`` from the parsed option or a new ``seed_type`` keyword, and a missing or oversized seed raises ``ProtocolError`` - (:issue:`1185`). With both fixed, all 179 IPv6 frames of the five - ``options-*.pcap`` captures rebuild byte for byte. + (:issue:`1185`). With both fixed, every IPv6 frame of the + ``options-*.pcap`` captures rebuilds byte for byte. * **a breaking change to** the SMF_DPD option's maker in ``HOPOPT`` and ``IPv6_Opts``: it overwrote the parsed DPD mode, so H-DPD rebuilt as I-DPD, and inferred the TaggerID type from the Tagger ID's length, so a 4-octet ``DEFAULT`` @@ -2341,11 +2385,11 @@ Fixed ``from_data`` raised ``struct.error``, and AH wrote its reserved field back as zero (:pr:`1338`). * HIP: Header Length, an empty parameter list, the PUZZLE lifetime and reserved bits - (:pr:`1342`); Locator flags (:issue:`1365`, :pr:`1367`); three ``BitField`` - namespaces' reserved bits (:issue:`1417`, :pr:`1419`); and the two fixed header + (:pr:`1342`); Locator flags (:issue:`1365`, :pr:`1367`); ``BitField`` +namespaces' reserved bits (:issue:`1417`, :pr:`1419`); and the two fixed header bits (:issue:`1459`, :pr:`1462`). * MH: zero-length PadN, reserved fields and BU/BA flag bits (:issue:`1324`, - :pr:`1345`); the reserved bits of 24 ``BitField`` flag namespaces + :pr:`1345`); the reserved bits of the ``BitField`` flag namespaces (:issue:`1418`, :pr:`1421`); and the reserved octets ``PaddingField`` sites capture (:pr:`1442`). * PCAP-NG: constructed and rebuilt blocks (:pr:`1339`), lengths and padding @@ -2382,7 +2426,7 @@ Fixed (:issue:`1680`, :pr:`1723`). * the warnings of a next layer replaced by ``Raw`` are dropped (:issue:`1580`, :pr:`1588`). -* IPv6-Route's three data-type readers name the routing type in their +* IPv6-Route's data-type readers name the routing type in their ``ProtocolError``, where each message read ``[TypeNo ]`` (:issue:`442`), and ``HTTP(..., version=1)`` parses from the frame's data rather than the drained file, where it raised ``ValueError`` (:issue:`447`, :pr:`451`). @@ -2401,15 +2445,21 @@ Fixed :pr:`543`). * the MPTCP option header reads ``length`` from bit 8, per :rfc:`8684#section-3`, rather than bit 1 (:issue:`553`, :pr:`558`). -* four Mobility Header enumerations defined inline from their RFCs +* Mobility Header enumerations defined inline from their RFCs (``FastBindingAcknowledgmentStatus``, ``IPv6AddressPrefixCode``, ``LocalizedRoutingStatus``, ``LMAAddressCode``) raise ``EnumValueError`` for an - unassigned value rather than minting ``Unassigned_N`` (:issue:`880`, :pr:`889`), - and all four inherit ``get`` and its ``default`` from ``EnumLookup`` rather than + unassigned value rather than minting ``Unassigned_N`` (:issue:`880`, :pr:`889`), and + each inherits ``get`` and its ``default`` from ``EnumLookup`` rather than overriding it (:issue:`935`, :pr:`940`). * ``pcapkit.protocols.protocol.__all__`` lists ``Protocol`` beside ``ProtocolBase``, so ``from pcapkit.protocols.protocol import *`` binds the module's public class (:issue:`1770`, :pr:`1778`). +* the data and schema packages re-export what their submodules list in ``__all__``, + under the prefixes their siblings use: ``pcapkit.protocols.data.link`` exports the + L2TP data model as ``L2TP`` and ``L2TP_Flags`` (:issue:`1797`, :pr:`1806`), and the + Mobility Header models, the HOPOPT and IPv6-Opts ``SMFDPDOption`` and + ``QuickStartOption``, PCAP-NG's ``PacketBlock`` and ``PACK_*`` options and HTTP/2's + ``Settings`` reach their packages and the aggregators (:issue:`1807`, :pr:`1815`). pcapkit.toolkit --------------- @@ -2418,7 +2468,7 @@ Added ~~~~~ * ``util/pyshark_encap_map.py``, a generator that regenerates - ``ENCAP_TYPE_TO_LINKTYPE`` (152 entries) and ``FILTER_NAME_TO_LINKTYPE`` (58) in + ``ENCAP_TYPE_TO_LINKTYPE`` and ``FILTER_NAME_TO_LINKTYPE`` in place inside ``pcapkit/toolkit/pyshark.py``, so the two tables :pr:`850` hand-built stop being hand-maintained. It lives under ``util/`` rather than ``pcapkit/vendor/`` because ``Vendor``'s base class is hardwired to an HTTP fetch and a generated @@ -2448,19 +2498,19 @@ Fixed it also read an ``ipv6_frag.nh`` that ``dpkt`` does not have, and passed fragment offsets unscaled (:issue:`351`, :issue:`370`, :pr:`385`, :pr:`395`). ``scapy`` never loaded its layer registry, so that engine did not dissect at all (:pr:`409`), and its - IPv4 fragment offset reached reassembly unscaled (:issue:`483`, :pr:`484`). The four - IPv6 adapters disagreed about whether the 8-octet Fragment header belongs to ``ihl``, - ``header`` and ``tl``; per :rfc:`8200` section 4.5 it belongs to none, and all four - now agree (:issue:`415`, :pr:`424`). + IPv4 fragment offset reached reassembly unscaled (:issue:`483`, :pr:`484`). The + IPv6 adapters disagreed about whether the 8-octet Fragment header belongs to + ``ihl``, ``header`` and ``tl``; per :rfc:`8200` section 4.5 it belongs to none, and + they now agree (:issue:`415`, :pr:`424`). * ``pypcapfile``'s ``IP.src``/``.dst`` are dotted-decimal ASCII text in a ``ctypes.c_char_p`` (per ``pypcapfile`` 0.12.0's ``ip.py``), not packed 4-byte values, so ``ipaddress.IPv4Address(ipv4.src)`` raised ``AddressValueError`` and ``ipv4_header()``'s ``struct.pack('...II', ipv4.src, ...)`` raised ``struct.error`` on - every call. A single ``_parse_ipv4_address()`` chokepoint replaces the four call - sites, also accepting packed ``bytes`` or an ``int``. The same engine's + every call. A single ``_parse_ipv4_address()`` chokepoint replaces the separate + call sites, also accepting packed ``bytes`` or an ``int``. The same engine's ``IP.opt``/``.payload`` carry hex-ASCII rather than raw bytes once decoding stops - short of the transport layer; a new ``_maybe_unhex()`` covers the three sites that - read them. Not ``breaking``: nothing on this path worked on 3.10/3.11, so correct + short of the transport layer; a new ``_maybe_unhex()`` covers the sites + that read them. Not ``breaking``: nothing on this path worked on 3.10/3.11, so correct behaviour cannot regress a working caller (:issue:`743`). * ``pypcapfile``'s ``_read_a_packet(layers=0)`` hexlifies a whole frame into ASCII text and stops, but ``_decode()`` fed that into ``Ethernet(packet.packet, layers=1)``, @@ -2483,14 +2533,14 @@ Fixed ``frame.encap_type``, which is distinct per encapsulation, and the fallback is gone. The two tables were measured rather than transcribed (Wireshark's source is not on the build host): each entry round-tripped through ``editcap -F pcap -T `` and read - back with ``tshark``. ``ENCAP_TYPE_TO_LINKTYPE`` grows to 152 entries with no DLT - serving two keys, and ``FILTER_NAME_TO_LINKTYPE`` from :pr:`838`'s two entries to 58, - every root filter name single-DLT across the 157 (of 226) encapsulations this host's - Wireshark can write. ``pcapkit/toolkit/pyshark.py`` reaches 100% coverage under new - tests that fail 26 ways against the unfixed tree; live pyshark parsing stays - unexercised, since ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). + back with ``tshark``. ``ENCAP_TYPE_TO_LINKTYPE`` grows to cover every measured + encapsulation with no DLT serving two keys, and ``FILTER_NAME_TO_LINKTYPE`` grows + from :pr:`838`'s hand-built entries, every root filter name single-DLT across the + encapsulations this host's Wireshark can write. + ``pcapkit/toolkit/pyshark.py`` reaches full coverage under new tests that fail + against the unfixed tree; live pyshark parsing stays unexercised, since ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). * ``pypcapfile``'s ``tcp_reassembly`` set ``last`` one past the segment. - ``TCP_Packet.last`` is inclusive, and the four other adapters set + ``TCP_Packet.last`` is inclusive, and the other adapters set ``seqnum + len(payload) - 1``, as this one now does too (:issue:`1100`). * scapy's ``packet2dict`` dumped nested packets as ``{}`` and mutated its input (:pr:`1287`); the dpkt reassembly adapters build their data from the record's own @@ -2519,14 +2569,14 @@ Added * ``pcapkit.utilities.logging`` as a real interface: ``get_logger()`` for per-module children, ``configure()`` to set level, handler, stream, format or propagation at - runtime, ``reset()`` to return to library-neutral, and ``ensure_output()``. Seventeen - modules now log under their own ``__name__``, so a consumer can silence + runtime, ``reset()`` to return to library-neutral, and ``ensure_output()``. Modules + now log under their own ``__name__``, so a consumer can silence ``pcapkit.foundation.registry`` while keeping ``pcapkit.foundation.extraction`` (:pr:`384`). * ``pcapkit.utilities.config``, one module for every tunable limit, each read once from a ``PCAPKIT_*`` environment variable: the frame-layer and extension-header limits, the pypcapfile engine's TCP-over-IPv6 warning limit, the dumpkit fallback, - the charset-detection cache, the read chunk and the three zero-padding guards. A + the charset-detection cache, the read chunk and the zero-padding guards. A bad value warns with ``ConfigWarning`` and falls back to the default; ``-1`` lifts a limit. The homes these limits had in the prereleases keep no re-export (:issue:`1726`, :pr:`1743`). @@ -2537,8 +2587,8 @@ Changed * ``pcapkit`` no longer configures logging at import. It installs a ``NullHandler`` and sets no level, so verbosity is inherited from the application instead of being seized by whichever library was imported second; the old stderr handler stays as the - ``PCAPKIT_DEVMODE`` opt-in. Three consequences: the previous behaviour is - ``configure(logging.INFO, stream=sys.stderr)``; 38 registry and extractor ``info`` + ``PCAPKIT_DEVMODE`` opt-in. Consequences: the previous behaviour is + ``configure(logging.INFO, stream=sys.stderr)``; the registry and extractor ``info`` calls became ``debug``, so those messages are invisible even at ``INFO``; and the handler is no longer ``logger.handlers[0]``. ``verbose=`` output stays on stdout and is not logging (:pr:`384`). @@ -2556,7 +2606,7 @@ Changed Fixed ~~~~~ -* two more gaps the :issue:`514` keyword audit turned up, neither covered by a test: +* further gaps the :issue:`514` keyword audit turned up, none covered by a test: ``StreamEOFError``'s docstring did not say that ``@prepare`` always raises it with ``quiet=True`` (the end-of-stream convention ``StructError`` follows via its own ``eof=True``), so nothing pinned that silence against regression; and @@ -2584,11 +2634,11 @@ pcapkit.vendor Fixed ~~~~~ -* thirteen ``re.sub`` sites under ``pcapkit/vendor/`` passed ``re.MULTILINE`` as the +* ``re.sub`` sites under ``pcapkit/vendor/`` passed ``re.MULTILINE`` as the fourth *positional* argument, which is ``count``, not ``flags``: substitution capped at 8 matches, and a ``DeprecationWarning`` on Python 3.13+ (a future ``TypeError``). - Found by an AST sweep rather than grep, since 8 of the 13 put the flag on a - continuation line. The flag moves to ``flags=`` at all 13 sites (``default.py`` plus + Found by an AST sweep rather than grep, since most put the flag on a continuation + line. The flag moves to ``flags=`` at every site (``default.py`` plus one each in ``hip/eddsa_curve.py``, ``http/frame.py``, ``http/method.py``, ``http/status_code.py``, ``ipv4/router_alert.py``, ``ipv6/option.py``, ``ipv6/router_alert.py``, ``ipv6/tagger_id.py``, ``reg/ethertype.py``, @@ -2607,20 +2657,21 @@ Fixed ``_unregistered_member``, which deliberately does not register, so ``cls(value)`` missed and re-entered ``_missing_``. :pr:`861` fixed the *generated* modules but not the two *crawlers*, so :pr:`861`'s CI stayed green until a live regeneration - reintroduced the defect; :pr:`861`'s review had sampled 5 of 82 regenerations, - missing these two. Both crawlers now collapse to the single returning line the other - 101 registries use, and the two const files are regenerated. A new guard at the + reintroduced the defect; :pr:`861`'s review had sampled only some + regenerations, missing these two. Both crawlers now collapse to the single returning + line the other registries use, and the two const files are regenerated. A new guard + at the crawler-emission layer requires a hard-coded ``_missing_`` body to end in a return and no emitted line to re-enter ``_missing_`` for the same value -- deliberately narrower than "every emitted line must return", which ``Vendor.process``'s shape - legitimately breaks. The guard sees only 35 of 96 crawlers; the other 61 are covered + legitimately breaks. The guard sees only some crawlers; the rest are covered behaviourally. Closes :issue:`866` (:pr:`867`). * ``python -m pcapkit.vendor `` could not fail: ``run()`` swallowed every crawler exception as a filterable ``VendorRuntimeWarning`` and ``main()`` always returned ``0``, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about - failed sub-vendors without failing as a whole -- three pieces land. + failed sub-vendors without failing as a whole -- the pieces below land. **Exit code** (``pcapkit/vendor/__main__.py``): ``run()`` returns whether its target succeeded; ``main()`` attempts every target and returns ``1`` if any raised, writing the failing target's qualified name and @@ -2644,8 +2695,8 @@ Fixed ``0``. A per-target ``git checkout -- `` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes :issue:`872` (:pr:`873`). -* three follow-ups from :pr:`873`'s review, none changing merged behaviour: one - unpinned invariant and two inaccurate pieces of prose/lint. **1.** +* follow-ups from :pr:`873`'s review, none changing merged + behaviour: an unpinned invariant and inaccurate prose and lint. **1.** ``except BaseException:`` in ``_snapshot_and_restore`` is pinned: a test asserts that a ``KeyboardInterrupt`` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose ``_dest_path`` is an @@ -2661,10 +2712,10 @@ Fixed since ``copy2`` reopens the path) and the semantically identical unindented form of the same ``else:``. Closes :issue:`875` (:pr:`876`). * ``TLSKeyLabel`` tracked the Mozilla NSS key-log wiki, but - ``draft-ietf-opsawg-pcapng-06`` S4.7 now points at ``[I-D.ietf-tls-keylogfile]`` - (draft-05), published as :rfc:`9850`, "The SSLKEYLOGFILE Format for TLS", whose S4.2 + ``draft-ietf-opsawg-pcapng-06`` section 4.7 now points at ``[I-D.ietf-tls-keylogfile]`` + (draft-05), published as :rfc:`9850`, "The SSLKEYLOGFILE Format for TLS", whose section 4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, ``TLSKeyLabel`` - was missing ``ECH_SECRET`` and ``ECH_CONFIG`` (:rfc:`9850` SS2.3/4.2), so + was missing ``ECH_SECRET`` and ``ECH_CONFIG`` (:rfc:`9850` sections 2.3 and 4.2), so ``TLSKeyLabel('ECH_SECRET')`` raised ``ValueError`` on a legitimate Decryption Secrets Block; both are added, ``ECH_SECRET`` keeping the file's ``# nosec B105`` bandit convention for ``*_SECRET`` members. ``RSA`` has no row in :rfc:`9850`'s @@ -2732,7 +2783,7 @@ Added path as untested as the broken one. The generator writes ``big_endian.pcap`` (magic ``a1 b2 c3 d4``), ``big_endian_nanosecond.pcap`` (``a1 b2 3c 4d``, the first fixture to take that branch of the magic-number table) and ``little_endian.pcap`` - (``d4 c3 b2 a1``), carrying the *same three records* in each, so tests can assert + (``d4 c3 b2 a1``), carrying the *same records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet ``snaplen``), so ``incl_len`` and ``orig_len`` differ and cannot both be @@ -2740,8 +2791,8 @@ Added three through ``extract()`` and derives expectations by walking each file's record chain with ``struct``; a unit-tier case in ``test_header_frame_unit.py`` builds a two-record big-endian capture in memory, so the fixture-free selection CI runs on - every push catches it too. All four fail on the unfixed tree (the three - fixture-backed by that ``ValueError``, the in-memory one by + every push catches it too. Each fails on the unfixed tree (the + fixture-backed ones by that ``ValueError``, the in-memory one by ``AssertionError: 3106905 != 1500000000``) while the little-endian twin passes on both, showing the records themselves are not the variable (:issue:`605`). @@ -2752,13 +2803,13 @@ Changed ``examples/samples`` -- one letter apart, holding different things -- are now ``examples/captures`` and ``examples/generators``. * the README is a landing page now, and Markdown rather than reStructuredText. - ``README.rst`` (424 lines) became ``README.md`` (102), keeping what a reader arriving + ``README.rst`` became a much shorter ``README.md``, keeping what a reader arriving from PyPI or a search result needs (what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, where the documentation lives). **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in ``docs/source/index.rst``, so they are linked rather than - restated; the two blocks that existed nowhere else moved: **Testing** is now + restated; the blocks that existed nowhere else moved: **Testing** is now ``docs/source/contributing/testing.rst``, and the ``pipenv`` and ``make setup`` block joined the Installation section of ``docs/source/index.rst``. Requested by the project owner, and a deliberate exception to the reStructuredText convention, for the @@ -2773,48 +2824,51 @@ Changed * ``CODE_OF_CONDUCT.md`` moves from Contributor Covenant 1.4 to 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md - rather than transcribed. Three things needed deciding. 3.0's two ``[NOTE`` + rather than transcribed. Some things needed deciding. 3.0's ``[NOTE`` placeholders are filled or removed: the reporting channel names ``jarryshaw@icloud.com`` (the contact 1.4 carried and ``SECURITY.md`` uses as email fallback) plus GitHub's report-abuse form for a report about the maintainer, which a single-maintainer project cannot otherwise cover; the enforcement placeholder is an instruction to the adopter and is removed. 3.0's plural "Community Moderators" become - the singular maintainer (eight occurrences), this repository having no moderators. - The four-rung ladder (Warning, Temporarily Limited Activities, Temporary Suspension, - Permanent Ban) is a suggestion and is **kept**, each rung mapping to a lever one + the singular maintainer, this repository having no moderators. + The enforcement ladder (Warning, Temporarily Limited Activities, Temporary + Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a + lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so the attribution names version 3.0, links the permanent ``version/3/0/`` URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the share-alike term covers this document only (the code remains BSD-3-Clause, ``LICENSE`` - untouched). The ladder renders as four list items each nesting three, checked against + untouched). The ladder renders as nested list items, checked against GitHub's Markdown API; :pr:`613` had to repair the 1.4 file when a stray list marker collapsed it into one nested item (:pr:`624`). * reconciled which private (``_xxx``) attributes and methods the Sphinx build documents, replacing an ad hoc mix with one rule: document the contract, hide the recipe. A member every subclass must implement, or whose shape a caller depends on, stays documented despite its underscore; a private helper that only keeps one method - short does not. Seven module-level directives for pure implementation detail + short does not. Module-level directives for pure implementation detail (``esp._resolve``, ``esp._CRYPTO``, ``ngap._convert``, ``ngap._revert``, ``ngap._PYCRATE``, ``ngap._PDU_LOCK``, ``pypcapfile._NamedStream``) were dropped, - while 39 ``autoattribute`` directives were added for class-private state that *is* + while ``autoattribute`` directives were added for class-private state that *is* contract, among them ``Extractor._flag_f``, the ``PyPCAP`` and ``PCAP_CT`` engines' - ``_backend`` (the only two of the six third-party engines with one), ``TraceFlow``'s + ``_backend`` (the only third-party engines with one), ``TraceFlow``'s internal fields, ``FieldBase``/``Field`` internals, the built-in ``PCAP`` engine's ``_gbhdr``, ``_vinfo``, ``_nnsec`` and ``_dlink`` (shared with ``PyPCAP``, - ``PCAP_CT`` and ``PyPCAPFile``) and ``PCAPNG``'s ``_ctx``/``_ctx_list``. The 128 - runtime definitions of ``_missing_`` (121 under ``pcapkit.const``, 7 inline in + ``PCAP_CT`` and ``PyPCAPFile``) and ``PCAPNG``'s ``_ctx``/``_ctx_list``. The + runtime definitions of ``_missing_`` (under ``pcapkit.const``, and inline in ``pcapkit.protocols``) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in ``docs/source/pcapkit/const/index.rst`` cross-referenced from ``registry.rst`` (``conf.py`` already excludes ``_missing_`` from every ``autoclass``). ``CONTRIBUTING.md`` gained the rule as a named section. - The one new Sphinx warning, a bare ``Type`` newly rendered by ``TraceFlow._foutio``'s - directive, is :issue:`709`. No line under ``pcapkit/`` changed (:issue:`684`). + The new Sphinx warning, a bare ``Type`` newly rendered by + ``TraceFlow._foutio``'s directive, is :issue:`709`. No line under ``pcapkit/`` + changed (:issue:`684`). * ``examples/captures/out.json``, ``out.plist``, ``out.txt`` and ``pcapng.txt`` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. ``pcapng.txt`` is exactly that: it still recorded - ``packet -> NIL`` for the four Enhanced Packet Blocks of ``dhcp.pcapng`` long after - :pr:`683` gave those blocks their captured octets back. Rather than regenerate it again - and leave the drift free to recur, the four files are removed from the index and folded + ``packet -> NIL`` for the Enhanced Packet Blocks of ``dhcp.pcapng`` long after + :pr:`683` gave those blocks their captured octets back. Rather than regenerate it + again and leave the drift free to recur, these files are removed from the index and + folded into ``examples/captures/``'s blanket ``.gitignore`` rule; the genuine inputs ``in.pcap`` and ``dhcp.pcapng`` stay tracked. ``examples/legacy_smoke/Makefile`` and its ``README.md`` now describe regenerating these reports via ``make fixtures``, and @@ -2829,21 +2883,21 @@ Changed ``isort``, and ``isort`` joins the ``test`` extra rather than being excluded (``lint.yml`` installs none, so unlike ``mypy`` there is no existing check to defer to). No workflow changed, every pytest job already building on ``.[test]``; - ``gated_scopes()`` goes 250 to 251 and ``dependency_gate_gaps()`` holds at 18. The - third instance of the hazard :issue:`745` exists to stop (:issue:`766`). + ``gated_scopes()`` gains this scope and ``dependency_gate_gaps()`` is unchanged. + Another instance of the hazard :issue:`745` exists to stop (:issue:`766`). * ``tests/vendor/test_vendor_reg_apptype_generator_unit.py`` gated on ``mypy`` with no ``MODULE_PROVIDERS`` entry, so ``_top_level_providers()`` died on a bare ``KeyError: 'mypy'`` once the gate became visible -- a further instance of the hazard :issue:`745` exists to stop, like :issue:`766` above. - ``MODULE_PROVIDERS['mypy'] = ('mypy',)`` fixes the lookup, and the three bare + ``MODULE_PROVIDERS['mypy'] = ('mypy',)`` fixes the lookup, and the bare ``MODULE_PROVIDERS[...]`` subscripts route through that function, which raises an ``AssertionError`` naming the module, key and table on any future gap. ``mypy`` is deliberately not added to a ``pyproject.toml`` extra: no extra carries it, it is a ``Pipfile`` dev-dependency that ``lint.yml`` installs and already runs whole-package, and ``lint.yml`` runs no pytest job for ``pytest_jobs()`` to see, so the fix is a ``DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']`` entry on ``test``, - ``engine-tests`` and ``gate``. ``dependency_gate_gaps()`` goes 15 to 18 - ``(flag, job)`` pairs, all three new and expected; (:issue:`779`). + ``engine-tests`` and ``gate``. ``dependency_gate_gaps()`` gains + the new ``(flag, job)`` pairs, each expected; (:issue:`779`). * ``engine='pyshark'`` had no CI job driving it through a real ``tshark``-parsed capture. ``pypcap-parity`` installs ``pyshark`` but its ``apt-get install`` named no ``tshark`` package, so ``PyShark.unsupported_reason()`` declined the engine, the @@ -2888,15 +2942,17 @@ Changed (:pr:`856`). * the prose the post-:issue:`719` defect wave added or touched, checked against the code; no code changes. ``MesgIDOption.seconds`` in the Mobility Header schema said - seconds since 1970 where it holds an :rfc:`1305` NTP count from 1900; two - ``HTTP._guess_version`` comments predated the HTTP/2 gate; and the + seconds since 1970 where it holds an :rfc:`1305` NTP count from 1900; + ``HTTP._guess_version`` + comments predated the HTTP/2 gate; and the ``_parse_next_layer`` note miscounted the captured octets. ``Extractor._open_output`` now says only the PCAP engine passes header fields, ``Extractor.run`` and - the PCAP-NG engine's ``run`` list the ``FormatError`` for PCAP output from a PCAP-NG input, and + the PCAP-NG engine's ``run`` list the ``FormatError`` for PCAP output from a PCAP-NG + input, and the extraction and engine pages document the PCAP-output path (:pr:`1179`, :pr:`1180`, :pr:`1186`, :issue:`1103`). * **a breaking change to** the ``all`` extra: it carries the core addons only -- - ``cli``, ``crypto`` and ``NGAP``. The three third-party engines 1.4.x bundled in it + ``cli``, ``crypto`` and ``NGAP``. The third-party engines 1.4.x bundled in it (``dpkt``, ``scapy`` and ``pyshark``) and the vendor crawler's ``requests`` and ``beautifulsoup4`` are no longer in it; the engines keep their own ``DPKT``, ``Scapy`` and ``PyShark`` extras and the crawler its ``vendor`` extra, and @@ -2911,43 +2967,50 @@ Changed ``bdist_wheel`` override, since ``build_py`` and ``install`` already convert the tree the wheel archives, and ``MANIFEST.in``'s ``prune test``, which named no directory; ``make profile`` runs in ``examples/legacy_smoke`` (:issue:`1773`, - :pr:`1780`). + :pr:`1780`). ``MANIFEST.in`` also drops the lines that matched nothing -- + ``global-include *.rst``, ``include pcapkit/_extern`` and + ``exclude pcapkit/protocols/*/NotImplemented`` -- and the source distribution and wheel are + unchanged (:issue:`1794`, :pr:`1813`). +* the contributor pages state no counts that a test or the Python version updater + pinned, and each pin checks the fact it stood for instead (:issue:`1811`, + :pr:`1817`). Fixed ~~~~~ -* 45 places where a documentation page contradicted the code (:pr:`413`), ambiguous - cross-references and five autodoc signature failures (:pr:`416`), and - ``Extractor``'s documented exception plus 40 phantom or stale ``Args:`` labels +* places where a documentation page contradicted the code (:pr:`413`), + ambiguous cross-references and autodoc signature failures (:pr:`416`), and + ``Extractor``'s documented exception plus phantom or stale ``Args:`` labels (:pr:`501`). * both halves of what the ``Changelog drift`` gate told an author, in ``util/changelog_md.py``. ``ResidualMarkupError`` named the entry file but numbered - its lines against the *converted* body, which rule 6 joins onto one line per block (55 - lines against the file's 580 on this entry), so a reported number could not reach most - of the file, and four roles on two source lines were all reported as ``line 46``. The + its lines against the *converted* body, which rule 6 joins onto one line per block, + so a reported number could not reach most of the file, and roles on separate source + lines were all reported as one line. The conversion now carries a source map, so every complaint cites a line of the file the message names, one per construct, in the entry's order (:issue:`588`). Rule 2 accepted only the bare-number spelling of Sphinx's ``:rfc:`` role, so :rfc:`6554#section-3` fell past it and was reported as a role the rules do not cover; both spellings now convert, the anchor carried into the link target and the link text taken from - Sphinx's own (:issue:`592`). Regenerating ``CHANGELOG.md`` is byte-identical over all - 37 committed entries: the fix changes what the gate *says*, not what it emits. -* ``examples/generators/options.py``'s ``TCP_BASE`` spelled three header fields with + Sphinx's own (:issue:`592`). Regenerating ``CHANGELOG.md`` is byte-identical over + every committed entry: the fix changes what the gate *says*, not what it emits. +* ``examples/generators/options.py``'s ``TCP_BASE`` spelled header fields with names ``TCP.make`` does not declare, so every option fixture in ``examples/captures/options-tcp.pcap`` was built from values the generator had not asked for. ``make`` ends in ``**kwargs`` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. ``'seq': 1`` was - ``seq_no``, so all 25 captured frames carried sequence number 0; ``'urgent_pointer'`` + ``seq_no``, so every captured frame carried sequence number 0; ``'urgent_pointer'`` was ``urgent`` (requested value and default both 0); and ``'ack_flag': False`` was ``ack`` while ``'ack': 0`` bound to ``ack`` itself, the acknowledgement *flag* rather than the number (``ack_no``), so that pair was wrong both ways. Each is renamed to the parameter ``make`` declares, keeping the intended values. ``TCP(**TCP_BASE).info.seq`` was 0 against a declared 1 and is now 1, and - ``options-tcp.pcap`` changes in exactly 25 bytes (the low octet of each frame's - sequence number), with no case status or ``EXPECTED_FAILURES`` entry moving. The - module's six other base mappings were audited: five pass nothing their ``make`` does + ``options-tcp.pcap`` changes only in the low octet of each frame's sequence number, + with no case status or ``EXPECTED_FAILURES`` entry moving. The + module's other base mappings were audited: the rest pass nothing their ``make`` does not declare, and HIP's ``extension`` is declared by ``HIP.read``, this library's - documented way of forwarding a read-path keyword, not a sixth instance (:issue:`602`). + documented way of forwarding a read-path keyword, not another instance + (:issue:`602`). * ``util/bump_version.py`` left ``CITATION.cff`` naming the previous release. Nothing else maintains that file, so every bump since :pr:`615` would have stranded the ``version`` and ``date-released`` that GitHub's "Cite this repository" button, @@ -2956,21 +3019,23 @@ Fixed *published* release and moving only one would assert that 1.5.0b5 was released on the day 1.5.0b4 was. The date is UTC, because seven of the thirty most recent bumps were made late evening US-Eastern, where a local date is a day behind the publish; the bump - triggers ``create-release.yml``, the median gap to the PyPI upload is three minutes, and + triggers ``create-release.yml``, the median gap to the PyPI upload is three minutes, + and the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment header, key ordering and each field's quoting survive (``cff-version`` and a - ``references`` entry's own ``version`` are anchored out at column zero), and the result + ``references`` entry's own ``version`` are anchored out at column zero), and the + result is checked with ``cffconvert --validate``. An absent file is reported on stderr and skipped rather than failing the vendor cron before its ``git commit``, which would discard a whole registry crawl for a documentation file; a file with no ``version`` field raises before anything is written, since reporting success while rewriting nothing is the staleness being fixed. The script gains a ``main()`` guard (it ran the - whole bump at import, hence no testable surface), and the ``import pcapkit`` fallback in + whole bump at import, hence no testable surface), and the ``import pcapkit`` + fallback in its version reader, which returned ``"1.5.0b4'\n"`` (closing quote and newline, which ``packaging`` rejects), is fixed; it never worked and went unnoticed because the only caller installs the package first. A new gate asserts the committed file still names the packaged version, covering hand-made version changes that never run this script - (40 of the 159 commits that moved ``__version__`` on ``main``, 11 of the latest 25) (:pr:`625`). * ``LICENSE``'s copyright notice began the term at 2018, a year after the work it covers. The repository's first commit is ``c57f7d0b7``, dated 2017-11-07, so the @@ -2993,12 +3058,15 @@ Fixed nothing, and automation was considered and rejected, since automating a value that need not be current is worse than not carrying it. ``conf.py`` is untouched (a self-maintaining footer showing a range is conventional and correct). The licence - body is unaltered: one hunk at line 3, and the ``DAMAGE.`` that :pr:`615` repaired + body is unaltered: one hunk, on the copyright line, and the ``DAMAGE.`` that + :pr:`615` repaired from ``DAMAGE.s`` is still clean (:pr:`630`). * ``CITATION.cff`` shipped in no source distribution. ``MANIFEST.in`` has an ``include`` - each for ``README.md``, ``LICENSE`` and ``CHANGELOG.md`` but none for the citation file, - and its two ``global-include`` patterns (``*.rst``, ``*.py``) cannot match a ``.cff``. - Removing all three ``include`` lines and rebuilding shows ``README.md`` and ``LICENSE`` + each for ``README.md``, ``LICENSE`` and ``CHANGELOG.md`` but none for the citation + file, + and its ``global-include`` patterns (``*.rst``, ``*.py``) cannot match a + ``.cff``. Removing those ``include`` lines and rebuilding shows ``README.md`` and + ``LICENSE`` shipping anyway (setuptools adds the latter from ``license_files``, recorded as ``License-File: LICENSE`` in ``PKG-INFO``) while ``CHANGELOG.md`` vanishes, so only ``CHANGELOG.md`` is load-bearing, and a citation file, which no packaging default @@ -3010,11 +3078,14 @@ Fixed ``date-released`` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets ``RepositoryCitationTests`` in ``tests/project/test_bump_version.py`` (the gate :pr:`625` added for hand-authored - bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is not - shipped in the source distribution". ``python -m build --sdist`` gives 860 archive - entries before and 861 after (``pypcapkit-1.5.0b4/CITATION.cff`` the only difference), the - shipped copy is byte-identical, and ``twine check --strict`` passes on both (:pr:`631`). -* two inaccurate claims in the :pr:`630` entry above, caught in review after :pr:`630` + bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is + not + shipped in the source distribution". ``python -m build --sdist`` gives + one more archive entry after (``pypcapkit-1.5.0b4/CITATION.cff`` the only + difference), the + shipped copy is byte-identical, and ``twine check --strict`` passes on both + (:pr:`631`). +* inaccurate claims in the :pr:`630` entry above, caught in review after :pr:`630` had merged; both concern the history of ``LICENSE``, which is untouched here. 2017 was and remains the right answer, so this is a changelog-prose correction. The entry said the file "until then was stock MPL text carrying no author notice at all, so 2018 is @@ -3023,45 +3094,54 @@ Fixed description of the era: the pre-BSD file was MIT, then GPL v3, then Apache 2.0, then MPL 2.0; the MIT and GPL texts both carried an author notice naming **2017**; and ``2018`` first appeared in the Apache notice of 2018-12-08 as a single year, current - when written. The claim also contradicted the entry's own dating of the first commit to - 2017-11-07. The history supports a stronger argument than the one written, which is why - this is corrected rather than deleted: 2017 is the year asserted from the first commit, - and :pr:`630` restores it. The ``BSD 3-Clause License`` title line is reattributed from + when written. The claim also contradicted the entry's own dating of the first commit + to + 2017-11-07. The history supports a stronger argument than the one written, which is + why + this is corrected rather than deleted: 2017 is the year asserted from the first + commit, + and :pr:`630` restores it. The ``BSD 3-Clause License`` title line is reattributed + from opensource.org's template to choosealicense.com's, whose first line is that title and which carries the ``Copyright (c) [year], [fullname]`` comma this file uses. The error came from generalising the pre-BSD era off two blobs that were the same file - (``bc836cfa2^:LICENSE`` and ``1c69341dc:LICENSE``); enumerating all 14 commits that + (``bc836cfa2^:LICENSE`` and ``1c69341dc:LICENSE``); enumerating every commit that touched ``LICENSE`` caught it (:pr:`638`). * the release workflow published to PyPI and to Anaconda with nothing a human had to approve. ``create-release.yml``'s ``pypi`` job had ``environment: release`` commented out while the ``id-token: write`` from the same upstream snippet was re-added live below it, so the block read as disabled as a unit when only its gate still was; the ``conda`` job had no ``environment:`` at all. Neither needed a tag: the workflow also - fires on completion of ``Vendor Update``, so a scheduled registry crawl that bumped the - version was enough to publish to two public indexes, a median three minutes from bump to - upload. Four jobs reach outside the run and all four are now gated: ``github`` on + fires on completion of ``Vendor Update``, so a scheduled registry crawl that bumped + the + version was enough to publish to two public indexes, a median three minutes from + bump to + upload. Every job that reaches outside the run is now gated: ``github`` on ``github-release`` (the GitHub Release and the ``v*`` tag), ``tag`` on ``conda-tag`` - (the commit pushed to ``main``), ``pypi`` on ``pypi``, ``conda`` on ``anaconda``. One - environment per credential rather than a shared ``release``, because approval is - granted to an environment and not a job, so one ``release`` approved once would release - both indexes together. PyPI has to be answerable alone: an accepted version number - cannot be reused, and ``skip-existing: true`` makes the re-upload *succeed* having - published nothing, so an unintended publish permanently consumes the number the - intended release wanted. The workflow half is only half the fix: an ``environment:`` - naming one that does not exist is created implicitly with no protection rules and the - job proceeds unapproved, so this is inert until each of the four exists in repository - settings with a required reviewer, and each one's "Deployment branches and tags" must + (the commit pushed to ``main``), ``pypi`` on ``pypi``, ``conda`` on ``anaconda``. + One environment per credential rather than a shared ``release``, because approval is + granted to an environment and not a job, so one ``release`` approved once would + release both indexes together. PyPI has to be answerable alone: an accepted version + number cannot be reused, and ``skip-existing: true`` makes the re-upload *succeed* + having published nothing, so an unintended publish permanently consumes the number + the intended release wanted. The workflow half is only half the fix: an + ``environment:`` naming one that does not exist is created implicitly with no + protection rules and the job proceeds unapproved, so this is inert until each of + these environments exists in repository settings with a required reviewer, and each + one's "Deployment branches and tags" must stay unrestricted or the ``tags: v*`` trigger fails outright instead of pausing. ``github-pages`` is the standing example, carrying no protection rule since 2021. The new test asserts the general rule, that no job running a publishing action or a - ``git push`` may omit an ``environment:``, so a publishing job added later without a gate + ``git push`` may omit an ``environment:``, so a publishing job added later without a + gate fails rather than ships (:issue:`641`). -* three test comments stated an exact count of committed captures under +* test comments stated an exact count of committed captures under ``examples/captures/``, a number that drifts whenever the tracked set changes and in - one case was wrong when written: ``tests/_tiers.py`` said "six," ``tests/test_tier_guard.py`` - "the moment a seventh capture is committed," and ``tests/integration/_helpers.py`` - "four," which undercounted on the day it was written. All three now describe the - invariant (``_tiers.py``: "the moment somebody commits another capture... or stops + one case was wrong when written: ``tests/_tiers.py`` said "six," + ``tests/test_tier_guard.py`` "the moment a seventh capture is committed," and + ``tests/integration/_helpers.py`` "four," which undercounted on the day it was + written. Each now describes the invariant (``_tiers.py``: "the moment somebody + commits another capture... or stops committing one"; ``_helpers.py``: "...are fixtures -- some of them committed --..."; ``test_tier_guard.py`` drops its ordinal). Prose-only, verified with ``coverage.parser.PythonParser`` that no executable statement moved (:issue:`700`). @@ -3079,7 +3159,7 @@ Fixed to match the tracked names still passes; the test catches a set that is wrong when it runs (:issue:`708`). * ``pcapkit.all.__all__`` listed ``OSPF``, ``RARP`` and ``DRARP`` on its Link Layer - line after :issue:`719` moved them (:issue:`1124`), and omitted 18 names its packages + line after :issue:`719` moved them (:issue:`1124`), and omitted names its packages export, among them ``ESP``, ``HTTPv1``, ``HTTPv2``, ``PCAPNG`` and ``EnumRegistry``, so ``from pcapkit.all import *`` did not bind them. ``pcapkit.utilities`` stays unlisted deliberately, so generic helpers such as ``warn`` do not arrive through @@ -3098,13 +3178,28 @@ Fixed (:issue:`1649`, :pr:`1669`), and keeps the release's name and notes when it uploads (:issue:`1657`, :pr:`1678`). Vendor Update commits the new modules a crawl adds (:issue:`1685`, :pr:`1688`). -* package ``__all__`` lists match what each package holds: six disagreed with it, +* package ``__all__`` lists match what each package holds: several disagreed with it, and ``from pcapkit.protocols.data import *`` raised ``AttributeError`` on the unbound ``HOPOPT_QuickStartOption`` (:issue:`515`, :pr:`527`); the public attributes several packages left out are listed (:issue:`533`, :pr:`544`). -* the documentation build's 36 "more than one target found" warnings, fixed where +* the documentation build's "more than one target found" warnings, fixed where they arise: an implicit reference to ``dict``, ``type`` or ``set`` no longer matches a registry member such as ``TCP.dict``, an enum member's docstring holding a URL no longer gains a spurious type field, and the hand-written ``TypeVar`` directives take the bare name under their ``.. module::`` (:issue:`1775`, :pr:`1790`). +* the Mermaid diagrams render at their natural size, shrunk only to the page width, + where each was fitted into a fixed box, and their ``click`` targets are relative + to the page, so they resolve under the site's ``/PyPCAPKit/`` prefix + (:issue:`1103`, :pr:`1819`). +* ``pcapkit-cli``'s help strings say what each option does: ``-o`` describes the + output rather than the input, an extension is appended only under + ``--auto-extension``, and each format flag names the flags that take precedence + over it (:issue:`1796`, :pr:`1810`). +* the Conda package: its vendored dependencies are the package's ``dependencies`` + and its ``all`` extra, with their requirements, shipped with their data files + rather than as ``.py`` files only (:issue:`1814`, :pr:`1821`); the ``osx-64`` + package is built on an Intel runner, where converting the arm64 build shipped arm64 binaries + (:issue:`1822`, :pr:`1825`); and on Intel macOS it pins ``cryptography`` 48.0.1, + which has a wheel for that platform where the newer pin has none (:issue:`1827`, + :pr:`1828`).