diff --git a/CHANGELOG.md b/CHANGELOG.md index 911f20311..53955a869 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ ## 1.5.0 -- unreleased -The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#1828](https://github.com/JarryShaw/PyPCAPKit/pull/1828). +The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#1895](https://github.com/JarryShaw/PyPCAPKit/pull/1895). Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and `1.5.0b4` to `1.5.0b13` (2026-09-20 to 2026-10-16), dated in UTC and all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` reshipped it -- an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels ([#497](https://github.com/JarryShaw/PyPCAPKit/pull/497), [#498](https://github.com/JarryShaw/PyPCAPKit/pull/498)). @@ -11,22 +11,23 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - **a breaking change to** `pcapkit.const.reg.apptype`. `AppType` becomes a memberless base over one registry per transport protocol -- `TCP`, `UDP`, `SCTP` and `DCCP` -- because `aenum` refuses to subclass an enumeration that has members. `isinstance(TCP.http, AppType)` still holds and `AppType.get(port, proto=...)` proxies to the registry, so nothing under `pcapkit.protocols` changes. **The rows IANA assigns no port, and those it assigns no transport protocol, stop being members**; they become list-table docstrings on the registry that owns them. `__members_proto__` is now a per-registry `MultiDict` keyed on port, so every service IANA registers on port 80 survives. `get()` answers with the canonical service, curated from `/etc/services` because IANA names no precedence, and `get_all()` reaches the aliases ([#754](https://github.com/JarryShaw/PyPCAPKit/pull/754)). - the `%`-formatted `raise ValueError(...)` calls in the generated `pcapkit.const.reg.apptype.apptype`, left by [#759](https://github.com/JarryShaw/PyPCAPKit/issues/759), are f-strings ([#792](https://github.com/JarryShaw/PyPCAPKit/issues/792)). The remainder followed: the `_missing_` guards in `pcapkit.vendor.tcp.flags`, `.ftp.command` and `.http.method`, `AppType`'s `__new__`/`__repr__`/`__str__`, and the `extend_enum` fallbacks in `get()` and in the span-handling tail. The `consider-using-f-string` disable drops from `tcp.flags` and `reg.apptype.apptype` and stays on `ftp.command` and `http.method`, whose `__repr__` still used `%`. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows ([#798](https://github.com/JarryShaw/PyPCAPKit/issues/798)). - the `__repr__` methods [#798](https://github.com/JarryShaw/PyPCAPKit/issues/798) left on `%` (in `pcapkit/const/ftp/command.py` and `pcapkit/const/http/method.py`) are f-strings, so the disable drops from both files and both `pcapkit/vendor/` templates. The other bespoke templates (`{const,vendor}/ftp/return_code.py`, `{const,vendor}/http/status_code.py`, `vendor/http/frame.py`) and `vendor/default.py` still carry it, so [#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)'s claim that nothing in `pcapkit/{const,vendor}/{ftp,http}/` needs it holds for this pair only; `repr()` output is unchanged ([#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)). -- `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)'s (above) preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a `'..._0x%s' % hex(value)[2:].upper().zfill(4)` expression, and the exact `exempt_hits == 53` bound stays. Test-only. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). +- `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for the hex-suffixed names [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878) (below) preserves moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a `'..._0x%s' % hex(value)[2:].upper().zfill(4)` expression, and the exact `exempt_hits == 53` bound stays. Test-only. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). - **a breaking change to** `AppType.proto`: it is the single transport of the registry the member lives in, not every transport IANA assigned the service, and `register_apptype` takes its transports as varargs ([#806](https://github.com/JarryShaw/PyPCAPKit/issues/806), [#815](https://github.com/JarryShaw/PyPCAPKit/pull/815)). Aliases show in `repr()` and `str()`, and can be registered per transport ([#807](https://github.com/JarryShaw/PyPCAPKit/issues/807), [#837](https://github.com/JarryShaw/PyPCAPKit/pull/837)). - **a breaking change to** `TransportProtocol`: its base moves from `aenum.IntFlag` to `aenum.IntEnum`, so `|` between two members gives a plain `int` rather than a composite member ([#808](https://github.com/JarryShaw/PyPCAPKit/issues/808), [#836](https://github.com/JarryShaw/PyPCAPKit/pull/836)). - **a breaking change to** `TLSKeyLabel`: the hand-written class in `pcapkit.protocols.misc.pcapng` becomes the generated `pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` (`PCAPNG_TLSKeyLabel` in `pcapkit.const.pcapng`), crawled from the IANA registry [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) creates ([#886](https://github.com/JarryShaw/PyPCAPKit/issues/886), [#890](https://github.com/JarryShaw/PyPCAPKit/pull/890)), and the old path is removed ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). - **a breaking change to** `Method.get`: the method token is case-sensitive, per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so `Method.get('get')` no longer resolves to `GET` ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), [#907](https://github.com/JarryShaw/PyPCAPKit/pull/907)). It also looks a key up by value, so `BASELINE-CONTROL` and `VERSION-CONTROL` resolve ([#908](https://github.com/JarryShaw/PyPCAPKit/issues/908), [#915](https://github.com/JarryShaw/PyPCAPKit/pull/915)). - **a breaking change to** `ExtensionHeader`: it is generated from IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers registry's flag, and the two disagree on 147 (`BIT-EMU`) ([#925](https://github.com/JarryShaw/PyPCAPKit/issues/925), [#926](https://github.com/JarryShaw/PyPCAPKit/pull/926)). - the TCP and UDP `AppType` registries load on first use rather than at import ([#1538](https://github.com/JarryShaw/PyPCAPKit/issues/1538), [#1560](https://github.com/JarryShaw/PyPCAPKit/pull/1560)). +- **a breaking change to** `LLACode`: `Wilcard` is spelled `Wildcard` and `MH` is `MN`, the names [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) gives codes 0 and 2, with no alias for either old name ([#1848](https://github.com/JarryShaw/PyPCAPKit/issues/1848), [#1872](https://github.com/JarryShaw/PyPCAPKit/pull/1872)). #### Fixed - constant lookups that rejected a value the registry defines. `RouterAlert(0)` is the only value [RFC 2113](https://datatracker.ietf.org/doc/html/rfc2113) defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPX `Socket(0)` is that protocol's own default, so `bytes(IPX(...))` crashed on its own defaults; and the FTP `_missing_` overrides were plain methods rather than classmethods, so every unregistered value raised `TypeError` instead of extending the enumeration ([#492](https://github.com/JarryShaw/PyPCAPKit/issues/492), [#503](https://github.com/JarryShaw/PyPCAPKit/pull/503)). -- the string-keyed `get()` in `pcapkit.const.ftp.command` and `pcapkit.const.http.method` tested membership with the raw key but registered `key.upper()`, so the *first* lowercase or mixed-case token raised `TypeError: 'RETR' already in use`. Reachable from wire data: `pcapkit.protocols.application.ftp` matches with `re.I` and passes the match verbatim, and [RFC 959 Section 5](https://datatracker.ietf.org/doc/html/rfc959#section-5) makes FTP commands case-insensitive, so `retr file.txt` was a valid request the library could not parse. `get()` and `_missing_` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate `GET` would have lacked the real one's `safe` and `idempotent` attributes ([#582](https://github.com/JarryShaw/PyPCAPKit/issues/582), [#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). `http.method` no longer folds case: [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1) makes the method token case-sensitive, so `Method.get` matches it exactly ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), below). +- the string-keyed `get()` in `pcapkit.const.ftp.command` and `pcapkit.const.http.method` tested membership with the raw key but registered `key.upper()`, so the *first* lowercase or mixed-case token raised `TypeError: 'RETR' already in use`. Reachable from wire data: `pcapkit.protocols.application.ftp` matches with `re.I` and passes the match verbatim, and [RFC 959 Section 5](https://datatracker.ietf.org/doc/html/rfc959#section-5) makes FTP commands case-insensitive, so `retr file.txt` was a valid request the library could not parse. `get()` and `_missing_` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate `GET` would have lacked the real one's `safe` and `idempotent` attributes ([#582](https://github.com/JarryShaw/PyPCAPKit/issues/582), [#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). `http.method` no longer folds case: [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1) makes the method token case-sensitive, so `Method.get` matches it exactly ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), above). - `get()`'s documented `default` was ignored on the integer path across the generated `pcapkit.const` tree, because `_missing_` cannot see the caller's `default` -- `Hardware.get(99999, 0)` raised `ValueError` instead of returning `0`. The integer path now consults `default` before letting the error escape. `-1`, the placeholder the generated signature already carried, still separates "no default" from "a default"; [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859) replaced it with an identity sentinel once the shared base [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below) put `get()` behind one call site. The sweep covers the integer registries generated from the shared template, not only those [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584) named; `pcapng` `OptionType` and `reg` `AppType` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix ([#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)). - `_missing_` in the Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). Each now ends in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and most const modules ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s new ones included) already did, so `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. Minting through `extend_enum` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a member per bit pattern. The range guard is untouched ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). - constant registries rejected an invalid value differently from the built-in `enum`, or did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. These now raise a bare `ValueError`, as `enum.IntEnum` does and the other modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in every generated `get()` body (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register; the [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) and [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) entries below record how each came to return an unregistered member instead. The flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed; `Flags` accepts `0 <= value <= 0xFFFF`. A new `tests/const/test_const_enum_builtin_parity.py` asserts that every registry raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how the `IntFlag` ones escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). -- **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) (below) made it an unregistered member. `Transport._make_port` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk.** Separately, the crawler rendered one `_missing_` branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the branches naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). +- **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) (below) made it an unregistered member. `Transport._make_port` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk**; in 1.4.1, `TCP(srcport=N)` raised `AttributeError` for any bare `int`. Separately, the crawler rendered one `_missing_` branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the branches naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). - **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` was still an `aenum.IntFlag` and a member's `proto` carried the whole set IANA assigned the service, so `tcp | udp` was an ordinary value to pass back in; the `AppType.proto` and `TransportProtocol` entries above record how [#806](https://github.com/JarryShaw/PyPCAPKit/issues/806) and [#808](https://github.com/JarryShaw/PyPCAPKit/issues/808) later retired both. Over every multi-transport member this misrouted port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). - `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. Every generated registry is updated; the non-minting `get()` miss applies to all of them, but the `_missing_` fix changes behaviour only for those with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; the reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a `FILTER_NAME_TO_LINKTYPE` map (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. The other minting call sites in `_missing_`, `get` and the hand-written `mh.py` and `ngap.py` were deferred; the `register`/`register_alias` sites are not deferred but the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). - **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: archived range branches that were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 values that** `_missing_` **resolves (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). @@ -42,7 +43,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - the `AppType` registry's dict union is defined only on interpreters whose `dict` has one ([#1694](https://github.com/JarryShaw/PyPCAPKit/issues/1694), [#1703](https://github.com/JarryShaw/PyPCAPKit/pull/1703)). - NGAP's `ProcedureCode` and `ProtocolIE` are generated into `pcapkit.const.ngap` from `pycrate`'s compiled NGAP specification, with no network fetch. An unseen value gets a non-registering member, where every unknown key used to mint a member at the shared value `-1` and so aliased every other ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#899](https://github.com/JarryShaw/PyPCAPKit/pull/899)). - `FEATCode.get` matches case-insensitively, as [RFC 5797 Section 2](https://datatracker.ietf.org/doc/html/rfc5797#section-2) defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold ([#903](https://github.com/JarryShaw/PyPCAPKit/issues/903), [#913](https://github.com/JarryShaw/PyPCAPKit/pull/913)). -- `register` refuses what `get` refuses. `TCP.register(70000, 'x')` and `Hardware.register(70000, 'x')` minted a member that `get` then returned, because no registry overrode the `_validate_value` hook `register` calls. Each generated registry, `AppType` included, now overrides it with the check its `_missing_` makes, raising `EnumValueError`. `get(value, default)` on a value the hook refuses still returns `default`, but now also logs the `EnumValueError` at `CRITICAL`, as any loud pcapkit error is ([#1792](https://github.com/JarryShaw/PyPCAPKit/issues/1792), [#1818](https://github.com/JarryShaw/PyPCAPKit/pull/1818), [#1816](https://github.com/JarryShaw/PyPCAPKit/issues/1816), [#1823](https://github.com/JarryShaw/PyPCAPKit/pull/1823)). +- `register` refuses what `get` refuses. `TCP.register(70000, 'x')` and `Hardware.register(70000, 'x')` minted a member that `get` then returned, because no registry overrode the `_validate_value` hook `register` calls. Each generated registry, `AppType` included, now overrides it with the check its `_missing_` makes, raising `EnumValueError`. `get(value, default)` on a value the hook refuses still returns `default` and logs nothing ([#1792](https://github.com/JarryShaw/PyPCAPKit/issues/1792), [#1818](https://github.com/JarryShaw/PyPCAPKit/pull/1818), [#1816](https://github.com/JarryShaw/PyPCAPKit/issues/1816), [#1823](https://github.com/JarryShaw/PyPCAPKit/pull/1823); the quiet miss is [#1845](https://github.com/JarryShaw/PyPCAPKit/issues/1845), [#1873](https://github.com/JarryShaw/PyPCAPKit/pull/1873), under `pcapkit.corekit`). - `Command.get` refuses a key that is not a `str` with `EnumValueError`, as `Method.get` does. `get(b'\x01')` minted a member keyed by `bytes`, and `get(-1)` raised `AttributeError` whatever its `default` ([#1824](https://github.com/JarryShaw/PyPCAPKit/issues/1824), [#1826](https://github.com/JarryShaw/PyPCAPKit/pull/1826)). ### pcapkit.corekit @@ -63,7 +64,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - **a breaking change to** `@final`, now enforced at runtime on `Info` and `Schema`. `info_final` and `schema_final` both end `return final(cls)`, so every finalised class carried `typing.final`'s `__final__` marker and nothing read it. `@final` **alone on an** `Info` **or** `Schema` **subclass -- without** `@info_final` **or** `@schema_final` **-- now raises** `InfoError` **or** `SchemaError` **at first construction**, the class being marked final but never finalised and so carrying no generated `__init__`. `__init_subclass__` cannot catch that, since `final` applies after class creation, so the check sits in the existing one-shot `FinalisedState.NONE` branch and a finalised class pays nothing. **Deriving from a finalised class raises too**, from new `Info.__init_subclass__` and `Schema.__init_subclass__` hooks (before, only `EnumSchema` had one). `SchemaError` **is a** `ValueError`, so code catching `TypeError` around a subclass declaration will not see it. `@info_final @final` and `@final @info_final` both finalise silently; the re-entry check keys on `__finalised__` rather than `__final__`, because only the former records *the decorator* having run. `@info_final` twice warns and returns the class unchanged. `pcapkit.utilities.compat` takes `final` from `typing_extensions` below 3.11 rather than 3.8, because `typing.final` only records `__final__` from 3.11. The guard cannot fire on the library's own classes ([#778](https://github.com/JarryShaw/PyPCAPKit/issues/778)). - **a breaking change to** `Info.to_dict()`: it returns an `OrderedMultiDict` at every level rather than a `dict`, so a key that repeats, such as a second IPv6 extension header of one type, survives a `from_dict(to_dict())` rebuild. `type(d) is dict` no longer holds ([#1453](https://github.com/JarryShaw/PyPCAPKit/issues/1453), [#1466](https://github.com/JarryShaw/PyPCAPKit/pull/1466), [#1484](https://github.com/JarryShaw/PyPCAPKit/issues/1484), [#1523](https://github.com/JarryShaw/PyPCAPKit/pull/1523)). - **a breaking change to** `pcapkit.utilities.decorators`: it moves to `pcapkit.corekit.decorators`, with no re-export. `seekset`, `beholder` and `prepare` are no longer importable from `pcapkit.utilities`, and the module logs as `pcapkit.corekit.decorators` ([#1735](https://github.com/JarryShaw/PyPCAPKit/issues/1735), [#1737](https://github.com/JarryShaw/PyPCAPKit/pull/1737)). -- **a breaking change to** the sentinels: all live in `pcapkit.corekit.sentinels`, which exports the objects `NULL`, `NO_VALUE` and `NO_DEFAULT` but not their types, and `NoValue` is spelled `NO_VALUE` ([#916](https://github.com/JarryShaw/PyPCAPKit/pull/916), [#911](https://github.com/JarryShaw/PyPCAPKit/issues/911), [#922](https://github.com/JarryShaw/PyPCAPKit/pull/922), [#937](https://github.com/JarryShaw/PyPCAPKit/issues/937), [#939](https://github.com/JarryShaw/PyPCAPKit/pull/939)). No moved name keeps a re-export, so `NoValueType` has only that home, not its 1.4.x one in `pcapkit.corekit.fields.field` ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). +- **a breaking change to** the sentinels: all live in `pcapkit.corekit.sentinels`, which exports the objects `NULL`, `NO_VALUE` and `NO_DEFAULT` and their common base `Sentinel`, but not their types, and `NoValue` is spelled `NO_VALUE` ([#916](https://github.com/JarryShaw/PyPCAPKit/pull/916), [#911](https://github.com/JarryShaw/PyPCAPKit/issues/911), [#922](https://github.com/JarryShaw/PyPCAPKit/pull/922), [#937](https://github.com/JarryShaw/PyPCAPKit/issues/937), [#939](https://github.com/JarryShaw/PyPCAPKit/pull/939)). No moved name keeps a re-export, so `NoValueType` has only that home, not its 1.4.x one in `pcapkit.corekit.fields.field` ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). #### Fixed @@ -76,9 +77,9 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `FieldBase.unpack` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) -- the shortfall was zero-filled with `rjust()`, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now `ljust()`, correct for **both** byte orders: one octet of a four-octet little-endian `120` read as `2013265920`, and three octets of a four-octet big-endian `0x01020304` as `0x10203`. A full read is untouched ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but separate defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Further contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). - the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A companion `bounded_area` clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). -- **a breaking change to** `FieldBase.length`: it called `struct.calcsize` on a template built from a negative resolved length (e.g. `'-5s'`, from a `length=` callback whose running counter had been overdrawn), raising a bare `struct.error` that ordinary caller code does not catch. The chokepoint every affected schema shares now raises `ProtocolError`, covering `application/{ftp,httpv1,httpv2,ngap}.py`, `internet/{hip,ipv6_route,mh}.py`, `link/ethernet.py`, `misc/pcapng.py` and `transport/sctp.py` with no other file changed; a non-negative length is unaffected. This closes the follow-on the `httpv2` length-guard entry above filed as out of scope ([#805](https://github.com/JarryShaw/PyPCAPKit/issues/805)). -- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it first converts registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and the `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those: the other const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). These also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other registries ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). -- **a breaking change to** `EnumRegistry.register()`, extended: the const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype` among them) are left alone, so **every other class** inherits `EnumRegistry` ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the maintainer's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). +- **a breaking change to** `FieldBase.length`: it called `struct.calcsize` on a template built from a negative resolved length (e.g. `'-5s'`, from a `length=` callback whose running counter had been overdrawn), raising a bare `struct.error` that ordinary caller code does not catch. The chokepoint every affected schema shares now raises `ProtocolError`, covering `application/{ftp,httpv1,httpv2,ngap}.py`, `internet/{hip,ipv6_route,mh}.py`, `link/ethernet.py`, `misc/pcapng.py` and `transport/sctp.py` with no other file changed; a non-negative length is unaffected. This closes the follow-on the `httpv2` length-guard entry under `pcapkit.protocols` filed as out of scope ([#805](https://github.com/JarryShaw/PyPCAPKit/issues/805)). +- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the owner's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it first converts registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and the `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those: the other const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). These also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other registries ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). +- **a breaking change to** `EnumRegistry.register()`, extended: the const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype` among them) are left alone, so **every other class** inherits `EnumRegistry` ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the owner's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). - `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). - **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but each `cls(default)` call site reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, each site now resolves `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). - `ProtoChain.__add__` returned the left operand's cached `protocols` and `aliases`: once `a.protocols` had been read, `a + ProtoChain(IPv4)` reported `chain == 'Ethernet:IPv4'` but `protocols == (Ethernet,)`. `copy.copy(self)` carried the cache across; the result is now built fresh, as `from_list` builds one, and `+=` follows, since it falls back to `__add__` ([#1099](https://github.com/JarryShaw/PyPCAPKit/issues/1099)). @@ -97,6 +98,8 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - negative lengths raise `ProtocolError` rather than a bare stdlib error: `FieldBase.length` tells a malformed template from a negative resolved length ([#825](https://github.com/JarryShaw/PyPCAPKit/issues/825), [#827](https://github.com/JarryShaw/PyPCAPKit/pull/827)), and `NumberField` checks a negative resolved `length` and a negative `bit_length` ([#828](https://github.com/JarryShaw/PyPCAPKit/issues/828), [#829](https://github.com/JarryShaw/PyPCAPKit/pull/829), [#831](https://github.com/JarryShaw/PyPCAPKit/issues/831), [#834](https://github.com/JarryShaw/PyPCAPKit/pull/834)). - `ModuleDescriptor.klass` raises `ProtocolError` for a class name that resolves to nothing, or one never given, rather than a bare `AttributeError`; the registry modules share one `NULL` sentinel compared by identity, where each had its own string ([#832](https://github.com/JarryShaw/PyPCAPKit/issues/832), [#833](https://github.com/JarryShaw/PyPCAPKit/issues/833), [#835](https://github.com/JarryShaw/PyPCAPKit/pull/835)). - `PortEnumField` given a `proto` with no `.name` -- a string such as `'tcp'`, or a composite such as `TransportProtocol.tcp | TransportProtocol.udp`, which is a bare `int` -- raised `AttributeError` on a port the registry does not hold. The miss takes its transport from the registry's `__transport__`, as `AppType.get` does ([#1774](https://github.com/JarryShaw/PyPCAPKit/issues/1774), [#1781](https://github.com/JarryShaw/PyPCAPKit/pull/1781)). +- `EnumLookup.get` answers a value miss quietly, whether `default` answers it or the miss is raised: a registry's `_validate_value` rejection logs nothing and installs no exception hook there, while `EnumRegistry.register` still reports the same rejection loudly ([#1845](https://github.com/JarryShaw/PyPCAPKit/issues/1845), [#1873](https://github.com/JarryShaw/PyPCAPKit/pull/1873)). +- each sentinel is a singleton, as `None` is: calling its type, `copy`, `deepcopy` and `pickle` at every protocol give back the one object, and `NO_VALUE` reprs as its own name rather than as an object address. Their new common base, `Sentinel`, is public; a sentinel's type cannot be subclassed, and neither it nor the sentinel takes an attribute write. A pickle written before this change at protocol 0 or 1 still loads, though not as the singleton ([#1841](https://github.com/JarryShaw/PyPCAPKit/issues/1841), [#1842](https://github.com/JarryShaw/PyPCAPKit/issues/1842), [#1876](https://github.com/JarryShaw/PyPCAPKit/pull/1876)). ### pcapkit.dumpkit @@ -139,7 +142,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `Extractor` had the ownership of its input stream inverted: a handle it opened itself, from `fin` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. `_cleanup` closed under `not self._flag_s` (the flag that gated the `open()`), and the `SeekableReader` wrapping a non-seekable input asked for `stream_closing=not self._flag_s` in the same wrong direction. The leak was one descriptor per extraction, the root cause of the `ResourceWarning` that reddened [#577](https://github.com/JarryShaw/PyPCAPKit/pull/577), [#596](https://github.com/JarryShaw/PyPCAPKit/pull/596) and [#600](https://github.com/JarryShaw/PyPCAPKit/pull/600) ([#606](https://github.com/JarryShaw/PyPCAPKit/issues/606) had only made the test immune). Both conditions now follow ownership via a single `Extractor._owns_input`, and a finaliser releases the handle of an extraction *abandoned* before end of file (`auto=False`, iterated part way and dropped); the files [#610](https://github.com/JarryShaw/PyPCAPKit/issues/610) names emit no such warning. `__exit__` deliberately still closes unconditionally, since a caller who scopes an `Extractor` with `with` asked for that ([#610](https://github.com/JarryShaw/PyPCAPKit/issues/610)). - `extract(..., no_eof=True)` never returned. End of stream was detected and `ExtractionWarning: EOF reached` fired, but every loop that handles it (`record_frames`, `__next__`, `__call__`) read `if self._flag_n: continue`, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- `__main__` sets `no_eof` for `fin='-'`, and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one: `prepare` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (`Extractor._note_eof_progress`). A pipe is unaffected, since a read there *blocks* while the writer is open but idle. This also fixes `pcapkit -`, which hung on **any** finished stdin. **One deliberate narrowing**: a *seekable* input does not block, so a file still being appended to ends at the data present when the extraction reached it; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own. Pre-existing and untouched: an append landing *mid-record* raises `ValueError: read length must be non-negative or -1`. The end-to-end tests are bounded by a **child process** rather than `tests._support.time_limit`, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than a red test. Guesses at the cause are ruled out: the parse path's `except Exception` handlers are never entered during the spin, and the memory is the retry loop's `EOF reached` records, which the runner retains ([#620](https://github.com/JarryShaw/PyPCAPKit/issues/620)). - `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and several dispatchable classes are named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The guard deliberately reads "key present and the incumbent is a different class", not mere presence ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave its sibling registrars the same identity guard): this function is the funnel every wrapper registrar ends in, so registering one class under two codes (`register_tcp` then `register_udp`, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514). The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). -- the `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` that named a bare `Type`, which Sphinx resolves against every class named `Type` in the project rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. These sites now spell it `~typing.Type[Dumper]`, among the sites [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) lists ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). The other bare `Type` sites, in hand-written `:type:` fields under `docs/source/pcapkit/foundation/`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). +- the `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` that named a bare `Type`, which Sphinx resolves against every class named `Type` in the project rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. These sites now spell it `~typing.Type[Dumper]`, among the sites [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) lists. The other bare `Type` sites, in hand-written `:type:` fields under `docs/source/pcapkit/foundation/`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). - `register_protocol`'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (`incumbent is not protocol`, an identity check from [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)), but the message built both operands with a bare `repr()`. A factory defining a same-named closure-local class on every call (as `tests/protocols/test_construction_keyword_check_unit.py`'s `_protocol_class` does) produces two distinct classes sharing one `__module__` and `__qualname__`, so a real overwrite read "overwriting X with X." The message now compares the two reprs and, only when they coincide, appends each object's `id()`; `__module__`/`__qualname__` would not disambiguate, being what the repr already renders ([#710](https://github.com/JarryShaw/PyPCAPKit/issues/710)). - more registrars warned on mere key presence rather than an actual overwrite, outside the wording of [#718](https://github.com/JarryShaw/PyPCAPKit/issues/718)'s identity guard (`incumbent is not None and incumbent is not new`, landed by [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)), whose issue named only code-keyed registrars: `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, and `register_dumper` on both `Extractor` and `TraceFlow`. Each now compares the incumbent by identity before warning; both `register_dumper` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings ([#739](https://github.com/JarryShaw/PyPCAPKit/issues/739)). - **the class registrars listed below** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the `register_dumper` methods, and from inside `abc` for the others. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in those that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocol `register` classmethods is fixed separately by [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). @@ -155,6 +158,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `format='pcap'` is kept when an unavailable engine falls back to the default one ([#1264](https://github.com/JarryShaw/PyPCAPKit/issues/1264), [#1430](https://github.com/JarryShaw/PyPCAPKit/pull/1430)). - `extract()` accepts a file object with no `name` as `fin` ([#1506](https://github.com/JarryShaw/PyPCAPKit/issues/1506), [#1509](https://github.com/JarryShaw/PyPCAPKit/pull/1509)), and reads every frame from a pipe or other non-seekable stream ([#1586](https://github.com/JarryShaw/PyPCAPKit/issues/1586), [#1595](https://github.com/JarryShaw/PyPCAPKit/pull/1595)). - the third-party engines read what the capture holds. The dpkt engine keeps a frame dpkt cannot parse rather than leaking `AttributeError` ([#1351](https://github.com/JarryShaw/PyPCAPKit/issues/1351), [#1377](https://github.com/JarryShaw/PyPCAPKit/pull/1377)), applies each PCAP-NG interface's link type and timestamp resolution rather than the first one's ([#1379](https://github.com/JarryShaw/PyPCAPKit/issues/1379), [#1396](https://github.com/JarryShaw/PyPCAPKit/pull/1396)), and reads float timestamps, Simple Packet Blocks and raw-IP frames ([#1550](https://github.com/JarryShaw/PyPCAPKit/pull/1550)). The scapy engine resets its interfaces at each PCAP-NG section ([#1522](https://github.com/JarryShaw/PyPCAPKit/issues/1522), [#1546](https://github.com/JarryShaw/PyPCAPKit/pull/1546)), takes a packet's link type and clock from its interface ([#1563](https://github.com/JarryShaw/PyPCAPKit/pull/1563)), and reads each record whole ([#1717](https://github.com/JarryShaw/PyPCAPKit/issues/1717), [#1733](https://github.com/JarryShaw/PyPCAPKit/pull/1733)). The pyshark engine skips non-packet PCAP-NG blocks ([#1515](https://github.com/JarryShaw/PyPCAPKit/issues/1515), [#1531](https://github.com/JarryShaw/PyPCAPKit/pull/1531)), and the pypcapfile engine reads the PCAP records pypcapfile drops ([#1512](https://github.com/JarryShaw/PyPCAPKit/issues/1512), [#1569](https://github.com/JarryShaw/PyPCAPKit/pull/1569)). dpkt and scapy write real PCAP traces ([#1507](https://github.com/JarryShaw/PyPCAPKit/issues/1507), [#1511](https://github.com/JarryShaw/PyPCAPKit/pull/1511)). +- a callable passed as `verbose=` is called on every engine. The scapy, dpkt, pyshark, pypcap, pcap-ct and pypcapfile engines replaced it with their own printer, which now runs only for `verbose=True`, its output unchanged ([#1874](https://github.com/JarryShaw/PyPCAPKit/issues/1874), [#1880](https://github.com/JarryShaw/PyPCAPKit/pull/1880)). ### pcapkit.interface @@ -198,7 +202,7 @@ Import paths, before to after: Layer-limited extraction: `layer='link'` no longer sets the termination flag on OSPF and `layer='application'` now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every `layer=` value, including `'internet'` (`IPv4:OSPFIGP` before and after, as IPv4 ends the chain itself). - renames with no compatibility alias left behind: PCAP-NG `Option` subclasses spell the namespace class keyword `ns=` instead of `namespace=` ([#439](https://github.com/JarryShaw/PyPCAPKit/issues/439)). - `tests/protocols/transport/test_tcp_udp_unit.py` now reaches the MP_JOIN dispatchers through `TCP()` itself, instead of assigning a Python `set` to `_flags` on a bare `TCP.__new__(TCP)`. A `set` answers the same membership tests, so every branch ran while the attribute had neither the type production assigns nor the ordering that governs when it exists -- how [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587) stayed invisible. Reverting [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)'s hoist, or restoring the `cast('Enum_Flags', 0)` seed, now fails the file. Test-only ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- building a protocol through its constructor with a keyword that names nothing now raises `UnsupportedCall` instead of discarding it. **This is a behaviour change to a public API**: every `make` in the tree ends its signature with `**kwargs` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602), [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) and [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556). The check sits in `ProtocolBase.__init__` rather than `make`, because `__post_init__` passes one `**kwargs` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by `read` legitimately travels through `make` (`HIP.read` declares `extension`). The accepted set is the union of every keyword-taking parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__` across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new `__keywords__` opts a class out: a set, for a keyword read out of `**kwargs` by name (`ESP.read` with `packet`); and `None`, for a dispatcher whose real signature belongs to a class chosen at call time (only `HTTP.make`). The message names the near neighbour (`seq` reports *did you mean 'seq_no'?*). `from_data` warns `UnknownFieldWarning` rather than raising for a key its own `_make_data` returned, which the caller cannot fix; a keyword passed to `from_data` itself raises, as for a direct construction ([#1147](https://github.com/JarryShaw/PyPCAPKit/pull/1147)). The leftover copies of [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)'s misspelt TCP base, in `examples/generators/dispatch.py` and `tests/protocols/transport/`, are fixed. Some `_make_data` keys that `make` does not declare are reported, not fixed: `Frame._make_data` returns `ts_src` where `make` declares `ts_sec`, `L2TPv2._make_data` returns `prio` where it declares `priority`, and `Header._make_data` returns a `magic_number` that `Header.make` does not take. A *direct* `SomeProtocol.make(...)` call is not checked and still discards silently ([#617](https://github.com/JarryShaw/PyPCAPKit/issues/617)). +- building a protocol through its constructor with a keyword that names nothing now raises `UnsupportedCall` instead of discarding it. **This is a behaviour change to a public API**: every `make` in the tree ends its signature with `**kwargs` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602), [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) and [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556). The check sits in `ProtocolBase.__init__` rather than `make`, because `__post_init__` passes one `**kwargs` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by `read` legitimately travels through `make` (`HIP.read` declares `extension`). The accepted set is the union of every keyword-taking parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__` across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new `__keywords__` opts a class out: a set, for a keyword read out of `**kwargs` by name (`ESP.read` with `packet`); and `None`, for a dispatcher whose real signature belongs to a class chosen at call time (only `HTTP.make`). The message names the near neighbour (`seq` reports *did you mean 'seq_no'?*). `from_data` warns `UnknownFieldWarning` rather than raising for a key its own `_make_data` returned, which the caller cannot fix; a keyword passed to `from_data` itself raises, as for a direct construction ([#1147](https://github.com/JarryShaw/PyPCAPKit/pull/1147)). The leftover copies of [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)'s misspelled TCP base, in `examples/generators/dispatch.py` and `tests/protocols/transport/`, are fixed. Some `_make_data` keys that `make` does not declare are reported, not fixed: `Frame._make_data` returns `ts_src` where `make` declares `ts_sec`, `L2TPv2._make_data` returns `prio` where it declares `priority`, and `Header._make_data` returns a `magic_number` that `Header.make` does not take. A *direct* `SomeProtocol.make(...)` call is not checked and still discards silently ([#617](https://github.com/JarryShaw/PyPCAPKit/issues/617)). - **a breaking change to** `Application`: it no longer forbids payload dispatch outright. It encodes "no further protocol layer above this one", which undissected trailer bytes are not, so `_decode_next_layer` and `_import_next_layer` accept the `-1` sentinel -- the rest of the packet, resolving to `Raw`, or to `NoPayload` when nothing remains -- and `__post_init__` keeps a payload that `read` set instead of overwriting it with `NoPayload`. A real protocol number (any `proto` other than `-1`) is still refused with `UnsupportedCall`. A subclass that calls either method with `-1` changes from raising to succeeding; no subclass in the package does ([#719](https://github.com/JarryShaw/PyPCAPKit/issues/719)). - the `Protocol` and `ProtocolMeta` docstrings. `Protocol` claimed to be the base of every protocol family, though nothing in the package subclasses it, and `ProtocolMeta` claimed to generate attributes its empty body does not. `Protocol` is now documented as the base for third-party protocols, and `ProtocolBase` as the one the families derive from ([#1101](https://github.com/JarryShaw/PyPCAPKit/issues/1101)). - **a breaking change to** `OSPF_CrytographicAuthentication`: the misspelling is corrected to `OSPF_CryptographicAuthentication`, and `CrytographicAuthentication` in `pcapkit.protocols.schema.application.ospf` and `pcapkit.protocols.data.application.ospf` to `CryptographicAuthentication`, with no alias left for the old spelling. Both OSPF names also move to the application group of `schema.__all__` and `data.__all__`, after [#719](https://github.com/JarryShaw/PyPCAPKit/issues/719)'s move ([#1090](https://github.com/JarryShaw/PyPCAPKit/issues/1090)). @@ -207,6 +211,7 @@ Layer-limited extraction: `layer='link'` no longer sets the termination flag on - **a breaking change to** IPv6 extension headers: an unrecognised or unparsable one is parsed generically, per [RFC 6564](https://datatracker.ietf.org/doc/html/rfc6564), rather than collapsing the packet to `Raw` ([#891](https://github.com/JarryShaw/PyPCAPKit/issues/891), [#904](https://github.com/JarryShaw/PyPCAPKit/pull/904)). `IPv6_Ext` does it, and is also the shared base of every extension header ([#917](https://github.com/JarryShaw/PyPCAPKit/issues/917), [#924](https://github.com/JarryShaw/PyPCAPKit/pull/924)). - **a breaking change to** `pack()`: it no longer keeps the header it builds on the instance it is called on, which in 1.4.1 too left `schema` disagreeing with `info`: `UDP(srcport=1234).pack()` reset `schema.srcport` to 0 while `info.srcport` kept 1234. Once an instance has been parsed or constructed, `pack()` builds on a shallow copy, so `schema`, `info` and what `make()` sets, such as TCP's `connection`, stay as they were. The `Frame`, `PCAPNG` and `HTTPv2` overrides do the same, and re-packing a PCAP-NG block that carries an option allowed only once no longer raises. Code that relied on `pack()` to re-sync `schema` calls `make()` or constructs a new instance ([#1769](https://github.com/JarryShaw/PyPCAPKit/issues/1769), [#1791](https://github.com/JarryShaw/PyPCAPKit/pull/1791)). - **a breaking change to** the HOPOPT, IPv6-Opts and Mobility Header padding: two octets are one `PadN` with no data (`01 00`) rather than two `Pad1` options, as [RFC 8200 Section 4.2](https://datatracker.ietf.org/doc/html/rfc8200#section-4.2) and [RFC 6275 Section 6.2.2](https://datatracker.ietf.org/doc/html/rfc6275#section-6.2.2) ask. Only a freshly built header with exactly two octets of padding changes; a parsed one that carried two `Pad1` options still rebuilds byte for byte ([#1793](https://github.com/JarryShaw/PyPCAPKit/issues/1793), [#1812](https://github.com/JarryShaw/PyPCAPKit/pull/1812)). +- **a breaking change to** `NTPTimestamp`: it and `PMIPv6Timestamp` are `Info` classes in `pcapkit.protocols.data.internet.mh` rather than tuples, and `NTPTimestamp` keeps no alias at its 1.4.x home in `pcapkit.protocols.internet.mh`. As Mappings they no longer index, unpack, compare or hash as a tuple does; `.seconds` and `.fraction` still work, and the dumpers write both as keys ([#1886](https://github.com/JarryShaw/PyPCAPKit/issues/1886), [#1887](https://github.com/JarryShaw/PyPCAPKit/pull/1887)). #### Fixed @@ -233,13 +238,13 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - as a consequence of the above, the unhandled `MemoryError` in `pcapkit/protocols/protocol.py`'s `self._file.read()` call on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). - documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, whereas a `SwitchField` always resolves to a concrete field ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- the assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, red on `main` since [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) merged. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- the assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, which [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) turned red on `main`. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - the HIP `SOLUTION` builder sized the parameter with `4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)`, an invalid shorthand for two fields. [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5) gives `Length` as `4 + RHASH_len / 4` over a `Random #I` and a `Puzzle solution #J` of `RHASH_len / 8` octets **each**, and `/ 4` equals twice `/ 8` only because `RHASH_len` is a whole number of octets. For an arbitrary `bit_length()` the builder emitted an *odd* contents width, which its own reader refused: `_read_param_solution`'s `(schema.len - 4) % 2` guard exists because `SolutionParameter` splits that width into two equal halves. `random=0x1` with `solution=0xfff` declared `len=7` and raised `ProtocolError: HIPv2: [ParamNo 321] invalid format` on input the library had produced. The undersized length was worse: both fields take their width from that `len`, so `solution=0xfff` was packed into one octet and came back as `0xff`, silently. Fixed to `4 + 2 * math.ceil(max(...) / 8)`, the form the sibling `PUZZLE` builder uses, which is even by construction. Loosening the reader was rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). - `TCP.read` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. A dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). - **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading** `frame.len` **or** `frame.cap_len` **from a** `.pcap` **gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). - the HIP `PUZZLE` and `SOLUTION` builders derived wire-format values from the payload value instead of the data model, and each was wrong. **This changes the** `PuzzleParameter` **and** `SolutionParameter` **data models and the octets both parameters emit.** The width of `Random #I` and `Puzzle solution #J` came from `int.bit_length()` alone, so every leading zero octet was dropped on re-serialisation: a `SOLUTION` read with `Length = 20` rebuilt as `Length = 6`, a `PUZZLE` with `Length = 12` as `Length = 5`, silently. That width is `RHASH_len / 8` octets [[RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4), [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5)], a property of the Responder's HIT Suite, so both data models now carry it as `rhash_len` and both builders prefer it; a new `rhash_len=` keyword states it for a build from scratch, which under HIPv2 is the only place it can come from, since `RSA,DSA/SHA-256` is the REQUIRED HIT Suite [[RFC 7401 Section 5.2.10](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.10)] and makes the field 32 octets rather than 8. On the mandatory path about one parameter in 256 has a zero top octet and lost it ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653)). `SOLUTION`'s second contents octet is `Reserved`, "zero when sent, ignored when received" [[RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5), and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) identically], not the `Lifetime` that only [RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4) defines, which pcapkit encoded there as `2^(value - 32)` seconds. It wrote `0x20`, `0x21`, `0x25` or `0x2b` into a field the RFC requires to be zero, and could not write that zero: a conformant `SOLUTION` with `Reserved` `0x00` parsed to `timedelta(0)` and could not be re-serialised, escaping a bare `ValueError` from `math.log2(0.0)` that was not a `BaseError`. The field is renamed `reserved`, defaults to zero, and is carried verbatim across a round trip ([#654](https://github.com/JarryShaw/PyPCAPKit/issues/654)). And neither builder read its own `version` keyword, so `version=1` and `version=2` computed identical lengths, where [RFC 5201 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.4) and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) state both fields as literally 8 bytes and `Length` as literally 12 and 20. Under HIPv1 each builder accepted only a `bit_length()` of 57..64 and built, for anything narrower, a parameter this library's reader rejects -- the shape of [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608), in the `PUZZLE` builder that [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629) never opened ([#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). The remaining `math.log2` sites, in `PUZZLE` where a lifetime is real, raise `ProtocolError` (`(BaseError, ValueError)`, so a caller written around the bare exception still catches it). Migration: `SolutionParameter.lifetime` is now `reserved` and an `int` rather than a `timedelta`; `PuzzleParameter` and `SolutionParameter` gained a required `rhash_len`; and `_make_param_solution` no longer takes `lifetime=`. Each was reachable end to end only once [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608) was fixed in [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629), which removed the parity guard that had failed these rebuilds loudly; the round trip then stopped raising and started quietly emitting a different parameter, which is why they were fixed together ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653), [#654](https://github.com/JarryShaw/PyPCAPKit/issues/654), [#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). - every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **every HIP parameter but** `LOCATOR_SET`, excluded below. **This changes the octets those parameters write, and the** `length` **their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; every padding site computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. Every site but `LOCATOR_SET`'s (the `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and the record lengths in `pcapkit/protocols/internet/hip.py`) now goes through one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other parameters move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). -- HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only `_make_http_*` method that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at each of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since each passes `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). +- HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only `_make_http_*` method that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at each of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since each passes `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668), below) ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). - MPTCP error messages carried a doubled separator, rendering as `TCP: : [OptNo 30] 1: invalid flags combination`. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty field reads as a value that failed to interpolate. The sites were `_read_mptcp_join` and `_make_mptcp_join` for the invalid flag combination, and both guards of `_make_mptcp_dss` for missing required fields; every other message in the file was already spelled correctly, and none is doubled now. Pre-existing since 2023-04-10. The new test asserts each message in full, since the existing `assertIn` on the body passes either side of the change ([#649](https://github.com/JarryShaw/PyPCAPKit/issues/649)). - names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. Each is now in its module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as the sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values, and reports exactly those findings on the unfixed tree. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). - every PCAP-NG packet block now carries the captured octets it declares. Every Enhanced Packet Block of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those block types, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack` extracted them from the block schema, and `ProtocolBase.__init__` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the packet types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). @@ -255,7 +260,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - **a breaking change to** `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` each minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). Each now peeks at what `.get()` would consult and call it only once the value resolves; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and most other namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: some sample captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). - `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the frame and the request left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). - `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and the genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: `TCP:80/8080` traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). -- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation` and `OptionType` convert their unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; those with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a later review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (an independent mint site of its own) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). +- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), above), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation` and `OptionType` convert their unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; those with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a later review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (an independent mint site of its own) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). - `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. Every such `Raises:` clause already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). - `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Other guards named the wrong option too: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). - `httpv1.HTTP` refused every HTTP/1 message with no header fields, which [[RFC 9112 Section 2.1](https://datatracker.ietf.org/doc/html/rfc9112#section-2.1)] allows (`*( field-line CRLF )`): `b'GET / HTTP/1.0\r\n\r\n'` raised `ProtocolError: HTTP: invalid format`, because `_read_http_header` treated a header with no CRLF as malformed. So `HTTPv1(method='GET', uri='/', http_version='1.1')` failed to read back the bytes its own `make` built, and `HTTP._guess_version` answered `unknown HTTP version`, since `_test_start_line` mirrored the same check. A CRLF-less header is now a start line with zero fields, in the parser and the predicate alike. That check was also what refused the HTTP/2 connection preface, whose header is exactly `PRI * HTTP/2.0`; it is now refused by name instead, to the same message ([#1041](https://github.com/JarryShaw/PyPCAPKit/issues/1041)). @@ -277,7 +282,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - **a breaking change to** `from_data` on a capture truncated by snaplen: `IPv4`, `IPv6` and `UDP` rebuilt their length field from the payload at hand, writing the recorded length, so `big_endian.pcap` frame 3 came back as `45000052` where `450004a2` was on the wire. `_make_data` now passes the parsed length through to `make()`'s new `total_length` (IPv4, UDP) or `payload_length` (IPv6) keyword, which computes the length as before when omitted, so `IPv4(payload=...)` is unchanged ([#1155](https://github.com/JarryShaw/PyPCAPKit/issues/1155)). - **a breaking change to** the layer chain of a capture cut inside a header: a next layer parsed from too few captured octets was zero-filled, and `from_data` rebuilt the fabricated octets. `IPv4.read` handed UDP the declared payload length when nothing past the IPv4 header was captured, so `test.pcapng` frame 3 rebuilt with 8 invented UDP octets ([#1167](https://github.com/JarryShaw/PyPCAPKit/issues/1167)). Every `_import_next_layer` now goes through one guard: no captured octets gives `NoPayload`, and fewer than the next layer's `__length_hint__` gives `Raw` unless the parsed header fits anyway, as a 12-octet L2TPv2 ZLB does. A header-only IPv6 packet is `IPv6` rather than `IPv6:UDP:Raw`, and frame 3 is `Ethernet:Raw`, rebuilt byte for byte ([#1170](https://github.com/JarryShaw/PyPCAPKit/issues/1170)). The guard needed honest hints, so `IPv6_Route` (4), `MH` (6), `HOPOPT` and `IPv6_Opts` (2) now hint their 8-octet minimum header ([#1173](https://github.com/JarryShaw/PyPCAPKit/issues/1173)), and so does the `IPv6_Ext` fallback, which hinted 2 ([#1178](https://github.com/JarryShaw/PyPCAPKit/issues/1178)). - **a breaking change to** `L2TPv2`, against [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) section 3.1. `Type` was inverted; it is now `Data = 0` and `Control = 1`, regenerated in `pcapkit.const` from the fixed `pcapkit.vendor` table, so the same bytes parse to the other `flags.type`, and a default `make()` writes T=0 where it wrote T=1. A control message now gets `L` and a Length field automatically, and the offset pad, which `from_data` zero-filled, is kept as `info.padding` -- a new key in `info` and in dumped output ([#1175](https://github.com/JarryShaw/PyPCAPKit/issues/1175)). `from_data` also cleared `L` and dropped the Length field: `_make_data` returned it as `length`, which `ProtocolBase` takes as the parse length. `make()`'s `length` keyword is now `total_length`, written as given; a call still passing `length` loses `L` silently ([#1172](https://github.com/JarryShaw/PyPCAPKit/issues/1172)). `make()` also gains `length_flag` and `padding`, and raises `ProtocolError` for a `total_length` with `length_flag=False`. -- **a breaking change to** `IPv6.from_data`: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden `__exthdr__` key, absent from `to_dict` and dumped output, and re-emitted ahead of the upper layer. Sample frames that rebuilt wrong now rebuild exactly, and those that rebuilt with wrong bytes raised `AttributeError` instead ([#1177](https://github.com/JarryShaw/PyPCAPKit/issues/1177)). Those were older defects, now fixed: `HIP.alias` read `_info` before `read` had set it, which also broke a standalone HIP parse and `HIP.from_data` ([#1184](https://github.com/JarryShaw/PyPCAPKit/issues/1184)), and the MPL option's maker called `bit_length()` on the `IPv6Address` seed of an S=0 option. MPL in `HOPOPT` and `IPv6_Opts` now takes `S` from the parsed option or a new `seed_type` keyword, and a missing or oversized seed raises `ProtocolError` ([#1185](https://github.com/JarryShaw/PyPCAPKit/issues/1185)). With both fixed, every IPv6 frame of the `options-*.pcap` captures rebuilds byte for byte. +- **a breaking change to** `IPv6.from_data`: it dropped the extension-header chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden `__exthdr__` key, absent from `to_dict` and dumped output, and re-emitted ahead of the upper layer. Sample frames that rebuilt wrong now rebuild exactly, though some that had rebuilt with wrong bytes raised `AttributeError` instead ([#1177](https://github.com/JarryShaw/PyPCAPKit/issues/1177)), from older defects fixed with it: `HIP.alias` read `_info` before `read` had set it, which also broke a standalone HIP parse and `HIP.from_data` ([#1184](https://github.com/JarryShaw/PyPCAPKit/issues/1184)), and the MPL option's maker called `bit_length()` on the `IPv6Address` seed of an S=0 option. MPL in `HOPOPT` and `IPv6_Opts` now takes `S` from the parsed option or a new `seed_type` keyword, and a missing or oversized seed raises `ProtocolError` ([#1185](https://github.com/JarryShaw/PyPCAPKit/issues/1185)). With both fixed, every IPv6 frame of the `options-*.pcap` captures rebuilds byte for byte. - **a breaking change to** the SMF_DPD option's maker in `HOPOPT` and `IPv6_Opts`: it overwrote the parsed DPD mode, so H-DPD rebuilt as I-DPD, and inferred the TaggerID type from the Tagger ID's length, so a 4-octet `DEFAULT` Tagger ID rebuilt as IPv4. Both now come from the parsed option, or from `mode` and a new `tid_type` keyword. A Tagger ID that does not fit its type raises `ProtocolError`, so a 17-octet one does where it raised `FieldValueError`, and `tid=b''` builds a NULL Tagger ID where it raised `ValueError` ([#1190](https://github.com/JarryShaw/PyPCAPKit/issues/1190)). An H-DPD option with an empty Hash Assist Value raises `ProtocolError` where it raised a bare `IndexError` ([#1193](https://github.com/JarryShaw/PyPCAPKit/issues/1193)). - a `ForwardMatchField` that read short at the end of the stream rewound by its declared width, not by the octets it read, so the fields after it re-read the preceding octet. An SMF_DPD option in the last two octets of a `HOPOPT` or `IPv6_Opts` header was re-read from the padding and raised a bare `TypeError`; it now raises `FieldValueError`, as CALIPSO and MPL do. A full read rewinds as before ([#1195](https://github.com/JarryShaw/PyPCAPKit/issues/1195)). - `HIP()` with no parameters wrote Header Length 0, below [RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401) section 5.1's minimum of 4 for the two HITs, so it failed to read back with `ProtocolError`; it now writes 4. `read` names a Header Length below 4 in a new `invalid header length` message. Lengths 0 to 3 already raised `ProtocolError`, so only the message changes ([#1189](https://github.com/JarryShaw/PyPCAPKit/issues/1189)). @@ -318,6 +323,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - Mobility Header enumerations defined inline from their RFCs (`FastBindingAcknowledgmentStatus`, `IPv6AddressPrefixCode`, `LocalizedRoutingStatus`, `LMAAddressCode`) raise `EnumValueError` for an unassigned value rather than minting `Unassigned_N` ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#889](https://github.com/JarryShaw/PyPCAPKit/pull/889)), and each inherits `get` and its `default` from `EnumLookup` rather than overriding it ([#935](https://github.com/JarryShaw/PyPCAPKit/issues/935), [#940](https://github.com/JarryShaw/PyPCAPKit/pull/940)). - `pcapkit.protocols.protocol.__all__` lists `Protocol` beside `ProtocolBase`, so `from pcapkit.protocols.protocol import *` binds the module's public class ([#1770](https://github.com/JarryShaw/PyPCAPKit/issues/1770), [#1778](https://github.com/JarryShaw/PyPCAPKit/pull/1778)). - the data and schema packages re-export what their submodules list in `__all__`, under the prefixes their siblings use: `pcapkit.protocols.data.link` exports the L2TP data model as `L2TP` and `L2TP_Flags` ([#1797](https://github.com/JarryShaw/PyPCAPKit/issues/1797), [#1806](https://github.com/JarryShaw/PyPCAPKit/pull/1806)), and the Mobility Header models, the HOPOPT and IPv6-Opts `SMFDPDOption` and `QuickStartOption`, PCAP-NG's `PacketBlock` and `PACK_*` options and HTTP/2's `Settings` reach their packages and the aggregators ([#1807](https://github.com/JarryShaw/PyPCAPKit/issues/1807), [#1815](https://github.com/JarryShaw/PyPCAPKit/pull/1815)). +- the MPL option's `seed` in the HOPOPT and IPv6-Opts schemas is annotated `int | IPv6Address`, since an S=0 option holds the source address ([#1840](https://github.com/JarryShaw/PyPCAPKit/issues/1840), [#1866](https://github.com/JarryShaw/PyPCAPKit/pull/1866)). ### pcapkit.toolkit @@ -376,6 +382,8 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - with no terminal on stdin, a failed fetch re-raises rather than waiting for an operator to save the page, a wait that printed without end under a closed or piped stdin ([#522](https://github.com/JarryShaw/PyPCAPKit/issues/522), [#542](https://github.com/JarryShaw/PyPCAPKit/pull/542)). - each crawler derives its `const/` path from the `pcapkit.vendor` root, so a crawler at any depth, such as `vendor/reg/apptype/tcp.py`, writes to the right file ([#741](https://github.com/JarryShaw/PyPCAPKit/pull/741)). - `pcapkit.vendor.ipx` names its `sys.path` temporary `_path` rather than `path`, which read as a public name of the package, and the public-API test covers `pcapkit.vendor`, which it used to skip ([#1772](https://github.com/JarryShaw/PyPCAPKit/issues/1772), [#1779](https://github.com/JarryShaw/PyPCAPKit/pull/1779)). +- the generated const comments render as written: a URL reference followed by another bracket no longer autolinks into it, a leading enumerator such as `(S)` is escaped, and a hyphenated word is no longer split across lines ([#1846](https://github.com/JarryShaw/PyPCAPKit/issues/1846), [#1847](https://github.com/JarryShaw/PyPCAPKit/issues/1847), [#1871](https://github.com/JarryShaw/PyPCAPKit/pull/1871)). +- without the `vendor` extra, `pcapkit-vendor` and `python -m pcapkit.vendor` exit 1 with one line naming the missing packages and `pypcapkit[vendor]`, where they died with a traceback, and `--help` and `--version` still work. Importing a crawler then raises `ModuleNotFoundError` ([#1850](https://github.com/JarryShaw/PyPCAPKit/issues/1850), [#1875](https://github.com/JarryShaw/PyPCAPKit/pull/1875)). ### Project infrastructure @@ -384,23 +392,24 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - an end-to-end test tier ([#376](https://github.com/JarryShaw/PyPCAPKit/pull/376)), sample-capture generators so a fresh clone can rebuild every fixture ([#340](https://github.com/JarryShaw/PyPCAPKit/pull/340)), a Dockerised engine benchmark covering every supported Python version ([#410](https://github.com/JarryShaw/PyPCAPKit/pull/410)), and registry round-trip coverage that records the entries which cannot close the cycle rather than skipping them ([#440](https://github.com/JarryShaw/PyPCAPKit/pull/440), [#504](https://github.com/JarryShaw/PyPCAPKit/pull/504)). - coverage for the untested half of the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation: a TCP or IPv4 option whose declared length asks for more data than the capture holds, pinning that it still parses with the short read zero-padded rather than rejected. The test [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) left covers only an option area with no data behind it; a candidate fix for [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554) turned the untested half into an unwrapped `FieldValueError` while the rest of the suite stayed green ([#571](https://github.com/JarryShaw/PyPCAPKit/pull/571), [#572](https://github.com/JarryShaw/PyPCAPKit/issues/572)). - `CITATION.cff`, citation metadata in Citation File Format 1.2.0, which GitHub renders as the "Cite this repository" button and which citation managers and dependency inventories read directly. It is the machine-readable half of the attribution BSD-3-Clause already asks for. Validated with `cffconvert --validate` and against the published 1.2.0 schema; `doi` and `orcid` are omitted rather than invented, since neither exists for this project and both are checked formats, so a wrong value would still validate. The licence is unchanged (BSD-3-Clause, no `NOTICE`). Alongside it the copyright line moves from `2018-2023` to `2018-2026` (`LICENSE` was its only occurrence; `docs/source/conf.py` derives its own from the current year), and a stray `s` after the closing `DAMAGE.` of the licence text, present since the Mozilla-to-BSD relicence, is removed so the wording matches canonical BSD-3-Clause ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)). -- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository, which is why [#605](https://github.com/JarryShaw/PyPCAPKit/issues/605) survived its own code review: the fix to a misspelt key left the corrected path as untested as the broken one. The generator writes `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`), carrying the *same records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet `snaplen`), so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives each file through `extract()` and derives expectations by walking its record chain with `struct`; a unit-tier case in `test_header_frame_unit.py` builds a big-endian capture in memory, so the fixture-free selection CI runs on every push catches it too. Each fails on the unfixed tree (the fixture-backed ones by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000`) while the little-endian twin passes on both, showing the records themselves are not the variable ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). +- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository, which is why [#605](https://github.com/JarryShaw/PyPCAPKit/issues/605) survived its own code review: the fix to a misspelled key left the corrected path as untested as the broken one. The generator writes `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`), carrying the *same records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet `snaplen`), so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives each file through `extract()` and derives expectations by walking its record chain with `struct`; a unit-tier case in `test_header_frame_unit.py` builds a big-endian capture in memory, so the fixture-free selection CI runs on every push catches it too. Each fails on the unfixed tree (the fixture-backed ones by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000`) while the little-endian twin passes on both, showing the records themselves are not the variable ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). +- local type stubs under `typings/` for dpkt, pyshark, `pcap` and pypcapfile, declaring what pcapkit uses of each, so mypy checks the engines against them. `mypy.ini` reads them through `mypy_path`, and neither the sdist nor the wheel ships them ([#1878](https://github.com/JarryShaw/PyPCAPKit/issues/1878), [#1882](https://github.com/JarryShaw/PyPCAPKit/pull/1882)). #### Changed - renames with no compatibility alias left behind: `examples/sample` and `examples/samples` -- one letter apart, holding different things -- are now `examples/captures` and `examples/generators`. - the README is a landing page now, and Markdown rather than reStructuredText. `README.rst` became a much shorter `README.md`, keeping what a reader arriving from PyPI or a search result needs (what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, where the documentation lives). **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in `docs/source/index.rst`, so they are linked rather than restated; the blocks that existed nowhere else moved: **Testing** is now `docs/source/contributing/testing.rst`, and the `pipenv` and `make setup` block joined the Installation section of `docs/source/index.rst`. Requested by the project owner, and a deliberate exception to the reStructuredText convention, for the README only, since its renderers are GitHub and PyPI rather than Sphinx. `setup.py` reads `README.md` and declares `text/markdown`, and `MANIFEST.in` gains `include README.md`, which is belt-and-braces rather than load-bearing: setuptools' `sdist` ships a `README.md` regardless. Of the `include` lines only `CHANGELOG.md` is load-bearing, measured in the [#631](https://github.com/JarryShaw/PyPCAPKit/pull/631) entry below. `examples/benchmark/Dockerfile` copies `README.md`, while the `README.md` that `examples/benchmark/benchmark.py` names is the benchmark suite's own ([#619](https://github.com/JarryShaw/PyPCAPKit/pull/619)). -- `CODE_OF_CONDUCT.md` moves from Contributor Covenant 1.4 to 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than transcribed. Some things needed deciding. 3.0's `[NOTE` placeholders are filled or removed: the reporting channel names `jarryshaw@icloud.com` (the contact 1.4 carried and `SECURITY.md` uses as email fallback) plus GitHub's report-abuse form for a report about the maintainer, which a single-maintainer project cannot otherwise cover; the enforcement placeholder is an instruction to the adopter and is removed. 3.0's plural "Community Moderators" become the singular maintainer, this repository having no moderators. The enforcement ladder (Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so the attribution names version 3.0, links the permanent `version/3/0/` URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the share-alike term covers this document only (the code remains BSD-3-Clause, `LICENSE` untouched). The ladder renders as nested list items, checked against GitHub's Markdown API; [#613](https://github.com/JarryShaw/PyPCAPKit/pull/613) had to repair the 1.4 file when a stray list marker collapsed it into one nested item ([#624](https://github.com/JarryShaw/PyPCAPKit/pull/624)). +- `CODE_OF_CONDUCT.md` moves from Contributor Covenant 1.4 to 3.0, at the owner's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than transcribed. Some things needed deciding. 3.0's `[NOTE` placeholders are filled or removed: the reporting channel names `jarryshaw@icloud.com` (the contact 1.4 carried and `SECURITY.md` uses as email fallback) plus GitHub's report-abuse form for a report about the maintainer, which a single-maintainer project cannot otherwise cover; the enforcement placeholder is an instruction to the adopter and is removed. 3.0's plural "Community Moderators" become the singular maintainer, this repository having no moderators. The enforcement ladder (Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so the attribution names version 3.0, links the permanent `version/3/0/` URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the share-alike term covers this document only (the code remains BSD-3-Clause, `LICENSE` untouched). The ladder renders as nested list items, checked against GitHub's Markdown API; [#613](https://github.com/JarryShaw/PyPCAPKit/pull/613) had to repair the 1.4 file when a stray list marker collapsed it into one nested item ([#624](https://github.com/JarryShaw/PyPCAPKit/pull/624)). - reconciled which private (`_xxx`) attributes and methods the Sphinx build documents, replacing an ad hoc mix with one rule: document the contract, hide the recipe. A member every subclass must implement, or whose shape a caller depends on, stays documented despite its underscore; a private helper that only keeps one method short does not. Module-level directives for pure implementation detail (`esp._resolve`, `esp._CRYPTO`, `ngap._convert`, `ngap._revert`, `ngap._PYCRATE`, `ngap._PDU_LOCK`, `pypcapfile._NamedStream`) were dropped, while `autoattribute` directives were added for class-private state that *is* contract, among them `Extractor._flag_f`, the `PyPCAP` and `PCAP_CT` engines' `_backend` (the only third-party engines with one), `TraceFlow`'s internal fields, `FieldBase`/`Field` internals, the built-in `PCAP` engine's `_gbhdr`, `_vinfo`, `_nnsec` and `_dlink` (shared with `PyPCAP`, `PCAP_CT` and `PyPCAPFile`) and `PCAPNG`'s `_ctx`/`_ctx_list`. The runtime definitions of `_missing_` (under `pcapkit.const`, and inline in `pcapkit.protocols`) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in `docs/source/pcapkit/const/index.rst` cross-referenced from `registry.rst` (`conf.py` already excludes `_missing_` from every `autoclass`). `CONTRIBUTING.md` gained the rule as a named section. The new Sphinx warning, a bare `Type` newly rendered by `TraceFlow._foutio`'s directive, is [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709). No line under `pcapkit/` changed ([#684](https://github.com/JarryShaw/PyPCAPKit/issues/684)). - `examples/captures/out.json`, `out.plist`, `out.txt` and `pcapng.txt` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. `pcapng.txt` is exactly that: it still recorded `packet -> NIL` for the Enhanced Packet Blocks of `dhcp.pcapng` long after [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) gave those blocks their captured octets back. Rather than regenerate it again and leave the drift free to recur, these files are removed from the index and folded into `examples/captures/`'s blanket `.gitignore` rule; the genuine inputs `in.pcap` and `dhcp.pcapng` stay tracked. `examples/legacy_smoke/Makefile` and its `README.md` now describe regenerating these reports via `make fixtures`, and `tests/project/test_capture_tracking.py` pins the invariant ([#685](https://github.com/JarryShaw/PyPCAPKit/issues/685)). - `tests/project/test_isort_clean.py` gated on `isort` with a `try`/`except ImportError` inside `setUpClass`, which `tests/_dependency_gates.py` cannot see (`_gates_of()` walks a decorator list, never a function body), and `isort` was in no `pyproject.toml` extra, so the file reported `OK (skipped=1)` on every CI leg and the dependency-gate guard could not tell. It now gates on a module-level `HAS_ISORT` and a class-level `@unittest.skipUnless` so `gated_scopes()` counts it, `MODULE_PROVIDERS` maps `isort`, and `isort` joins the `test` extra rather than being excluded (`lint.yml` installs none, so unlike `mypy` there is no existing check to defer to). No workflow changed, every pytest job already building on `.[test]`; `gated_scopes()` gains this scope and `dependency_gate_gaps()` is unchanged. Another instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop ([#766](https://github.com/JarryShaw/PyPCAPKit/issues/766)). -- `tests/vendor/test_vendor_reg_apptype_generator_unit.py` gated on `mypy` with no `MODULE_PROVIDERS` entry, so `_top_level_providers()` died on a bare `KeyError: 'mypy'` once the gate became visible -- a further instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop, like [#766](https://github.com/JarryShaw/PyPCAPKit/issues/766) above. `MODULE_PROVIDERS['mypy'] = ('mypy',)` fixes the lookup, and the bare `MODULE_PROVIDERS[...]` subscripts route through that function, which raises an `AssertionError` naming the module, key and table on any future gap. `mypy` is deliberately not added to a `pyproject.toml` extra: no extra carries it, it is a `Pipfile` dev-dependency that `lint.yml` installs and already runs whole-package, and `lint.yml` runs no pytest job for `pytest_jobs()` to see, so the fix is a `DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']` entry on `test`, `engine-tests` and `gate`. `dependency_gate_gaps()` gains the new `(flag, job)` pairs, each expected; ([#779](https://github.com/JarryShaw/PyPCAPKit/issues/779)). +- `tests/vendor/test_vendor_reg_apptype_generator_unit.py` gated on `mypy` with no `MODULE_PROVIDERS` entry, so `_top_level_providers()` died on a bare `KeyError: 'mypy'` once the gate became visible -- a further instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop, like [#766](https://github.com/JarryShaw/PyPCAPKit/issues/766) above. `MODULE_PROVIDERS['mypy'] = ('mypy',)` fixes the lookup, and the bare `MODULE_PROVIDERS[...]` subscripts route through that function, which raises an `AssertionError` naming the module, key and table on any future gap. `mypy` is deliberately not added to a `pyproject.toml` extra: no extra carries it, it is a `Pipfile` dev-dependency that `lint.yml` installs and already runs whole-package, and `lint.yml` runs no pytest job for `pytest_jobs()` to see, so the fix is a `DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']` entry on `test`, `engine-tests` and `gate`. `dependency_gate_gaps()` gains the new `(flag, job)` pairs, each expected ([#779](https://github.com/JarryShaw/PyPCAPKit/issues/779)). - `engine='pyshark'` had no CI job driving it through a real `tshark`-parsed capture. `pypcap-parity` installs `pyshark` but its `apt-get install` named no `tshark` package, so `PyShark.unsupported_reason()` declined the engine, the extractor fell back to the built-in parser, and the only real-extraction assertion, `assertGreater(extractor.length, 0)`, passed against the fallback's output: a vacuous pass invisible in every skip count. `pypcap-parity` now installs `tshark` with the debconf pre-seed `engine-tests` carries, and the test proves which engine ran (no `EngineWarning`, `extractor._exnam == 'pyshark'`, a `PyShark` instance), as the PyPCAP parity test does. Only `tests/integration/test_engine_runtime.py` and `test_engine_parity.py` change; the length assertion is kept ([#846](https://github.com/JarryShaw/PyPCAPKit/pull/846)). - `engine-tests` ran a Python-version leg per interpreter regardless of which engines that interpreter could install, so `pyproject.toml`'s `PyPCAPFile = ["pypcapfile; python_version < '3.12'"]` marker resolved to *nothing* on 3.12--3.14, yet the `Engines Python 3.12/3.13/3.14` legs passed and, once required, gated merges while exercising an engine that was never installed. Rebuilt as a Python x engine matrix in which every cell has an expected outcome: `supported` cells install through the real extra and must pass; `unsupported` cells (`PyPCAPFile` on 3.12--3.15, `PyShark` on 3.14/3.15) install by raw `pip install` to bypass the marker and assert the engine's own `PYTHON_CEILING` decline; and `not-installable` cells (`PyPCAP` on 3.12--3.15) log a genuine C-extension build failure and skip the test step. The install step never passes silently in either direction. Python 3.15 is `continue-on-error` throughout. Job names change shape (`Engines Python X` becomes `Engines Python X (engine)`), so the branch-protection ruleset's required-check list was updated in the same window, lest a required check that stopped existing block every merge ([#849](https://github.com/JarryShaw/PyPCAPKit/pull/849)). - Ruleset `23497679`'s `required_status_checks` names exact check contexts, and GitHub rulesets match literally with no wildcard, so the list needs hand-editing on every matrix change. The old `Engines Python ` names in it are dead since [#849](https://github.com/JarryShaw/PyPCAPKit/pull/849); the `Compat Python 3.10`-`3.14` names stay live, emitted by `python-compatibility.yml`, and are untouched. Adds a job, `required-checks` (context `Required checks passed`), depending on `[test, integration, engine-tests, pypcap-parity]` with `if: always() && inputs.gate-only != true` and an explicit per-dependency check that fails loudly, naming the job, on anything but `success`: a bare `needs:` list would leave it *skipped* rather than failed when a dependency failed, and GitHub's docs list a skipped required check as passing. That context is meant to replace the ruleset's `test`, `integration`, `Engines` and `pypcap-parity` entries once the ruleset is updated separately, as a repository setting outside the change; the `Compat` names are excluded from the replacement. Only `.github/workflows/unit-tests.yml` changes; no library or test code ([#856](https://github.com/JarryShaw/PyPCAPKit/pull/856)). - the prose the post-[#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) defect wave added or touched, checked against the code; no code changes. `MesgIDOption.seconds` in the Mobility Header schema said seconds since 1970 where it holds an [RFC 1305](https://datatracker.ietf.org/doc/html/rfc1305) NTP count from 1900; `HTTP._guess_version` comments predated the HTTP/2 gate; and the `_parse_next_layer` note miscounted the captured octets. `Extractor._open_output` now says only the PCAP engine passes header fields, `Extractor.run` and the PCAP-NG engine's `run` list the `FormatError` for PCAP output from a PCAP-NG input, and the extraction and engine pages document the PCAP-output path ([#1179](https://github.com/JarryShaw/PyPCAPKit/pull/1179), [#1180](https://github.com/JarryShaw/PyPCAPKit/pull/1180), [#1186](https://github.com/JarryShaw/PyPCAPKit/pull/1186), [#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103)). - **a breaking change to** the `all` extra: it carries the core addons only -- `cli`, `crypto` and `NGAP`. The third-party engines 1.4.x bundled in it (`dpkt`, `scapy` and `pyshark`) and the vendor crawler's `requests` and `beautifulsoup4` are no longer in it; the engines keep their own `DPKT`, `Scapy` and `PyShark` extras and the crawler its `vendor` extra, and `pypcapfile` has a new `PyPCAPFile` extra of its own. A new `dev` extra carries what CI needs ([#910](https://github.com/JarryShaw/PyPCAPKit/issues/910), [#914](https://github.com/JarryShaw/PyPCAPKit/pull/914)). -- the `Scapy` and `dev` extras require `scapy>=2.6.0` ([#1572](https://github.com/JarryShaw/PyPCAPKit/issues/1572), [#1578](https://github.com/JarryShaw/PyPCAPKit/pull/1578)), and the `docs` extra `Sphinx>=7.3`, the first release with `versionremoved` ([#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). +- the `Scapy` and `dev` extras require `scapy>=2.6.0` ([#1572](https://github.com/JarryShaw/PyPCAPKit/issues/1572), [#1578](https://github.com/JarryShaw/PyPCAPKit/pull/1578)), and the `docs` extra `Sphinx>=7.3`, the first release with `versionremoved` ([#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)), and `sphinx-autodoc-typehints==3.15.0` ([#1856](https://github.com/JarryShaw/PyPCAPKit/issues/1856), [#1877](https://github.com/JarryShaw/PyPCAPKit/pull/1877)). The `test` extra gains `psutil`, `markdown-it-py` and `lxml`, which the scripts under `util/` use ([#1894](https://github.com/JarryShaw/PyPCAPKit/issues/1894), [#1896](https://github.com/JarryShaw/PyPCAPKit/pull/1896)). - the READMEs of `examples/benchmark` and `examples/legacy_smoke` are Markdown, `README.md` rather than `README.rst`, and `MANIFEST.in` ships them through `recursive-include examples *.md`. Stale build leftovers go: `setup.py`'s `bdist_wheel` override, since `build_py` and `install` already convert the tree the wheel archives, and `MANIFEST.in`'s `prune test`, which named no directory; `make profile` runs in `examples/legacy_smoke` ([#1773](https://github.com/JarryShaw/PyPCAPKit/issues/1773), [#1780](https://github.com/JarryShaw/PyPCAPKit/pull/1780)). `MANIFEST.in` also drops the lines that matched nothing -- `global-include *.rst`, `include pcapkit/_extern` and `exclude pcapkit/protocols/*/NotImplemented` -- and the source distribution and wheel are unchanged ([#1794](https://github.com/JarryShaw/PyPCAPKit/issues/1794), [#1813](https://github.com/JarryShaw/PyPCAPKit/pull/1813)). - the contributor pages state no counts that a test or the Python version updater pinned, and each pin checks the fact it stood for instead ([#1811](https://github.com/JarryShaw/PyPCAPKit/issues/1811), [#1817](https://github.com/JarryShaw/PyPCAPKit/pull/1817)). @@ -419,12 +428,13 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - `pcapkit.all.__all__` listed `OSPF`, `RARP` and `DRARP` on its Link Layer line after [#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) moved them ([#1124](https://github.com/JarryShaw/PyPCAPKit/issues/1124)), and omitted names its packages export, among them `ESP`, `HTTPv1`, `HTTPv2`, `PCAPNG` and `EnumRegistry`, so `from pcapkit.all import *` did not bind them. `pcapkit.utilities` stays unlisted deliberately, so generic helpers such as `warn` do not arrive through `import *` ([#1136](https://github.com/JarryShaw/PyPCAPKit/issues/1136)). - `examples/generators/options.py` wrote every HTTP/2 sample frame on stream 0, which [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) forbids for `DATA`, `HEADERS` and the other stream-level types, so those frames in `options-transport.pcap` decoded as `TCP:Raw`. `_httpv2_build` now puts `SETTINGS`, `PING` and `GOAWAY` on stream 0 and every other type on stream 1, with `PUSH_PROMISE` promising stream 2, and every HTTP/2 sample frame decodes as `TCP:HTTP/2`; no other generated capture changes ([#1156](https://github.com/JarryShaw/PyPCAPKit/issues/1156)). - `make_samples.py` builds fixtures from the tree it runs in, not the shared checkout ([#1343](https://github.com/JarryShaw/PyPCAPKit/issues/1343), [#1388](https://github.com/JarryShaw/PyPCAPKit/pull/1388)), and retries the Wireshark sample downloads on 429, 5xx and timeouts ([#1658](https://github.com/JarryShaw/PyPCAPKit/issues/1658), [#1659](https://github.com/JarryShaw/PyPCAPKit/pull/1659)). -- the Conda Update workflow tags and uploads each bump exactly once ([#1632](https://github.com/JarryShaw/PyPCAPKit/issues/1632), [#1641](https://github.com/JarryShaw/PyPCAPKit/pull/1641)), stays green when a merge races its push ([#1649](https://github.com/JarryShaw/PyPCAPKit/issues/1649), [#1669](https://github.com/JarryShaw/PyPCAPKit/pull/1669)), and keeps the release's name and notes when it uploads ([#1657](https://github.com/JarryShaw/PyPCAPKit/issues/1657), [#1678](https://github.com/JarryShaw/PyPCAPKit/pull/1678)). Vendor Update commits the new modules a crawl adds ([#1685](https://github.com/JarryShaw/PyPCAPKit/issues/1685), [#1688](https://github.com/JarryShaw/PyPCAPKit/pull/1688)). +- the Conda Update workflow tags and uploads each bump exactly once ([#1632](https://github.com/JarryShaw/PyPCAPKit/issues/1632), [#1641](https://github.com/JarryShaw/PyPCAPKit/pull/1641)), stays green when a merge races its push ([#1649](https://github.com/JarryShaw/PyPCAPKit/issues/1649), [#1669](https://github.com/JarryShaw/PyPCAPKit/pull/1669)), and keeps the release's name and notes when it uploads ([#1657](https://github.com/JarryShaw/PyPCAPKit/issues/1657), [#1678](https://github.com/JarryShaw/PyPCAPKit/pull/1678)). Vendor Update commits the new modules a crawl adds ([#1685](https://github.com/JarryShaw/PyPCAPKit/issues/1685), [#1688](https://github.com/JarryShaw/PyPCAPKit/pull/1688)), and runs `tests/const` and `tests/vendor` on the regenerated tree first, keeping only the modules they pass with ([#1853](https://github.com/JarryShaw/PyPCAPKit/issues/1853), [#1879](https://github.com/JarryShaw/PyPCAPKit/pull/1879)). - package `__all__` lists match what each package holds: several disagreed with it, and `from pcapkit.protocols.data import *` raised `AttributeError` on the unbound `HOPOPT_QuickStartOption` ([#515](https://github.com/JarryShaw/PyPCAPKit/issues/515), [#527](https://github.com/JarryShaw/PyPCAPKit/pull/527)); the public attributes several packages left out are listed ([#533](https://github.com/JarryShaw/PyPCAPKit/issues/533), [#544](https://github.com/JarryShaw/PyPCAPKit/pull/544)). - the documentation build's "more than one target found" warnings, fixed where they arise: an implicit reference to `dict`, `type` or `set` no longer matches a registry member such as `TCP.dict`, an enum member's docstring holding a URL no longer gains a spurious type field, and the hand-written `TypeVar` directives take the bare name under their `.. module::` ([#1775](https://github.com/JarryShaw/PyPCAPKit/issues/1775), [#1790](https://github.com/JarryShaw/PyPCAPKit/pull/1790)). - the Mermaid diagrams render at their natural size, shrunk only to the page width, where each was fitted into a fixed box, and their `click` targets are relative to the page, so they resolve under the site's `/PyPCAPKit/` prefix ([#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103), [#1819](https://github.com/JarryShaw/PyPCAPKit/pull/1819)). +- the API reference renders what the code declares. `Extractor._vfunc` keeps its annotation ([#1839](https://github.com/JarryShaw/PyPCAPKit/issues/1839), [#1867](https://github.com/JarryShaw/PyPCAPKit/pull/1867)), and the HTTP/2 frame schemas' and the engine classes' annotations resolve ([#1857](https://github.com/JarryShaw/PyPCAPKit/issues/1857), [#1870](https://github.com/JarryShaw/PyPCAPKit/pull/1870)). Under the pinned sphinx-autodoc-typehints 3.15.0 ([#1856](https://github.com/JarryShaw/PyPCAPKit/issues/1856), [#1877](https://github.com/JarryShaw/PyPCAPKit/pull/1877)), `conf.py` renders an `aenum` base as such, keeps a default as written (`byteorder=sys.byteorder` rather than the build host's value), gives a class the parameters of the `__init__` its `__new__` hides and an enumeration `(*values)`, keeps table ids unique, and stops the source index linking a missing page ([#1858](https://github.com/JarryShaw/PyPCAPKit/issues/1858) to [#1863](https://github.com/JarryShaw/PyPCAPKit/issues/1863), [#1881](https://github.com/JarryShaw/PyPCAPKit/pull/1881)). The generated const tables carry no version directives; each note is on the module's hand-written page instead ([#1884](https://github.com/JarryShaw/PyPCAPKit/issues/1884), [#1895](https://github.com/JarryShaw/PyPCAPKit/pull/1895)). - `pcapkit-cli`'s help strings say what each option does: `-o` describes the output rather than the input, an extension is appended only under `--auto-extension`, and each format flag names the flags that take precedence over it ([#1796](https://github.com/JarryShaw/PyPCAPKit/issues/1796), [#1810](https://github.com/JarryShaw/PyPCAPKit/pull/1810)). -- the Conda package: its vendored dependencies are the package's `dependencies` and its `all` extra, with their requirements, shipped with their data files rather than as `.py` files only ([#1814](https://github.com/JarryShaw/PyPCAPKit/issues/1814), [#1821](https://github.com/JarryShaw/PyPCAPKit/pull/1821)); the `osx-64` package is built on an Intel runner, where converting the arm64 build shipped arm64 binaries ([#1822](https://github.com/JarryShaw/PyPCAPKit/issues/1822), [#1825](https://github.com/JarryShaw/PyPCAPKit/pull/1825)); and on Intel macOS it pins `cryptography` 48.0.1, which has a wheel for that platform where the newer pin has none ([#1827](https://github.com/JarryShaw/PyPCAPKit/issues/1827), [#1828](https://github.com/JarryShaw/PyPCAPKit/pull/1828)). +- the Conda package: its vendored dependencies are the package's `dependencies` and its `all` extra, with their requirements, shipped with their data files rather than as `.py` files only ([#1814](https://github.com/JarryShaw/PyPCAPKit/issues/1814), [#1821](https://github.com/JarryShaw/PyPCAPKit/pull/1821)); the `osx-64` package is built on an Intel runner, where converting the arm64 build shipped arm64 binaries ([#1822](https://github.com/JarryShaw/PyPCAPKit/issues/1822), [#1825](https://github.com/JarryShaw/PyPCAPKit/pull/1825)); and on Intel macOS it pins `cryptography` 48.0.1, which has a wheel for that platform where the newer pin has none ([#1827](https://github.com/JarryShaw/PyPCAPKit/issues/1827), [#1828](https://github.com/JarryShaw/PyPCAPKit/pull/1828)). `util/conda_vendor.py --check` reports a package pinned differently under one marker ([#1852](https://github.com/JarryShaw/PyPCAPKit/issues/1852), [#1865](https://github.com/JarryShaw/PyPCAPKit/pull/1865)); the platform pins move to `conda/platform-pins.toml`, each with a review-by date, and Conda Update moves them up ([#1851](https://github.com/JarryShaw/PyPCAPKit/issues/1851), [#1885](https://github.com/JarryShaw/PyPCAPKit/pull/1885)); and the release's conda leg installs none of pcapkit's extras and smoke-tests the package before uploading it ([#1854](https://github.com/JarryShaw/PyPCAPKit/issues/1854), [#1855](https://github.com/JarryShaw/PyPCAPKit/issues/1855), [#1868](https://github.com/JarryShaw/PyPCAPKit/pull/1868)). --- diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index cb4e3dc68..157bb7824 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -17,7 +17,8 @@ so it has to keep existing, stay newest-first, and list exactly the pages the summary list does. - The preamble below is verbatim from the single-file changelog it replaces. + The preamble below is the single-file changelog's, which this page replaces, + with its notes set as ``.. note::`` admonitions. ========= Changelog @@ -25,11 +26,15 @@ Changelog All notable changes to PyPCAPKit are recorded here. - **Note** -- Versions before 1.5.0 are reconstructed from the git history, so +.. note:: + + Versions before 1.5.0 are reconstructed from the git history, so they summarise each release rather than enumerate every change. The record starts at 0.13.0 (2018-12-08); the earlier 0.x releases are not covered. - **Note** -- Post-releases (``X.Y.Z.postN``) are, from 1.0.1 onwards, automated +.. note:: + + Post-releases (``X.Y.Z.postN``) are, from 1.0.1 onwards, automated publications of the weekly registry refresh: a bot regenerates the vendor constant enumerations under ``pcapkit.const`` from the upstream IANA registries and bumps the version. They carry no library changes, and are diff --git a/docs/source/changelog/1.4.0.rst b/docs/source/changelog/1.4.0.rst index 20e88b726..c525a603e 100644 --- a/docs/source/changelog/1.4.0.rst +++ b/docs/source/changelog/1.4.0.rst @@ -13,7 +13,11 @@ a test suite. ``NameError: name 'Schema' is not defined`` (the import was type-checking-only), and past that, building a Window Scale or any Multipath TCP option, or passing a prebuilt option schema, raised too, and the TCP Fast Open cookie's length - check accepted lengths :rfc:`7413` rules out. + check accepted lengths :rfc:`7413` rules out. The same change fixed HOPOPT and + IPv6-Opts parsing, whose ``read`` took the schema class rather than the parsed + header and whose option readers could not take the ``options`` keyword they were + called with, and HTTP/2 construction, which computed a frame's length without its + flags. * **Changed** -- CI tests current Python versions again; every GitHub Actions job carries an explicit permissions block; the conda and cron release tooling was repaired, and Anaconda uploads use the official upload action. diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index ed608ca95..c616c1712 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -5,7 +5,7 @@ The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull -requests from :pr:`340` to :pr:`1828`. +requests from :pr:`340` to :pr:`1895`. Preceded by ``1.5.0a1`` (2026-09-15), ``1.5.0b1`` to ``1.5.0b3`` (all 2026-09-19) and ``1.5.0b4`` to ``1.5.0b13`` (2026-09-20 to 2026-10-16), dated in UTC and all @@ -13,10 +13,9 @@ published as prereleases and so resolved only by ``pip install --pre``. ``1.5.0b1`` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because ``twine check`` found a Sphinx-only ``:mod:`` role in ``README.rst``, which ``pyproject.toml`` declares as the -dynamic long description. ``1.5.0b2`` -reshipped it -- an existing version cannot republish -- and ``1.5.0b3`` followed -the CI change that stops a TestPyPI outage from costing a release its wheels -(:pr:`497`, :pr:`498`). +dynamic long description. ``1.5.0b2`` reshipped it -- an existing version cannot +republish -- and ``1.5.0b3`` followed the CI change that stops a TestPyPI outage from +costing a release its wheels (:pr:`497`, :pr:`498`). pcapkit.const ------------- @@ -54,8 +53,8 @@ Changed ``vendor/default.py`` still carry it, so :issue:`804`'s claim that nothing in ``pcapkit/{const,vendor}/{ftp,http}/`` needs it holds for this pair only; ``repr()`` output is unchanged (:issue:`804`). -* ``test_const_enum_no_mint.py``'s ``is_manufactured()`` exemption for - :pr:`878`'s (above) preserved hex-suffixed names moves from a per-*file* to a +* ``test_const_enum_no_mint.py``'s ``is_manufactured()`` exemption for the + hex-suffixed names :pr:`878` (below) preserves moves from a per-*file* to a per-*name-shape* one. The per-file ``HEX_SUFFIXED_NAME_EXEMPT_PATHS`` was shape-blind: rewriting a branch to ``'Xyplex_%d' % value`` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check @@ -85,6 +84,9 @@ Changed and the two disagree on 147 (``BIT-EMU``) (:issue:`925`, :pr:`926`). * the TCP and UDP ``AppType`` registries load on first use rather than at import (:issue:`1538`, :pr:`1560`). +* **a breaking change to** ``LLACode``: ``Wilcard`` is spelled ``Wildcard`` and ``MH`` + is ``MN``, the names :rfc:`5568` gives codes 0 and 2, with no alias for either old + name (:issue:`1848`, :pr:`1872`). Fixed ~~~~~ @@ -95,8 +97,7 @@ Fixed have; IPX ``Socket(0)`` is that protocol's own default, so ``bytes(IPX(...))`` crashed on its own defaults; and the FTP ``_missing_`` overrides were plain methods rather than classmethods, so every unregistered value raised ``TypeError`` - instead - of extending the enumeration (:issue:`492`, :pr:`503`). + instead of extending the enumeration (:issue:`492`, :pr:`503`). * the string-keyed ``get()`` in ``pcapkit.const.ftp.command`` and ``pcapkit.const.http.method`` tested membership with the raw key but registered ``key.upper()``, so the *first* lowercase or mixed-case token @@ -109,7 +110,7 @@ Fixed lacked the real one's ``safe`` and ``idempotent`` attributes (:issue:`582`, :issue:`583`). ``http.method`` no longer folds case: :rfc:`9110#section-9.1` makes the method token case-sensitive, so ``Method.get`` matches it exactly - (:issue:`896`, below). + (:issue:`896`, above). * ``get()``'s documented ``default`` was ignored on the integer path across the generated ``pcapkit.const`` tree, because ``_missing_`` cannot see the caller's ``default`` -- ``Hardware.get(99999, 0)`` raised ``ValueError`` instead of returning @@ -118,10 +119,10 @@ Fixed default" from "a default"; :pr:`859` replaced it with an identity sentinel once the shared base :pr:`858` (below) put ``get()`` behind one call site. The sweep covers the integer registries generated from the shared template, not only those :issue:`584` - named; ``pcapng`` - ``OptionType`` and ``reg`` ``AppType`` carry a bespoke integer fallback and are - deliberately left alone, since neither drops a default by raising. Not reachable - from wire data, so a contract fix rather than a parse fix (:issue:`584`). + named; ``pcapng`` ``OptionType`` and ``reg`` ``AppType`` carry a bespoke integer + fallback and are deliberately left alone, since neither drops a default by raising. + Not reachable from wire data, so a contract fix rather than a parse fix + (:issue:`584`). * ``_missing_`` in the Mobility Header flag enumerations (``BindingACKFlag``, ``BindingUpdateFlag``, ``HandoverACKFlag``, ``HandoverInitiateFlag``) ended in ``return cls(value)``, the constructor that had just failed, so every in-range value @@ -171,7 +172,8 @@ Fixed through; a valid but unassigned port still minted, until :pr:`874` (below) made it an unregistered member. ``Transport._make_port`` **passes an unvalidated** ``int``, **so** ``TCP.make(srcport=99999)`` **now raises - where it minted junk.** Separately, the crawler rendered one ``_missing_`` branch per + where it minted junk**; in 1.4.1, ``TCP(srcport=N)`` raised ``AttributeError`` for + any bare ``int``. Separately, the crawler rendered one ``_missing_`` branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: ``AppType.get(6010, proto='udp')`` gave a member whose ``proto`` read ``tcp``, @@ -199,9 +201,9 @@ Fixed updated; the non-minting ``get()`` miss applies to all of them, but the ``_missing_`` fix changes behaviour only for those with a bounded-range ``extend_enum`` branch. ``pcapkit/toolkit/scapy.py`` and - ``pcapkit/toolkit/pyshark.py`` fed - ``get()`` a string key with no default and now raise ``MissingKeyError`` instead of - receiving a placeholder; the reader sites in ``hopopt.py``/``ipv6_opts.py`` convert + ``pcapkit/toolkit/pyshark.py`` fed ``get()`` a string key with no default and now + raise ``MissingKeyError`` instead of receiving a placeholder; the reader sites in + ``hopopt.py``/``ipv6_opts.py`` convert the ``KeyError`` into ``ProtocolError``. Riding along: ``pcapkit/toolkit/pyshark.py`` gains a ``FILTER_NAME_TO_LINKTYPE`` map (``eth`` to ``ETHERNET``, ``tr`` to ``IEEE802_5``), consulted ahead of @@ -250,8 +252,7 @@ Fixed IANA-style range, and the files holding the classes :issue:`860` moves onto ``EnumRegistry`` (the ``AppType``, ``StatusCode``, ``ftp.return_code``, ``ftp.command``, ``Method`` and ``OptionType`` modules; :pr:`869` and :pr:`874`, - below). - Each conversion changes both the vendor crawler and the generated file, per + below). Each conversion changes both the vendor crawler and the generated file, per :pr:`838`'s and :pr:`858`'s precedent. The worked examples in ``docs/source/contributing/conventions/mint-criterion.rst`` keep ``Registered by Xerox`` under Mint (:pr:`861`). @@ -344,9 +345,9 @@ Fixed because no registry overrode the ``_validate_value`` hook ``register`` calls. Each generated registry, ``AppType`` included, now overrides it with the check its ``_missing_`` makes, raising ``EnumValueError``. ``get(value, default)`` on a value - the hook refuses still returns ``default``, but now also logs the - ``EnumValueError`` at ``CRITICAL``, as any loud pcapkit error is (:issue:`1792`, - :pr:`1818`, :issue:`1816`, :pr:`1823`). + the hook refuses still returns ``default`` and logs nothing (:issue:`1792`, + :pr:`1818`, :issue:`1816`, :pr:`1823`; the quiet miss is :issue:`1845`, + :pr:`1873`, under ``pcapkit.corekit``). * ``Command.get`` refuses a key that is not a ``str`` with ``EnumValueError``, as ``Method.get`` does. ``get(b'\x01')`` minted a member keyed by ``bytes``, and ``get(-1)`` raised ``AttributeError`` whatever its ``default`` (:issue:`1824`, @@ -442,9 +443,10 @@ Changed ``prepare`` are no longer importable from ``pcapkit.utilities``, and the module logs as ``pcapkit.corekit.decorators`` (:issue:`1735`, :pr:`1737`). * **a breaking change to** the sentinels: all live in ``pcapkit.corekit.sentinels``, - which exports the objects ``NULL``, ``NO_VALUE`` and ``NO_DEFAULT`` but not their - types, and ``NoValue`` is spelled ``NO_VALUE`` (:pr:`916`, :issue:`911`, - :pr:`922`, :issue:`937`, :pr:`939`). No moved name keeps a re-export, so + which exports the objects ``NULL``, ``NO_VALUE`` and ``NO_DEFAULT`` and their + common base ``Sentinel``, but not their types, and ``NoValue`` is spelled + ``NO_VALUE`` (:pr:`916`, :issue:`911`, :pr:`922`, :issue:`937`, :pr:`939`). No + moved name keeps a re-export, so ``NoValueType`` has only that home, not its 1.4.x one in ``pcapkit.corekit.fields.field`` (:issue:`1744`, :pr:`1754`). @@ -561,10 +563,9 @@ Fixed crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one - layer - up, where the missing information exists: a PCAP-NG block's Block Total Length is - authoritative and cross-checked against its trailing copy, so the option area is that - length less the fixed fields, ``captured_len`` and its padding, and an option + layer up, where the missing information exists: a PCAP-NG block's Block Total Length + is authoritative and cross-checked against its trailing copy, so the option area is + that length less the fixed fields, ``captured_len`` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through ``captured_len`` and leaves its options whole, so it never trips this -- which the :pr:`571` ``len(buffer) < length`` rejection could not @@ -595,7 +596,8 @@ Fixed ``application/{ftp,httpv1,httpv2,ngap}.py``, ``internet/{hip,ipv6_route,mh}.py``, ``link/ethernet.py``, ``misc/pcapng.py`` and ``transport/sctp.py`` with no other file changed; a non-negative length is unaffected. This closes the follow-on the - ``httpv2`` length-guard entry above filed as out of scope (:issue:`805`). + ``httpv2`` length-guard entry under ``pcapkit.protocols`` filed as out of scope + (:issue:`805`). * **a breaking change to** ``EnumRegistry.register()``: it now raises ``ValueError`` naming the existing member and pointing at ``register_alias()``, where it used to silently alias an already-registered value under the caller's new name -- ``aenum`` @@ -604,7 +606,7 @@ Fixed minted nothing and raised nothing, contradicting its docstring. ``pcapkit.corekit.enum.EnumRegistry`` is a new mixin implementing ``get``, ``get_all``, ``register``, ``register_alias``, ``register_aliases`` and - ``_unregistered_member`` once, per the maintainer's ruling on :issue:`842`; it + ``_unregistered_member`` once, per the owner's ruling on :issue:`842`; it first converts registries with no bespoke ``__new__`` (``ipv6.extension_header.ExtensionHeader``, ``tcp.flags.Flags`` and the ``mh.*_flag`` modules), and serves ``IntEnum``, ``IntFlag`` and ``StrEnum`` alike. @@ -613,9 +615,8 @@ Fixed contradictory ``register`` contracts until :pr:`858` (below). These also gain the generated template's ``KeyError`` catching on a missing name, so ``Flags.get('NOPE', 2048)`` returns ``Flags.ACK`` rather than raising; and a key - that is neither - ``int`` nor ``str``, which raised ``TypeError`` on the ``IntFlag`` registries - (``Flags.get(3.5)``), now raises ``ValueError``, tried as a value. On + that is neither ``int`` nor ``str``, which raised ``TypeError`` on the ``IntFlag`` + registries (``Flags.get(3.5)``), now raises ``ValueError``, tried as a value. On ``ExtensionHeader`` it raised ``KeyError``, the shift :pr:`858`'s entry attributes to the other registries (:pr:`855`). * **a breaking change to** ``EnumRegistry.register()``, extended: the const @@ -629,11 +630,10 @@ Fixed ``__new__`` (``ftp/command``, ``ftp/return_code``, ``http/method``, ``http/status_code``, ``pcapng/option_type`` and ``reg/apptype`` among them) are left alone, so **every other class** inherits ``EnumRegistry`` (:pr:`855`). - ``get()``'s dispatch shifts only - for a key that is neither ``int`` nor ``str`` (``KeyError``, tried as a name; now - ``ValueError``, tried as a value), a key type nothing in the library passes. - ``pcapkit/corekit/enums.py`` is renamed ``enum.py``, the maintainer's ruling - (:pr:`858`). + ``get()``'s dispatch shifts only for a key that is neither ``int`` nor ``str`` + (``KeyError``, tried as a name; now ``ValueError``, tried as a value), a key type + nothing in the library passes. ``pcapkit/corekit/enums.py`` is renamed ``enum.py``, + the owner's ruling (:pr:`858`). * ``EnumRegistry.get()``'s ``str``-key branch tried only a name lookup (``cls._member_map_[key]``) and on a miss raised or returned ``cls(default)``, never the *value* path, so on a ``StrEnum`` registry a key that resolved through @@ -657,12 +657,12 @@ Fixed ``register()`` mints, and ``get()`` only through the ranges ``_missing_`` mints, each site now resolves ``default`` through a plain ``_value2member_map_`` lookup; ``key`` resolution is unchanged, so - ``EtherType.get(0x0888)`` still mints - ``Xyplex_0x0888`` through its own ``_missing_``, as :issue:`775`'s deliberate - exception permits. A ``default`` naming no registered member now falls through to - the error ``key`` itself would have raised, rather than a fresh error about the - default -- the accepted cost of the ruling. The docstring's "It never mints" is - qualified to ``default`` only. So ``Hardware.get('Definitely-Not-A-Member', 40)``, + ``EtherType.get(0x0888)`` still mints ``Xyplex_0x0888`` through its own ``_missing_``, + as :issue:`775`'s deliberate exception permits. A ``default`` naming no registered + member now falls through to the error ``key`` itself would have raised, rather than a + fresh error about the default -- the accepted cost of the ruling. The docstring's "It + never mints" is qualified to ``default`` only. So + ``Hardware.get('Definitely-Not-A-Member', 40)``, with ``40`` in ``Hardware``'s declared-but-unassigned range, now raises ``KeyError`` naming the original key; its test waited on :pr:`865` (above), then landed once :pr:`865` merged. Closes :issue:`864` (:pr:`868`). @@ -724,6 +724,17 @@ Fixed which is a bare ``int`` -- raised ``AttributeError`` on a port the registry does not hold. The miss takes its transport from the registry's ``__transport__``, as ``AppType.get`` does (:issue:`1774`, :pr:`1781`). +* ``EnumLookup.get`` answers a value miss quietly, whether ``default`` answers it or + the miss is raised: a registry's ``_validate_value`` rejection logs nothing and + installs no exception hook there, while ``EnumRegistry.register`` still reports the + same rejection loudly (:issue:`1845`, :pr:`1873`). +* each sentinel is a singleton, as ``None`` is: calling its type, ``copy``, + ``deepcopy`` and ``pickle`` at every protocol give back the one object, and + ``NO_VALUE`` reprs as its own name rather than as an object address. Their new + common base, ``Sentinel``, is public; a sentinel's type cannot be subclassed, and + neither it nor the sentinel takes an attribute write. A pickle written before this + change at protocol 0 or 1 still loads, though not as the singleton (:issue:`1841`, + :issue:`1842`, :pr:`1876`). pcapkit.dumpkit --------------- @@ -861,10 +872,9 @@ Changed an alias, which every built-in does (hence no public class had a subclass, while every built-in derives from its ``*Base`` class). **This breaks out-of-tree code that subclasses one of these classes and relies on the derived key**; pass the - keyword, or - call the matching ``register_*`` function. Nothing the library ships is affected, - and the ``*Base`` classes remain importable. Silent failures are now loud: an - unrecognised class keyword raises ``UnsupportedCall`` instead of being swallowed by + keyword, or call the matching ``register_*`` function. Nothing the library ships is + affected, and the ``*Base`` classes remain importable. Silent failures are now loud: + an unrecognised class keyword raises ``UnsupportedCall`` instead of being swallowed by ``**kwargs`` (passing ``name=`` to a ``Reassembly`` subclass silently ignored the key, since ``protocol=`` is the real one), and so does ``Dumper``'s ``ext=`` without ``fmt=``. A class attribute is not an opt-in: ``__engine_name__`` and @@ -882,16 +892,14 @@ Changed ``TraceFlow`` (:issue:`1016`). They had accepted a subclass of the matching ``*Base`` class since :issue:`513`, which was a tolerance for pcapkit's own built-ins (every built-in derives from the base, none from the public class) rather - than - a contract: ``*Base`` is documented as internal, and the public class is the one + than a contract: ``*Base`` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses ``EngineBase``, ``ReassemblyBase`` or ``TraceFlowBase`` directly and registers through these functions now gets ``RegistryError`` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; ``Extractor._register_internal_engine`` and its sibling internal - registrars check the - base for internal registration, and are what the :issue:`513` regression test now - exercises. ``register_protocol`` is untouched (:pr:`1020`). + registrars check the base for internal registration, and are what the :issue:`513` + regression test now exercises. ``register_protocol`` is untouched (:pr:`1020`). * extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output (:pr:`420`). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% @@ -905,11 +913,11 @@ Changed ``_exeng`` attribute follows. The built-in engines subclass ``EngineBase`` directly, so none was ever an ``Engine``, which adds only the ``__init_subclass__`` registration hook. A type checker sees the change: code passing the - result to an ``Engine``-typed - parameter must now accept ``EngineBase``; and because the old declarations were - *unparameterised*, ``engine.read_frame()`` and ``record_header().read_frame()`` - used to be ``Any`` and are now the extractor's own frame type, so an assignment - that relied on ``Any`` there no longer type-checks (:issue:`1022`). + result to an ``Engine``-typed parameter must now accept ``EngineBase``; and because + the old declarations were *unparameterised*, ``engine.read_frame()`` and + ``record_header().read_frame()`` used to be ``Any`` and are now the extractor's own + frame type, so an assignment that relied on ``Any`` there no longer type-checks + (:issue:`1022`). * the documentation of the engine base classes: the member stubs (``name``, ``module``, ``registry``, ``extractor``, ``unsupported_reason``, ``run``, ``read_frame``, ``close`` and the rest) moved from ``Engine`` to ``EngineBase``, @@ -1001,10 +1009,9 @@ Fixed ``pcapkit.const.l2tp.type.Type``, an L2TP field-type enum; once :issue:`684` rendered ``TraceFlow._foutio``, the built docs pointed at the wrong class with no warning. These sites now spell it ``~typing.Type[Dumper]``, among the sites - :issue:`709` lists (:issue:`709`). The other bare ``Type`` sites, in hand-written - ``:type:`` fields under - ``docs/source/pcapkit/foundation/``, are fixed separately by :pr:`714` - (:issue:`709`). + :issue:`709` lists. The other bare ``Type`` sites, in hand-written + ``:type:`` fields under ``docs/source/pcapkit/foundation/``, are fixed separately by + :pr:`714` (:issue:`709`). * ``register_protocol``'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (``incumbent is not protocol``, an identity check from :pr:`681`), but the message built both operands with a bare ``repr()``. A factory @@ -1028,28 +1035,25 @@ Fixed argument, where a bare ``TypeError`` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the ``register_dumper`` methods, and from inside ``abc`` for the others. They are ``register_dumper``, ``register_engine``, - ``register_reassembly`` and - ``register_traceflow`` on ``Extractor``, ``TraceFlow.register_dumper`` (reached + ``register_reassembly`` and ``register_traceflow`` on ``Extractor``, + ``TraceFlow.register_dumper`` (reached through ``register_traceflow_dumper``), and ``pcapkit.foundation.registry.protocols.register_protocol``. Each guard gains an explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that accept a ``ModuleDescriptor`` it runs after the descriptor is unwrapped, so a - descriptor - naming a non-class attribute is rejected too, while ``register_protocol`` takes no - descriptor. No guard's target class changes, and a wrong class still raises + descriptor naming a non-class attribute is rejected too, while ``register_protocol`` + takes no descriptor. No guard's target class changes, and a wrong class still raises ``RegistryError`` as before. ``RegistryError`` subclasses ``TypeError``, so ``except TypeError`` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocol ``register`` classmethods is fixed separately by :issue:`1026` (:issue:`1021`). * a missing extraction engine package now gives a single ``EngineWarning`` rather than - a duplicate - (:issue:`1045`, :pr:`1046`). ``Extractor.import_test`` warned that the package was - absent, then ``Extractor.run`` warned again when it fell back to the default engine; - ``run`` now stays silent there, and ``import_test`` says what ``run`` used to: - ``engine () is not installed; using default engine instead``, the - module name in backticks. Only - code that matched ``import_test``'s old wording, or that expects the duplicate - warning, breaks. + a duplicate (:issue:`1045`, :pr:`1046`). ``Extractor.import_test`` warned that the + package was absent, then ``Extractor.run`` warned again when it fell back to the + default engine; ``run`` now stays silent there, and ``import_test`` says what ``run`` + used to: ``engine () is not installed; using default engine instead``, + the module name in backticks. Only code that matched ``import_test``'s old wording, or + that expects the duplicate warning, breaks. * ``Extractor`` defects: ``Extractor.output``'s ``UnsupportedCall`` named ``format`` rather than ``output``; ``make_name`` looked an unknown format up in the ``__output__`` ``defaultdict`` and so inserted it; and the Scapy engine never set @@ -1058,12 +1062,11 @@ Fixed * the DPKT, Scapy and PyShark engines call ``Extractor.record_header()`` at setup again, as they did until 2023, so the global-header step is shared with the default engine and sets ``_offmt`` for each of them. **Their dumped output changes**: JSON, - plist and tree - reports gain the leading ``Global Header`` record (``Section Header 1`` for PCAP-NG) - the default engine already writes, identical to its, and ``files=True`` writes that - record's file too; frame records are unchanged. ``Extractor.format`` now also answers - on a capture with no frames. ``format='pcap'`` failing on every engine is a separate - defect, :issue:`1127` (below) (:issue:`1104`). + plist and tree reports gain the leading ``Global Header`` record (``Section Header 1`` + for PCAP-NG) the default engine already writes, identical to its, and ``files=True`` + writes that record's file too; frame records are unchanged. ``Extractor.format`` now + also answers on a capture with no frames. ``format='pcap'`` failing on every engine is + a separate defect, :issue:`1127` (below) (:issue:`1104`). * ``format='pcap'`` failed on every engine with a ``TypeError`` from ``PCAPIO.__init__()``, missing ``protocol``: ``Extractor.__init__`` created the writer before any global header was read, so ``PCAPIO`` had no link type. The header step @@ -1109,6 +1112,10 @@ Fixed blocks (:issue:`1515`, :pr:`1531`), and the pypcapfile engine reads the PCAP records pypcapfile drops (:issue:`1512`, :pr:`1569`). dpkt and scapy write real PCAP traces (:issue:`1507`, :pr:`1511`). +* a callable passed as ``verbose=`` is called on every engine. The scapy, dpkt, + pyshark, pypcap, pcap-ct and pypcapfile engines replaced it with their own printer, + which now runs only for ``verbose=True``, its output unchanged (:issue:`1874`, + :pr:`1880`). pcapkit.interface ----------------- @@ -1210,14 +1217,12 @@ Added ``tests/protocols/internet/test_ipv6_extension_unit.py``, a test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over a spread of nonces, most with a bit length that is *not* a multiple of - eight, - where floor division and the ceiling disagree (:issue:`601`). + eight, where floor division and the ceiling disagree (:issue:`601`). * ``SOLUTION`` parameter width coverage in ``tests/protocols/internet/test_hip_unit.py``: a spread of widths, each asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length :rfc:`5201#section-5.2.5` requires - of HIPv1. - Some widths are deliberately *not* multiples of eight, where the defective + of HIPv1. Some widths are deliberately *not* multiples of eight, where the defective ``ceil(bits / 4)`` differs from the correct width (:issue:`608`). * the BSD loopback layer for ``LINKTYPE_NULL`` and ``LINKTYPE_LOOP``, and dissection of ``LINKTYPE_RAW`` by the version nibble (:pr:`1594`). @@ -1250,13 +1255,13 @@ Changed ``layer`` changes from ``'Link'`` to ``'Application'`` for OSPF, RARP and DRARP. ``OSPF`` now takes ``Application`` as its only base, and ``RARP`` becomes - ``class RARP(Application, ARP)`` - -- layer base first, because ``ARP``'s chain reaches ``Link``, which owns - ``__layer__``. ``ARP`` and ``InARP`` stay ``'Link'``. Dispatch keys are unchanged: - OSPF is still reached from ``Internet.__proto__`` at ``TransType.OSPFIGP`` and RARP - from ``Link.__proto__`` by EtherType. Leaving ``Link`` repoints ``OSPF.__proto__`` at - ``ProtocolBase.__proto__``, so OSPF no longer sees Link's EtherType entries -- which - nothing used, since it is reached by protocol number, and a ``-1`` lookup falls back to + ``class RARP(Application, ARP)`` -- layer base first, because ``ARP``'s chain reaches + ``Link``, which owns ``__layer__``. ``ARP`` and ``InARP`` stay ``'Link'``. Dispatch + keys are unchanged: OSPF is still reached from ``Internet.__proto__`` at + ``TransType.OSPFIGP`` and RARP from ``Link.__proto__`` by EtherType. Leaving ``Link`` + repoints ``OSPF.__proto__`` at ``ProtocolBase.__proto__``, so OSPF no longer sees + Link's EtherType entries -- which nothing used, since it is reached by protocol + number, and a ``-1`` lookup falls back to ``Raw`` in either registry without inserting. ``register`` and ``_read_protos`` still resolve, on ``ProtocolBase`` rather than ``Link``. ``RARP`` keeps Link's through ``ARP``. @@ -1293,11 +1298,11 @@ Changed 'seq_no'?*). ``from_data`` warns ``UnknownFieldWarning`` rather than raising for a key its own ``_make_data`` returned, which the caller cannot fix; a keyword passed to ``from_data`` itself raises, as for a direct construction (:pr:`1147`). The leftover - copies of :issue:`602`'s misspelt TCP base, in ``examples/generators/dispatch.py`` and - ``tests/protocols/transport/``, are fixed. Some ``_make_data`` keys that ``make`` does - not declare are reported, not fixed: ``Frame._make_data`` returns ``ts_src`` where - ``make`` declares ``ts_sec``, ``L2TPv2._make_data`` returns ``prio`` where it declares - ``priority``, and + copies of :issue:`602`'s misspelled TCP base, in + ``examples/generators/dispatch.py`` and ``tests/protocols/transport/``, are fixed. + Some ``_make_data`` keys that ``make`` does not declare are reported, not fixed: + ``Frame._make_data`` returns ``ts_src`` where ``make`` declares ``ts_sec``, + ``L2TPv2._make_data`` returns ``prio`` where it declares ``priority``, and ``Header._make_data`` returns a ``magic_number`` that ``Header.make`` does not take. A *direct* ``SomeProtocol.make(...)`` call is not checked and still discards silently (:issue:`617`). @@ -1348,6 +1353,12 @@ Changed as :rfc:`8200#section-4.2` and :rfc:`6275#section-6.2.2` ask. Only a freshly built header with exactly two octets of padding changes; a parsed one that carried two ``Pad1`` options still rebuilds byte for byte (:issue:`1793`, :pr:`1812`). +* **a breaking change to** ``NTPTimestamp``: it and ``PMIPv6Timestamp`` are ``Info`` + classes in ``pcapkit.protocols.data.internet.mh`` rather than tuples, and + ``NTPTimestamp`` keeps no alias at its 1.4.x home in + ``pcapkit.protocols.internet.mh``. As Mappings they no longer index, unpack, compare + or hash as a tuple does; ``.seconds`` and ``.fraction`` still work, and the dumpers + write both as keys (:issue:`1886`, :pr:`1887`). Fixed ~~~~~ @@ -1393,9 +1404,8 @@ Fixed remaining (:issue:`431`, :pr:`432`). Separately, wire-derived lengths in ``ipv6_opts``, CALIPSO, MPL, REG_INFO and HIP list callbacks underflowed below zero, which ``ListField``'s own ``while length > 0`` turned into a - silent - empty list; they are floored and raise instead (:pr:`449`, :pr:`456`, :pr:`460`, - :issue:`463`). + silent empty list; they are floored and raise instead (:pr:`449`, :pr:`456`, + :pr:`460`, :issue:`463`). * field widths and units, each measured against the specification: HIP's ``TRANSPORT_FORMAT_LIST``, ``NAT_TRAVERSAL_MODE`` and ``ESP_TRANSFORM`` list entries are two octets, not one [:rfc:`7401`, :rfc:`5770`, :rfc:`7402`] @@ -1472,15 +1482,13 @@ Fixed -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where ``_read_mptcp_capable`` raised ``ProtocolError: TCP: [OptNo 30] invalid format`` for anything but 12 and 20. In the - schema ``skey`` is now present - only for ``length > 4`` and ``rkey`` for ``length > 12``, and the fields - ``dl_len`` (16 bits, ``length > 20``) and ``checksum`` (2 octets, ``length > 22``) - are new. - ``_make_mptcp_capable`` takes ``dl_len`` and ``checksum`` too, where it used to - drop them silently through ``**kwargs``, and picks the length from how many of - ``skey``, ``rkey``, ``dl_len`` and ``checksum`` are not ``None``. A field given - without the one before it raises ``ProtocolError``. The data model - ``MPTCPCapable`` gains ``dl_len`` and ``checksum``, and ``skey`` becomes + schema ``skey`` is now present only for ``length > 4`` and ``rkey`` for + ``length > 12``, and the fields ``dl_len`` (16 bits, ``length > 20``) and ``checksum`` + (2 octets, ``length > 22``) are new. ``_make_mptcp_capable`` takes ``dl_len`` and + ``checksum`` too, where it used to drop them silently through ``**kwargs``, and picks + the length from how many of ``skey``, ``rkey``, ``dl_len`` and ``checksum`` are not + ``None``. A field given without the one before it raises ``ProtocolError``. The data + model ``MPTCPCapable`` gains ``dl_len`` and ``checksum``, and ``skey`` becomes ``Optional[int]``: code that constructs it directly must now pass both new fields, and ``to_dict()`` carries the new keys (:issue:`1042`). * IPv6-Route's RPL routing data was five octets wide at the front where :rfc:`6554` @@ -1593,9 +1601,8 @@ Fixed the captured group (:issue:`583`). * as a consequence of the above, the unhandled ``MemoryError`` in ``pcapkit/protocols/protocol.py``'s ``self._file.read()`` call on a truncated - PCAP-NG capture. - ``dhcp_little_endian.pcapng`` cut to 161, 641 or 1389 octets left a one-octet - read of a little-endian 32-bit block length, which ``rjust()`` turned into + PCAP-NG capture. ``dhcp_little_endian.pcapng`` cut to 161, 641 or 1389 octets left a + one-octet read of a little-endian 32-bit block length, which ``rjust()`` turned into ``0x78000000`` or ``0x84000000`` (1.88 to 2.06 GiB) and passed straight to ``self._file.read()`` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise ``MemoryError``; with more space the @@ -1628,8 +1635,8 @@ Fixed for the narrower reason the note now gives: ``ConditionalField``'s ``length`` forwards to the wrapped field without consulting the condition, whereas a ``SwitchField`` always resolves to a concrete field (:issue:`603`). -* the assertion :issue:`604` left pinning the old padding side, red on ``main`` - since :pr:`621` merged. +* the assertion :issue:`604` left pinning the old padding side, which :pr:`621` + turned red on ``main``. ``TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length`` expected a truncated TCP option's ``data`` as the synthesised zero octets *followed by* the real ones, as ``rjust()`` produced: ``Reserved_79`` declaring ``length=12`` @@ -1695,12 +1702,11 @@ Fixed untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until :pr:`614` (:issue:`618`). * the HIP ``PUZZLE`` and ``SOLUTION`` builders derived wire-format values from the - payload - value instead of the data model, and each was wrong. **This changes the** + payload value instead of the data model, and each was wrong. **This changes the** ``PuzzleParameter`` **and** ``SolutionParameter`` **data models and the octets both - parameters emit.** The width of - ``Random #I`` and ``Puzzle solution #J`` came from ``int.bit_length()`` alone, so - every leading zero octet was dropped on re-serialisation: a ``SOLUTION`` read with + parameters emit.** The width of ``Random #I`` and ``Puzzle solution #J`` came from + ``int.bit_length()`` alone, so every leading zero octet was dropped on + re-serialisation: a ``SOLUTION`` read with ``Length = 20`` rebuilt as ``Length = 6``, a ``PUZZLE`` with ``Length = 12`` as ``Length = 5``, silently. That width is ``RHASH_len / 8`` octets [:rfc:`7401#section-5.2.4`, :rfc:`7401#section-5.2.5`], a property of the @@ -1754,24 +1760,22 @@ Fixed ``pcapkit/protocols/schema/internet/hip.py`` and the record lengths in ``pcapkit/protocols/internet/hip.py``) now goes through one ``parameter_total_len`` and one ``parameter_padding_len`` (:issue:`651`). ``LOCATOR_SET`` kept the old - expression - because defects there cancelled exactly: its padding callback never received the - parameter's ``len`` (the nested ``Locator`` schemas share a packet context whose own - ``len`` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's ``len`` - was written in 4-octet units where the RFC's ``Length`` is a byte count; the result, - ``4 + 24n + 4``, equalled the RFC total ``24n + 8``. :issue:`679` later fixed both - sites. ``EncryptedParameter``'s ``data`` length callback is fixed in the same - change: it subtracted the sixteen ``iv`` octets but not the four ``reserved`` ones, - which cancelled the padding's four at ``Length % 8`` in ``{0, 5, 6, 7}``, so + expression because defects there cancelled exactly: its padding callback never + received the parameter's ``len`` (the nested ``Locator`` schemas share a packet + context whose own ``len`` shadowed it, so it saw 4 for an IPv6 locator), and the + parameter's ``len`` was written in 4-octet units where the RFC's ``Length`` is a byte + count; the result, ``4 + 24n + 4``, equalled the RFC total ``24n + 8``. :issue:`679` + later fixed both sites. ``EncryptedParameter``'s ``data`` length callback is fixed in + the same change: it subtracted the sixteen ``iv`` octets but not the four ``reserved`` + ones, which cancelled the padding's four at ``Length % 8`` in ``{0, 5, 6, 7}``, so correcting the padding alone would have made ``ENCRYPTED`` four octets too long at all eight residues. ``HIP.make``'s ``len = total_length // 8 + 4`` is not part of the defect: :rfc:`7401#section-5.1.3` defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a ``SEQ`` parameter's ``length`` is now 8 where it was 12, and the other parameters move likewise, so code comparing stored output byte for - byte or - asserting on ``Data_*Parameter.length`` sees the RFC's values. ``LOCATOR_SET`` is - unchanged in both respects until :issue:`679` (below). + byte or asserting on ``Data_*Parameter.length`` sees the RFC's values. ``LOCATOR_SET`` + is unchanged in both respects until :issue:`679` (below). ``examples/generators/options.py``'s ``HIP_COPIES`` stayed at two for ``R1_COUNTER``'s four-octet ``counter`` where :rfc:`7401#section-5.2.3` requires eight, which this defect had masked, until :issue:`672` widened ``counter`` and @@ -1803,8 +1807,8 @@ Fixed the plain ``int``, which nothing observes since each passes ``flags=None``. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded ``DATA``, ``HEADERS`` - or ``PUSH_PROMISE`` frame parses with its whole payload dropped (:issue:`668`). - (:issue:`652`, :issue:`650`). + or ``PUSH_PROMISE`` frame parses with its whole payload dropped (:issue:`668`, + below) (:issue:`652`, :issue:`650`). * MPTCP error messages carried a doubled separator, rendering as ``TCP: : [OptNo 30] 1: invalid flags combination``. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty @@ -1887,21 +1891,19 @@ Fixed conditional of this shape, ``TSOption.remainder`` in ``pcapkit/protocols/schema/internet/ipv4.py``, is correct: its ``else 0`` is intentional because ``ts_data`` consumes the whole option data area in that arm. The - new tests use - non-zero padding rather than the zeros :rfc:`9113#section-6.1` tells a sender to use, - so failing to subtract it yields a different byte string. Ships labelled breaking: - restoring a dropped payload moves the parse output of essentially every HTTP/2 - capture, and constructed frames change byte-for-byte, from malformed to correct - (:issue:`668`). + new tests use non-zero padding rather than the zeros :rfc:`9113#section-6.1` tells a + sender to use, so failing to subtract it yields a different byte string. Ships + labelled breaking: restoring a dropped payload moves the parse output of essentially + every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to + correct (:issue:`668`). * the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. ``EnumSchema.register`` was a bare ``cls.__enum__[code] = schema``, and is the schema half of the public option - registrars, so - one ``register_ipv4_option`` call replacing a built-in named the parser it displaced - and said nothing about the schema. ``EnumSchema.__init_subclass__`` reaches the same - registry without calling ``register``, so a subclass declared with a ``code=`` - keyword (the documented way to add a schema) stayed silent too; both are guarded, - and the declaration hook's branches are folded into one loop. Presence is a + registrars, so one ``register_ipv4_option`` call replacing a built-in named the parser + it displaced and said nothing about the schema. ``EnumSchema.__init_subclass__`` + reaches the same registry without calling ``register``, so a subclass declared with a + ``code=`` keyword (the documented way to add a schema) stayed silent too; both are + guarded, and the declaration hook's branches are folded into one loop. Presence is a faithful test only because ``_EnumRegistry.__missing__`` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The code-keyed parser registrars (on ``ProtocolBase``, @@ -1934,9 +1936,8 @@ Fixed ``__option_padding__`` fields: ``_TextField.__call__`` builds ``f'{length}s'`` unconditionally, so a negative became ``'-8s'`` and ``struct.calcsize`` raised an exception caught nowhere a frame loop looks. Both of :issue:`678`'s failure - families are gone; others remain: - ``ValueError: N is not a valid BlockType`` (filed as :issue:`701`) and - ``MemoryError`` from :pr:`593`'s 32-bit band. Bare exceptions + families are gone; others remain: ``ValueError: N is not a valid BlockType`` (filed as + :issue:`701`) and ``MemoryError`` from :pr:`593`'s 32-bit band. Bare exceptions in ``SystemdJournalExportBlock.post_process``, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (``struct.error``; a NUL-only line now ends the entry); a binary field's declared @@ -1951,12 +1952,11 @@ Fixed ``defaultdict`` would insert ``UnknownOption`` for a code nobody registered). The condition is identity-based like the parser registrars and :pr:`681`'s guard for ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the call itself - creates is exempt. Flooring - the decryption-secrets payloads that read ``__length__`` *whole* was tried and - reverted, since ``Schema.pack`` leaves that key at ``-1`` for "unknown". Ships - labelled breaking: well-formed captures are byte-identical, but any truncated - PCAP-NG now yields the frames before the cut where it raised, the last of which may - carry zero-padded octets, and some short-file depths move + creates is exempt. Flooring the decryption-secrets payloads that read ``__length__`` + *whole* was tried and reverted, since ``Schema.pack`` leaves that key at ``-1`` for + "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any + truncated PCAP-NG now yields the frames before the cut where it raised, the last of + which may carry zero-padded octets, and some short-file depths move from ``ProtocolError: unknown byteorder magic`` to ``StreamEOFError``. The :pr:`683` drift :issue:`685` tracks is unchanged (:issue:`678`). * HIP's ``R1_COUNTER``/``R1_Counter`` parameter packed 12 octets where @@ -2064,13 +2064,12 @@ Fixed consult and call it only once the value resolves; a genuine miss gets ``EnumField._unregistered_member``, a real member of the same registry built through its storage base's ``__new__``, so ``isinstance`` holds and the lookup table stops - growing. Riding along: - PCAP-NG option namespaces stop cross-contaminating: once a code was minted under - ``opt``, ``OptionType.get`` merged ``opt`` over the requested namespace and most - other namespaces answered ``opt_unknown`` for it; each now gets its own - ``_unknown``. **Breaking** because :issue:`575` asked for byte-identical output - across the sample captures, as :pr:`420` and :pr:`427` did, and this deliberately - misses that bar: some sample captures change, entirely from the new + growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code + was minted under ``opt``, ``OptionType.get`` merged ``opt`` over the requested + namespace and most other namespaces answered ``opt_unknown`` for it; each now gets its + own ``_unknown``. **Breaking** because :issue:`575` asked for byte-identical + output across the sample captures, as :pr:`420` and :pr:`427` did, and this + deliberately misses that bar: some sample captures change, entirely from the new ```` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, ``_unregistered_member`` installs a ``__reduce_ex__`` rebuilding an equivalent @@ -2103,7 +2102,7 @@ Fixed exposed to the fall-through arm, where the direct ``httpv1`` binding left it ``Raw`` regardless; a caller depending on that is the one who would notice (:issue:`682`). * **a breaking change to** more bespoke registries: step 2 of :issue:`860`, - ahead of ``AppType`` (:pr:`874`, below), bringing ``StatusCode``, ``ReturnCode``, + ahead of ``AppType`` (:pr:`874`, above), bringing ``StatusCode``, ``ReturnCode``, ``ResponseKind``, ``GroupingInformation``, ``OptionType``, ``FEATCode``, ``Command`` and ``Method`` onto ``EnumRegistry`` and applying :issue:`775`'s mint/unmint ruling to their ``get()`` as well as ``_missing_``. ``StatusCode``, ``ReturnCode``, @@ -2169,8 +2168,7 @@ Fixed duplicate guards now say ``[if_rxspeed]`` rather than ``[if_txspeed]``, and ``_read_option_epb_queue``'s length guard says ``[epb_queue]`` rather than ``[epb_packetid]``. Only message text changes; the exception and warning types are - unchanged - (:issue:`1038`). + unchanged (:issue:`1038`). * ``httpv1.HTTP`` refused every HTTP/1 message with no header fields, which [:rfc:`9112#section-2.1`] allows (``*( field-line CRLF )``): ``b'GET / HTTP/1.0\r\n\r\n'`` raised ``ProtocolError: HTTP: invalid format``, @@ -2235,10 +2233,10 @@ Fixed ``UnsupportedCall`` (:issue:`1126`). ``HTTP.from_data`` always raised ``invalid HTTP version: 0``, looking up a ``version`` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's ``_make_data`` also - returned the payload - ``length``, which ``ProtocolBase`` takes as the parse length, so even a direct call - failed as shorter than the 9-octet frame header; ``make`` now computes it, and - every HTTP frame in the sample captures rebuilds byte for byte (:issue:`1154`). + returned the payload ``length``, which ``ProtocolBase`` takes as the parse length, so + even a direct call failed as shorter than the 9-octet frame header; ``make`` now + computes it, and every HTTP frame in the sample captures rebuilds byte for byte + (:issue:`1154`). * Quick-Start options did not round-trip. The IPv4, HOPOPT and IPv6-Opts readers took the function from ``schema.func``, which only a parsed schema carries, so a built option raised ``AttributeError``; HOPOPT and IPv6-Opts also sized the 8-octet option @@ -2285,8 +2283,7 @@ Fixed the end -- zero octets after a trailing ``EOOL``, otherwise ``NOP`` options and an ``EOOL`` -- so a built header can be shorter than before; every TCP segment of the measured sample captures rebuilds its header octets, and ``options-*.pcap`` - regenerate unchanged - (:issue:`1151`). + regenerate unchanged (:issue:`1151`). * **a breaking change to** ``from_data`` on a capture truncated by snaplen: ``IPv4``, ``IPv6`` and ``UDP`` rebuilt their length field from the payload at hand, writing the recorded length, so ``big_endian.pcap`` frame 3 came back as ``45000052`` where @@ -2322,8 +2319,8 @@ Fixed chain, so a packet carrying a Routing header rebuilt 48 octets as 40. The chain is now kept in wire order under a hidden ``__exthdr__`` key, absent from ``to_dict`` and dumped output, and re-emitted ahead of the upper layer. Sample frames that - rebuilt wrong now rebuild exactly, and those that rebuilt with wrong bytes raised - ``AttributeError`` instead (:issue:`1177`). Those were older defects, now fixed: + rebuilt wrong now rebuild exactly, though some that had rebuilt with wrong bytes + raised ``AttributeError`` instead (:issue:`1177`), from older defects fixed with it: ``HIP.alias`` read ``_info`` before ``read`` had set it, which also broke a standalone HIP parse and ``HIP.from_data`` (:issue:`1184`), and the MPL option's maker called ``bit_length()`` on the ``IPv6Address`` seed of an S=0 option. MPL in @@ -2471,6 +2468,9 @@ Fixed Mobility Header models, the HOPOPT and IPv6-Opts ``SMFDPDOption`` and ``QuickStartOption``, PCAP-NG's ``PacketBlock`` and ``PACK_*`` options and HTTP/2's ``Settings`` reach their packages and the aggregators (:issue:`1807`, :pr:`1815`). +* the MPL option's ``seed`` in the HOPOPT and IPv6-Opts schemas is annotated + ``int | IPv6Address``, since an S=0 option holds the source address + (:issue:`1840`, :pr:`1866`). pcapkit.toolkit --------------- @@ -2548,10 +2548,9 @@ Fixed encapsulation whose DLT has a ``LinkType`` member, and ``FILTER_NAME_TO_LINKTYPE`` grows from :pr:`838`'s hand-built entries to every root filter name, pseudo-protocol roots aside, that resolved to a single ``LinkType`` member across the encapsulations - this host's Wireshark can write. - ``pcapkit/toolkit/pyshark.py`` reaches full coverage under new tests that fail - against the unfixed tree; live pyshark parsing stays unexercised, since - ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). + this host's Wireshark can write. ``pcapkit/toolkit/pyshark.py`` reaches full coverage + under new tests that fail against the unfixed tree; live pyshark parsing stays + unexercised, since ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). * ``pypcapfile``'s ``tcp_reassembly`` set ``last`` one past the segment. ``TCP_Packet.last`` is inclusive, and the other adapters set ``seqnum + len(payload) - 1``, as this one now does too (:issue:`1100`). @@ -2759,6 +2758,14 @@ Fixed * ``pcapkit.vendor.ipx`` names its ``sys.path`` temporary ``_path`` rather than ``path``, which read as a public name of the package, and the public-API test covers ``pcapkit.vendor``, which it used to skip (:issue:`1772`, :pr:`1779`). +* the generated const comments render as written: a URL reference followed by another + bracket no longer autolinks into it, a leading enumerator such as ``(S)`` is + escaped, and a hyphenated word is no longer split across lines (:issue:`1846`, + :issue:`1847`, :pr:`1871`). +* without the ``vendor`` extra, ``pcapkit-vendor`` and ``python -m pcapkit.vendor`` + exit 1 with one line naming the missing packages and ``pypcapkit[vendor]``, where + they died with a traceback, and ``--help`` and ``--version`` still work. Importing a + crawler then raises ``ModuleNotFoundError`` (:issue:`1850`, :pr:`1875`). Project infrastructure ---------------------- @@ -2791,16 +2798,15 @@ Added canonical BSD-3-Clause (:pr:`615`). * ``examples/generators/endian.py``, and the byte-order tests that read what it writes. There was no big-endian ``.pcap`` in the repository, which is why - :issue:`605` survived its own code review: the fix to a misspelt key left the + :issue:`605` survived its own code review: the fix to a misspelled key left the corrected path as untested as the broken one. The generator writes ``big_endian.pcap`` (magic ``a1 b2 c3 d4``), ``big_endian_nanosecond.pcap`` (``a1 b2 3c 4d``, the first fixture to take that branch of the magic-number table) and ``little_endian.pcap`` (``d4 c3 b2 a1``), carrying the *same records* in each, - so tests can assert - that byte order makes no difference to what is read rather than only matching - numbers written in a test. Frame 3 is captured short (1200 octets on the wire, - 96-octet ``snaplen``), so ``incl_len`` and ``orig_len`` differ and cannot both be - satisfied by one byte-swapped value. ``test_frame_endian_runtime.py`` drives each + so tests can assert that byte order makes no difference to what is read rather than + only matching numbers written in a test. Frame 3 is captured short (1200 octets on the + wire, 96-octet ``snaplen``), so ``incl_len`` and ``orig_len`` differ and cannot both + be satisfied by one byte-swapped value. ``test_frame_endian_runtime.py`` drives each file through ``extract()`` and derives expectations by walking its record chain with ``struct``; a unit-tier case in ``test_header_frame_unit.py`` builds a big-endian capture in memory, so the fixture-free selection CI runs on @@ -2808,6 +2814,10 @@ Added fixture-backed ones by that ``ValueError``, the in-memory one by ``AssertionError: 3106905 != 1500000000``) while the little-endian twin passes on both, showing the records themselves are not the variable (:issue:`605`). +* local type stubs under ``typings/`` for dpkt, pyshark, ``pcap`` and pypcapfile, + declaring what pcapkit uses of each, so mypy checks the engines against them. + ``mypy.ini`` reads them through ``mypy_path``, and neither the sdist nor the wheel + ships them (:issue:`1878`, :pr:`1882`). Changed ~~~~~~~ @@ -2835,7 +2845,7 @@ Changed that ``examples/benchmark/benchmark.py`` names is the benchmark suite's own (:pr:`619`). * ``CODE_OF_CONDUCT.md`` moves from Contributor Covenant 1.4 to 3.0, at the - maintainer's request. The text is the canonical 3.0 Markdown fetched from + owner's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than transcribed. Some things needed deciding. 3.0's ``[NOTE`` placeholders are filled or removed: the reporting channel names @@ -2846,12 +2856,11 @@ Changed the singular maintainer, this repository having no moderators. The enforcement ladder (Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a - lever one - person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so - the attribution names version 3.0, links the permanent ``version/3/0/`` URL, carries - the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the - share-alike term covers this document only (the code remains BSD-3-Clause, ``LICENSE`` - untouched). The ladder renders as nested list items, checked against + lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence + notice, so the attribution names version 3.0, links the permanent ``version/3/0/`` + URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and + says the share-alike term covers this document only (the code remains BSD-3-Clause, + ``LICENSE`` untouched). The ladder renders as nested list items, checked against GitHub's Markdown API; :pr:`613` had to repair the 1.4 file when a stray list marker collapsed it into one nested item (:pr:`624`). * reconciled which private (``_xxx``) attributes and methods the Sphinx build @@ -2881,8 +2890,7 @@ Changed ``packet -> NIL`` for the Enhanced Packet Blocks of ``dhcp.pcapng`` long after :pr:`683` gave those blocks their captured octets back. Rather than regenerate it again and leave the drift free to recur, these files are removed from the index and - folded - into ``examples/captures/``'s blanket ``.gitignore`` rule; the genuine inputs + folded into ``examples/captures/``'s blanket ``.gitignore`` rule; the genuine inputs ``in.pcap`` and ``dhcp.pcapng`` stay tracked. ``examples/legacy_smoke/Makefile`` and its ``README.md`` now describe regenerating these reports via ``make fixtures``, and ``tests/project/test_capture_tracking.py`` pins the invariant (:issue:`685`). @@ -2910,7 +2918,7 @@ Changed whole-package, and ``lint.yml`` runs no pytest job for ``pytest_jobs()`` to see, so the fix is a ``DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']`` entry on ``test``, ``engine-tests`` and ``gate``. ``dependency_gate_gaps()`` gains - the new ``(flag, job)`` pairs, each expected; (:issue:`779`). + the new ``(flag, job)`` pairs, each expected (:issue:`779`). * ``engine='pyshark'`` had no CI job driving it through a real ``tshark``-parsed capture. ``pypcap-parity`` installs ``pyshark`` but its ``apt-get install`` named no ``tshark`` package, so ``PyShark.unsupported_reason()`` declined the engine, the @@ -2955,13 +2963,11 @@ Changed * the prose the post-:issue:`719` defect wave added or touched, checked against the code; no code changes. ``MesgIDOption.seconds`` in the Mobility Header schema said seconds since 1970 where it holds an :rfc:`1305` NTP count from 1900; - ``HTTP._guess_version`` - comments predated the HTTP/2 gate; and the + ``HTTP._guess_version`` comments predated the HTTP/2 gate; and the ``_parse_next_layer`` note miscounted the captured octets. ``Extractor._open_output`` now says only the PCAP engine passes header fields, ``Extractor.run`` and the PCAP-NG engine's ``run`` list the ``FormatError`` for PCAP output from a PCAP-NG - input, and - the extraction and engine pages document the PCAP-output path (:pr:`1179`, + input, and the extraction and engine pages document the PCAP-output path (:pr:`1179`, :pr:`1180`, :pr:`1186`, :issue:`1103`). * **a breaking change to** the ``all`` extra: it carries the core addons only -- ``cli``, ``crypto`` and ``NGAP``. The third-party engines 1.4.x bundled in it @@ -2972,7 +2978,10 @@ Changed carries what CI needs (:issue:`910`, :pr:`914`). * the ``Scapy`` and ``dev`` extras require ``scapy>=2.6.0`` (:issue:`1572`, :pr:`1578`), and the ``docs`` extra ``Sphinx>=7.3``, the first release with - ``versionremoved`` (:pr:`1743`). + ``versionremoved`` (:pr:`1743`), and ``sphinx-autodoc-typehints==3.15.0`` + (:issue:`1856`, :pr:`1877`). The ``test`` extra gains ``psutil``, + ``markdown-it-py`` and ``lxml``, which the scripts under ``util/`` use + (:issue:`1894`, :pr:`1896`). * the READMEs of ``examples/benchmark`` and ``examples/legacy_smoke`` are Markdown, ``README.md`` rather than ``README.rst``, and ``MANIFEST.in`` ships them through ``recursive-include examples *.md``. Stale build leftovers go: ``setup.py``'s @@ -3032,23 +3041,20 @@ Fixed day 1.5.0b4 was. The date is UTC, because seven of the thirty most recent bumps were made late evening US-Eastern, where a local date is a day behind the publish; the bump triggers ``create-release.yml``, the median gap to the PyPI upload is three minutes, - and - the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment + and the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment header, key ordering and each field's quoting survive (``cff-version`` and a ``references`` entry's own ``version`` are anchored out at column zero), and the - result - is checked with ``cffconvert --validate``. An absent file is reported on stderr and - skipped rather than failing the vendor cron before its ``git commit``, which would + result is checked with ``cffconvert --validate``. An absent file is reported on stderr + and skipped rather than failing the vendor cron before its ``git commit``, which would discard a whole registry crawl for a documentation file; a file with no ``version`` field raises before anything is written, since reporting success while rewriting nothing is the staleness being fixed. The script gains a ``main()`` guard (it ran the whole bump at import, hence no testable surface), and the ``import pcapkit`` - fallback in - its version reader, which returned ``"1.5.0b4'\n"`` (closing quote and newline, which - ``packaging`` rejects), is fixed; it never worked and went unnoticed because the only - caller installs the package first. A new gate asserts the committed file still names - the packaged version, covering hand-made version changes that never run this script - (:pr:`625`). + fallback in its version reader, which returned ``"1.5.0b4'\n"`` (closing quote and + newline, which ``packaging`` rejects), is fixed; it never worked and went unnoticed + because the only caller installs the package first. A new gate asserts the committed + file still names the packaged version, covering hand-made version changes that never + run this script (:pr:`625`). * ``LICENSE``'s copyright notice began the term at 2018, a year after the work it covers. The repository's first commit is ``c57f7d0b7``, dated 2017-11-07, so the notice understated the term and contradicted ``docs/source/conf.py``, whose Sphinx @@ -3074,23 +3080,21 @@ Fixed that :pr:`615` repaired from ``DAMAGE.s`` is still clean (:pr:`630`). * ``CITATION.cff`` shipped in no source distribution. ``MANIFEST.in`` has an ``include`` each for ``README.md``, ``LICENSE`` and ``CHANGELOG.md`` but none for the citation - file, - and its ``global-include`` patterns (``*.rst``, ``*.py``) cannot match a + file, and its ``global-include`` patterns (``*.rst``, ``*.py``) cannot match a ``.cff``. Removing those ``include`` lines and rebuilding shows ``README.md`` and - ``LICENSE`` - shipping anyway (setuptools adds the latter from ``license_files``, recorded as - ``License-File: LICENSE`` in ``PKG-INFO``) while ``CHANGELOG.md`` vanishes, so only - ``CHANGELOG.md`` is load-bearing, and a citation file, which no packaging default - covers, is in the same position. The gap is invisible from the web UI, because GitHub - renders "Cite this repository" from the repository; citation managers, Zenodo and - dependency inventories read the published artifact, the surface that was missing it. - :pr:`615` flagged the omission and :pr:`619` did not address it. It matters because - :pr:`625` taught ``util/bump_version.py`` to keep the file's ``version`` and - ``date-released`` in step, and a release exercising that path would publish an sdist - omitting the artefact under test. It also lets ``RepositoryCitationTests`` in - ``tests/project/test_bump_version.py`` (the gate :pr:`625` added for hand-authored - bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is - not + ``LICENSE`` shipping anyway (setuptools adds the latter from ``license_files``, + recorded as ``License-File: LICENSE`` in ``PKG-INFO``) while ``CHANGELOG.md`` + vanishes, so only ``CHANGELOG.md`` is load-bearing, and a citation file, which no + packaging default covers, is in the same position. The gap is invisible from the web + UI, because GitHub renders "Cite this repository" from the repository; citation + managers, Zenodo and dependency inventories read the published artifact, the surface + that was missing it. :pr:`615` flagged the omission and :pr:`619` did not address it. + It matters because :pr:`625` taught ``util/bump_version.py`` to keep the file's + ``version`` and ``date-released`` in step, and a release exercising that path would + publish an sdist omitting the artefact under test. It also lets + ``RepositoryCitationTests`` in ``tests/project/test_bump_version.py`` (the gate + :pr:`625` added for hand-authored bumps) run against an unpacked sdist, where it + skipped itself with "CITATION.cff is not shipped in the source distribution". ``python -m build --sdist`` now differs from before only by the archive entry ``pypcapkit-1.5.0b4/CITATION.cff``, the shipped copy is byte-identical, and ``twine check --strict`` passes on both (:pr:`631`). @@ -3104,28 +3108,22 @@ Fixed MPL 2.0; the MIT and GPL texts both carried an author notice naming **2017**; and ``2018`` first appeared in the Apache notice of 2018-12-08 as a single year, current when written. The claim also contradicted the entry's own dating of the first commit - to - 2017-11-07. The history supports a stronger argument than the one written, which is - why - this is corrected rather than deleted: 2017 is the year asserted from the first - commit, - and :pr:`630` restores it. The ``BSD 3-Clause License`` title line is reattributed - from - opensource.org's template to choosealicense.com's, whose first line is that title and - which carries the ``Copyright (c) [year], [fullname]`` comma this file uses. The error - came from generalising the pre-BSD era off the blobs ``bc836cfa2^:LICENSE`` and - ``1c69341dc:LICENSE``, which were the same file; enumerating every commit that - touched ``LICENSE`` caught it (:pr:`638`). + to 2017-11-07. The history supports a stronger argument than the one written, which is + why this is corrected rather than deleted: 2017 is the year asserted from the first + commit, and :pr:`630` restores it. The ``BSD 3-Clause License`` title line is + reattributed from opensource.org's template to choosealicense.com's, whose first line + is that title and which carries the ``Copyright (c) [year], [fullname]`` comma this + file uses. The error came from generalising the pre-BSD era off the blobs + ``bc836cfa2^:LICENSE`` and ``1c69341dc:LICENSE``, which were the same file; + enumerating every commit that touched ``LICENSE`` caught it (:pr:`638`). * the release workflow published to PyPI and to Anaconda with nothing a human had to approve. ``create-release.yml``'s ``pypi`` job had ``environment: release`` commented out while the ``id-token: write`` from the same upstream snippet was re-added live below it, so the block read as disabled as a unit when only its gate still was; the ``conda`` job had no ``environment:`` at all. Neither needed a tag: the workflow also fires on completion of ``Vendor Update``, so a scheduled registry crawl that bumped - the - version was enough to publish to both public indexes, a median three minutes from - bump to - upload. Every job that reaches outside the run is now gated: ``github`` on + the version was enough to publish to both public indexes, a median three minutes from + bump to upload. Every job that reaches outside the run is now gated: ``github`` on ``github-release`` (the GitHub Release and the ``v*`` tag), ``tag`` on ``conda-tag`` (the commit pushed to ``main``), ``pypi`` on ``pypi``, ``conda`` on ``anaconda``. One environment per credential rather than a shared ``release``, because approval is @@ -3142,17 +3140,16 @@ Fixed ``github-pages`` is the standing example, carrying no protection rule since 2021. The new test asserts the general rule, that no job running a publishing action or a ``git push`` may omit an ``environment:``, so a publishing job added later without a - gate - fails rather than ships (:issue:`641`). + gate fails rather than ships (:issue:`641`). * test comments in ``tests/_tiers.py``, ``tests/test_tier_guard.py`` and ``tests/integration/_helpers.py`` stated an exact count of committed captures under ``examples/captures/``, a number that drifts whenever the tracked set changes, and ``_helpers.py``'s undercounted on the day it was written. Each now describes the invariant (``_tiers.py``: "the moment somebody - commits another capture... or stops - committing one"; ``_helpers.py``: "...are fixtures -- some of them committed --..."; - ``test_tier_guard.py`` drops its ordinal). Prose-only, verified with - ``coverage.parser.PythonParser`` that no executable statement moved (:issue:`700`). + commits another capture... or stops committing one"; ``_helpers.py``: "...are fixtures + -- some of them committed --..."; ``test_tier_guard.py`` drops its ordinal). + Prose-only, verified with ``coverage.parser.PythonParser`` that no executable + statement moved (:issue:`700`). * ``test_every_tracked_name_exists_and_matches_git`` in ``tests/test_tier_guard.py`` checked neither the git index nor the filesystem despite its name: it asserted only that the tracked-capture list was non-empty and no name contained a ``/``, so a @@ -3186,7 +3183,9 @@ Fixed (:issue:`1632`, :pr:`1641`), stays green when a merge races its push (:issue:`1649`, :pr:`1669`), and keeps the release's name and notes when it uploads (:issue:`1657`, :pr:`1678`). Vendor Update commits the new modules a crawl - adds (:issue:`1685`, :pr:`1688`). + adds (:issue:`1685`, :pr:`1688`), and runs ``tests/const`` and ``tests/vendor`` on + the regenerated tree first, keeping only the modules they pass with + (:issue:`1853`, :pr:`1879`). * package ``__all__`` lists match what each package holds: several disagreed with it, and ``from pcapkit.protocols.data import *`` raised ``AttributeError`` on the unbound ``HOPOPT_QuickStartOption`` (:issue:`515`, :pr:`527`); the public @@ -3201,6 +3200,17 @@ Fixed where each was fitted into a fixed box, and their ``click`` targets are relative to the page, so they resolve under the site's ``/PyPCAPKit/`` prefix (:issue:`1103`, :pr:`1819`). +* the API reference renders what the code declares. ``Extractor._vfunc`` keeps its + annotation (:issue:`1839`, :pr:`1867`), and the HTTP/2 frame schemas' and the engine + classes' annotations resolve (:issue:`1857`, :pr:`1870`). Under the pinned + sphinx-autodoc-typehints 3.15.0 (:issue:`1856`, :pr:`1877`), ``conf.py`` renders an + ``aenum`` base as such, keeps a default as written (``byteorder=sys.byteorder`` + rather than the build host's value), gives a class the parameters of the + ``__init__`` its ``__new__`` hides and an enumeration ``(*values)``, keeps table ids + unique, and stops the source index linking a missing page (:issue:`1858` to + :issue:`1863`, :pr:`1881`). The generated const tables carry no version + directives; each note is on the module's hand-written page instead (:issue:`1884`, + :pr:`1895`). * ``pcapkit-cli``'s help strings say what each option does: ``-o`` describes the output rather than the input, an extension is appended only under ``--auto-extension``, and each format flag names the flags that take precedence @@ -3211,4 +3221,9 @@ Fixed package is built on an Intel runner, where converting the arm64 build shipped arm64 binaries (:issue:`1822`, :pr:`1825`); and on Intel macOS it pins ``cryptography`` 48.0.1, which has a wheel for that platform where the newer pin has none (:issue:`1827`, - :pr:`1828`). + :pr:`1828`). ``util/conda_vendor.py --check`` reports a package pinned differently + under one marker (:issue:`1852`, :pr:`1865`); the platform pins move to + ``conda/platform-pins.toml``, each with a review-by date, and Conda Update moves + them up (:issue:`1851`, :pr:`1885`); and the release's conda leg installs none of + pcapkit's extras and smoke-tests the package before uploading it (:issue:`1854`, + :issue:`1855`, :pr:`1868`).