From 214423212d7953a7d412858108b22c7a1d37ba1b Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Thu, 15 Oct 2026 13:00:00 -0400 Subject: [PATCH] docs(changelog): second pass over the 1.5.0 changelog (#1103) - add 24 entries for user-visible PRs merged before #1196 that nothing cited: corekit, protocols, utilities and vendor fixes, NGAP's generated registries, FEATCode case folding and the package __all__ repairs - cite #789 and #1020 on the entries already describing them, and merge the #604 test-only entry that appeared twice - the #759, #848 and #417 entries give the release state: the TransportProtocol base, the value-aware legacy-sink rule, pycrate in all - TLSKeyLabel's home is const.pcapng.tls_key_label, EnumRegistry's is corekit.enum, and the #604 entry names main rather than mainline - the config entry lists the pypcapfile limit and calls the moved homes prerelease names; a | between TransportProtocol members gives an int - paraphrase six quoted owner rulings; drop "until now" and "this PR" - date 1.5.0b1 to b3 in UTC, re-measure the preamble's 669 citations, and give the index's 19 bare issue numbers their roles - regenerate CHANGELOG.md and re-pin process.rst to 292 entries Refs #1103 --- CHANGELOG.md | 67 ++++-- docs/source/changelog.rst | 30 +-- docs/source/changelog/1.5.0.rst | 197 +++++++++++++----- .../contributing/conventions/process.rst | 2 +- 4 files changed, 202 insertions(+), 94 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index acd2863c5a..72566b77da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,8 @@ ## 1.5.0 -- unreleased -The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing 611 issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754). +The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing 669 issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754). -Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), `1.5.0b3` (2026-09-19) and `1.5.0b4` to `1.5.0b12` (2026-09-20 to 2026-10-13), all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` reshipped it -- an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels ([#497](https://github.com/JarryShaw/PyPCAPKit/pull/497), [#498](https://github.com/JarryShaw/PyPCAPKit/pull/498)). +Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and `1.5.0b4` to `1.5.0b12` (2026-09-20 to 2026-10-13), dated in UTC and all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` reshipped it -- an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels ([#497](https://github.com/JarryShaw/PyPCAPKit/pull/497), [#498](https://github.com/JarryShaw/PyPCAPKit/pull/498)). ### pcapkit.const @@ -13,8 +13,8 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), ` - the three `__repr__` methods [#798](https://github.com/JarryShaw/PyPCAPKit/issues/798) left on `%` (`pcapkit/const/ftp/command.py:40,119`, `pcapkit/const/http/method.py:42`) are f-strings, so the disable drops from both files and both `pcapkit/vendor/` templates. The other bespoke templates (`{const,vendor}/ftp/return_code.py`, `{const,vendor}/http/status_code.py`, `vendor/http/frame.py`) and `vendor/default.py` still carry it, so [#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)'s claim that nothing in `pcapkit/{const,vendor}/{ftp,http}/` needs it holds for this pair only; `repr()` output is unchanged ([#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)). - `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)'s (above) 53 preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a `'..._0x%s' % hex(value)[2:].upper().zfill(4)` expression, and the exact `exempt_hits == 53` bound stays. Test-only. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). - **a breaking change to** `AppType.proto`: it is the single transport of the registry the member lives in, not every transport IANA assigned the service, and `register_apptype` takes its transports as varargs ([#806](https://github.com/JarryShaw/PyPCAPKit/issues/806), [#815](https://github.com/JarryShaw/PyPCAPKit/pull/815)). Aliases show in `repr()` and `str()`, and can be registered per transport ([#807](https://github.com/JarryShaw/PyPCAPKit/issues/807), [#837](https://github.com/JarryShaw/PyPCAPKit/pull/837)). -- **a breaking change to** `TransportProtocol`: its base moves from `aenum.IntFlag` to `aenum.IntEnum`, so members no longer compose with `|` ([#808](https://github.com/JarryShaw/PyPCAPKit/issues/808), [#836](https://github.com/JarryShaw/PyPCAPKit/pull/836)). -- **a breaking change to** `TLSKeyLabel`: the hand-written class in `pcapkit.protocols.misc.pcapng` becomes the generated `pcapkit.const.pcapng.TLSKeyLabel`, crawled from the IANA registry [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) creates ([#886](https://github.com/JarryShaw/PyPCAPKit/issues/886), [#890](https://github.com/JarryShaw/PyPCAPKit/pull/890)), and the old path is removed ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). +- **a breaking change to** `TransportProtocol`: its base moves from `aenum.IntFlag` to `aenum.IntEnum`, so `|` between two members gives a plain `int` rather than a composite member ([#808](https://github.com/JarryShaw/PyPCAPKit/issues/808), [#836](https://github.com/JarryShaw/PyPCAPKit/pull/836)). +- **a breaking change to** `TLSKeyLabel`: the hand-written class in `pcapkit.protocols.misc.pcapng` becomes the generated `pcapkit.const.pcapng.tls_key_label.TLSKeyLabel` (`PCAPNG_TLSKeyLabel` in `pcapkit.const.pcapng`), crawled from the IANA registry [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) creates ([#886](https://github.com/JarryShaw/PyPCAPKit/issues/886), [#890](https://github.com/JarryShaw/PyPCAPKit/pull/890)), and the old path is removed ([#1744](https://github.com/JarryShaw/PyPCAPKit/issues/1744), [#1754](https://github.com/JarryShaw/PyPCAPKit/pull/1754)). - **a breaking change to** `Method.get`: the method token is case-sensitive, per [RFC 9110 Section 9.1](https://datatracker.ietf.org/doc/html/rfc9110#section-9.1), so `Method.get('get')` no longer resolves to `GET` ([#896](https://github.com/JarryShaw/PyPCAPKit/issues/896), [#907](https://github.com/JarryShaw/PyPCAPKit/pull/907)). It also looks a key up by value, so `BASELINE-CONTROL` and `VERSION-CONTROL` resolve ([#908](https://github.com/JarryShaw/PyPCAPKit/issues/908), [#915](https://github.com/JarryShaw/PyPCAPKit/pull/915)). - **a breaking change to** `ExtensionHeader`: it is generated from IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers registry's flag, and the two disagree on 147 (`BIT-EMU`) ([#925](https://github.com/JarryShaw/PyPCAPKit/issues/925), [#926](https://github.com/JarryShaw/PyPCAPKit/pull/926)). - the TCP and UDP `AppType` registries load on first use rather than at import ([#1538](https://github.com/JarryShaw/PyPCAPKit/issues/1538), [#1560](https://github.com/JarryShaw/PyPCAPKit/pull/1560)). @@ -27,19 +27,21 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), ` - `_missing_` in the four Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). All four now end in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and most const modules ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s two new ones included) already did, so `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. Minting through `extend_enum` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a member per bit pattern. The range guard is untouched ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). - six of the 123 constant registries rejected an invalid value differently from the built-in `enum`, and three did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. All six now raise a bare `ValueError`, as `enum.IntEnum` does and 113 of the 117 modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those 113 with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in all 113 generated `get()` bodies (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register. The three flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed; `Flags` accepts `0 <= value <= 0xFFFF`. A new `tests/const/test_const_enum_builtin_parity.py` asserts that every one of the 123 registries raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how three of the six escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). - **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still mints. `Transport._make_port` **passes an unvalidated** `int`, **so** `TCP.make(srcport=99999)` **now raises where it minted junk.** Separately, the crawler rendered one `_missing_` branch per registry row, so the two spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the 762 naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). -- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` is an `aenum.IntFlag` and a member carries the whole set IANA assigned the service, so `tcp | udp` is an ordinary value to pass back in. Over every multi-transport member this misrouted two: port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). +- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` was still an `aenum.IntFlag` and a member's `proto` carried the whole set IANA assigned the service, so `tcp | udp` was an ordinary value to pass back in; the `AppType.proto` and `TransportProtocol` entries above record how [#806](https://github.com/JarryShaw/PyPCAPKit/issues/806) and [#808](https://github.com/JarryShaw/PyPCAPKit/issues/808) later retired both. Over every multi-transport member this misrouted two: port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). - `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the two shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. All 105 generated registries are updated; the non-minting `get()` miss applies to all 105, but the `_missing_` fix changes behaviour only for the **21** with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; four reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a two-entry `FILTER_NAME_TO_LINKTYPE` (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. The other minting call sites in `_missing_`, `get` and the hand-written `mh.py` and `ngap.py` were deferred; the 106 `register`/`register_alias` sites are not deferred but the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). - **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: three of its five archived range branches were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 values that** `_missing_` **resolves (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). -- `LinkType(209).name` returned `'IPMB_LINUX'`, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current `I2C_LINUX = 209` and `aenum` gives a value to the member defined first. `pcapkit/vendor/reg/linktype.py` now sinks any row whose note mentions "legacy" after every current row, so the current name is defined first; today that affects value 209 alone. `IPMB_LINUX` stays a reachable alias (`LinkType['IPMB_LINUX']` still resolves to 209). Not breaking: only the canonical name for the shared value moves ([#848](https://github.com/JarryShaw/PyPCAPKit/pull/848)). -- **a breaking change to** 82 more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) -- *"a final concrete assigned name -> mint; a notation for the readers -> unmint"*) across the 89 `_missing_` files still calling `extend_enum`. 80 registries convert wholly (164 branches), each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Two are mixed: `EtherType`'s `DEC Unassigned` (3 branches) and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." (1) convert, while 46 vendor-company names across 50 range blocks (Xyplex, Motorola, Walker Richer & Quinn and 43 more) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. 7 of the 89 are left alone: `CGAType`'s `Tag_` mint is not an IANA-style range, and the other 6 files sit on 9 classes not yet on `EnumRegistry` (`AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method`, `OptionType`), left to [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859)/[#860](https://github.com/JarryShaw/PyPCAPKit/issues/860). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. The worked examples in `docs/source/contributing/conventions/mint-criterion.rst` keep `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). +- `LinkType(209).name` returned `'IPMB_LINUX'`, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current `I2C_LINUX = 209` and `aenum` gives a value to the member defined first. `pcapkit/vendor/reg/linktype.py` sinks a row whose note mentions "legacy" after every current row, so the current name is defined first; in the registry crawled for this release that affects value 209 alone. `IPMB_LINUX` stays a reachable alias (`LinkType['IPMB_LINUX']` still resolves to 209). Not breaking: only the canonical name for the shared value moves ([#848](https://github.com/JarryShaw/PyPCAPKit/pull/848)). The rule sinks only a row whose value another row also holds, counting values inside ranges, so a current row that merely mentions a legacy name stays first ([#852](https://github.com/JarryShaw/PyPCAPKit/issues/852), [#854](https://github.com/JarryShaw/PyPCAPKit/pull/854)). +- **a breaking change to** 82 more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): a final, concrete assigned name mints, and a notation for readers does not) across the 89 `_missing_` files still calling `extend_enum`. 80 registries convert wholly (164 branches), each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Two are mixed: `EtherType`'s `DEC Unassigned` (3 branches) and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." (1) convert, while 46 vendor-company names across 50 range blocks (Xyplex, Motorola, Walker Richer & Quinn and 43 more) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. 7 of the 89 are left alone: `CGAType`'s `Tag_` mint is not an IANA-style range, and the other 6 files sit on 9 classes not yet on `EnumRegistry` (`AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method`, `OptionType`), left to [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859)/[#860](https://github.com/JarryShaw/PyPCAPKit/issues/860). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. The worked examples in `docs/source/contributing/conventions/mint-criterion.rst` keep `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). - **a breaking change to** `EtherType`'s `_missing_` ordering: values `0x0101`-`0x01FF` now resolve to their Old Xerox Experimental name instead of an `IEEE802_3_Length_Field_0x0101`-style one. The generated method tested `0x0000 <= value <= 0x05DC` (IEEE802.3 Length Field) first, and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike [#841](https://github.com/JarryShaw/PyPCAPKit/issues/841)'s IPX fix, these bounds come from the live IANA CSV, so the fix is general: `pcapkit/vendor/reg/ethertype.py`'s `process()` collects range rows and a new `EtherType._insert_range` places each ahead of the first already-placed range that fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only containment among the 56 ranges. Closes [#862](https://github.com/JarryShaw/PyPCAPKit/issues/862) ([#865](https://github.com/JarryShaw/PyPCAPKit/pull/865)). - **a breaking change to** `Method`: `__new__` called `str.__new__(cls)` with no argument, so all 40 registered members' `str` payload was empty regardless of value -- `str(Method.GET) == ''` and `Method.GET == 'GET'` was `False`; `.value` was always correct. [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) (above) had fixed the *unregistered* path only, leaving `str(Method('frob')) == 'frob'` beside `str(Method.GET) == ''`. Fixed to `str.__new__(cls, value)`, mirroring `Command.__new__`, in the generator (`pcapkit/vendor/http/method.py`) and the generated module. No sibling hand-rolled `__new__` has the defect (`OptionType`/`AppType` deliberately store a display string). Consequences, all corrections: `Method.GET == 'GET'` is `True`; `bool(Method.GET)` flips `False` to `True`; `json.dumps(Method.GET)` emits `"GET"` not `""`; `sorted()` is lexicographic rather than input-order-stable; and `{m: ... for m in Method}` gains 40 entries where it had 1, because all 40 members previously hashed as `''` and compared equal. Nothing under `pcapkit/protocols` relied on the old behaviour. Closes [#870](https://github.com/JarryShaw/PyPCAPKit/issues/870) ([#871](https://github.com/JarryShaw/PyPCAPKit/pull/871)). -- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 2 of 2 (the 8 non-`AppType` bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its four transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s 766 range branches and `get()`'s second mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that *"only IANA registered ones are legit values... get will not have sufficient information to create new ones"*. All 766+1 branches convert, including 8 naming a real service assigned to a whole port span (7 distinct names: `x11` twice, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that it is "an obsoleted path from a breaking change" ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). +- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 2 of 2 (the 8 non-`AppType` bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its four transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s 766 range branches and `get()`'s second mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that only IANA-registered values are legitimate and that `get()` lacks the information to create new ones. All 766+1 branches convert, including 8 naming a real service assigned to a whole port span (7 distinct names: `x11` twice, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). - closes the remaining scope of [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): `EtherType` (52 branches) and `Socket` (1), the last registries whose `_missing_` minted a permanent member, now return a non-registering `cls._unregistered_member(value, name)`. The issue's AST census of `extend_enum` in `_missing_`/`get` goes from 54 sites to the single `CGAType` carve-out (its mint is not an IANA-style range), from the original 1,169. `EtherType(0x0888)` used to grow `__members__` from 160 to 161 and return the identical object on a repeat; it now returns `` with `__members__` unchanged, absent from `_value2member_map_`, equal but not identical across lookups -- likewise `Socket(0x0010)`. The regenerated `EtherType` was not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape `test_const_enum_no_mint.py`'s `is_manufactured()` sweep forbids; rather than weaken the predicate, a narrow exemption covers exactly these two files, since the collision the check guards against cannot occur once `_unregistered_member` never registers ([#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)). - `OptionType.register()` and `Command.register()` built members that `get()` could not find again, and `OptionType.register_alias()` raised `not a registered OptionType` for every argument. Both registries now round-trip through `get()`, in the generated files and the vendor templates ([#1311](https://github.com/JarryShaw/PyPCAPKit/pull/1311), [#1330](https://github.com/JarryShaw/PyPCAPKit/issues/1330), [#1369](https://github.com/JarryShaw/PyPCAPKit/pull/1369)). - the Quick-Start function registries rejected 9 to 15, though [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) gives the Function field four bits; the whole 4-bit range now resolves ([#1350](https://github.com/JarryShaw/PyPCAPKit/issues/1350), [#1353](https://github.com/JarryShaw/PyPCAPKit/pull/1353)). - every three-digit FTP reply and HTTP status code resolves, including those no registry row names ([#1443](https://github.com/JarryShaw/PyPCAPKit/issues/1443), [#1444](https://github.com/JarryShaw/PyPCAPKit/pull/1444)). - the `AppType` registry's dict union is defined only on interpreters whose `dict` has one ([#1694](https://github.com/JarryShaw/PyPCAPKit/issues/1694), [#1703](https://github.com/JarryShaw/PyPCAPKit/pull/1703)). +- NGAP's `ProcedureCode` and `ProtocolIE` are generated into `pcapkit.const.ngap` from `pycrate`'s compiled NGAP specification, with no network fetch. An unseen value gets a non-registering member, where every unknown key used to mint a member at the shared value `-1` and so aliased every other ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#899](https://github.com/JarryShaw/PyPCAPKit/pull/899)). +- `FEATCode.get` matches case-insensitively, as [RFC 5797 Section 2](https://datatracker.ietf.org/doc/html/rfc5797#section-2) defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold ([#903](https://github.com/JarryShaw/PyPCAPKit/issues/903), [#913](https://github.com/JarryShaw/PyPCAPKit/pull/913)). ### pcapkit.corekit @@ -73,10 +75,10 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), ` - `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but two defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Three more contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). - the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to all fifteen variable-width option and record payloads in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A second, `bounded_area`, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The five non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). - **a breaking change to** `FieldBase.length`: it called `struct.calcsize` on a template built from a negative resolved length (e.g. `'-5s'`, from a `length=` callback whose running counter had been overdrawn), raising a bare `struct.error` that ordinary caller code does not catch. The chokepoint every affected schema shares now raises `ProtocolError`, covering `application/{ftp,httpv1,httpv2,ngap}.py`, `internet/{hip,ipv6_route,mh}.py`, `link/ethernet.py`, `misc/pcapng.py` and `transport/sctp.py` with no other file changed; a non-negative length is unaffected. This closes the follow-on the `httpv2` length-guard entry above filed as out of scope ([#805](https://github.com/JarryShaw/PyPCAPKit/issues/805)). -- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enums.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it converts the first six registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and four `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those six: the other 105 const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships two contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). The six also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the five `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other 105 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). +- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it converts the first six registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and four `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those six: the other 105 const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships two contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). The six also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the five `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other 105 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). - **a breaking change to** `EnumRegistry.register()`, extended: the 105 const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The 10 modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype`) are left alone, so **111 of the 127 classes** inherit `EnumRegistry`, up from 6 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the maintainer's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). - `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). -- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but both `cls(default)` call sites reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling (*"Only register can mint. get should not mint unless it falls through the _missing_'s minted ranges"*), both sites now resolve `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). +- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but both `cls(default)` call sites reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, both sites now resolve `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). - `ProtoChain.__add__` returned the left operand's cached `protocols` and `aliases`: once `a.protocols` had been read, `a + ProtoChain(IPv4)` reported `chain == 'Ethernet:IPv4'` but `protocols == (Ethernet,)`. `copy.copy(self)` carried the cache across; the result is now built fresh, as `from_list` builds one, and `+=` follows, since it falls back to `__add__` ([#1099](https://github.com/JarryShaw/PyPCAPKit/issues/1099)). - a `SchemaField` with no declared length handed its `-1` placeholder to the nested schema as that schema's length, so every item of such a list warned `packet length < 0`: 12 `SchemaWarning` for a two-locator HIP `LOCATOR_SET`, 6 for one MH CGA Parameters item. `SchemaField.unpack` now passes what is left in the buffer as a declared length -- not `None`, which would turn a truncated item's `FieldValueError` into `prepare()`'s quiet `StreamEOFError`. Fixed-length items are unchanged; the HTTP/2 `SETTINGS` pair, whose size is fixed, is given it in its own schema instead ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140), below) ([#1148](https://github.com/JarryShaw/PyPCAPKit/issues/1148)). - a signed `NumberField` decoded as unsigned, a zero-length `PayloadField` misread, and an empty `Raw` did not rebuild ([#1331](https://github.com/JarryShaw/PyPCAPKit/pull/1331)). @@ -87,6 +89,11 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), ` - fields keep the bits and octets they captured on rebuild: a `BitField`'s unnamed bits ([#1487](https://github.com/JarryShaw/PyPCAPKit/issues/1487), [#1495](https://github.com/JarryShaw/PyPCAPKit/pull/1495)), a `NumberField`'s bits outside `bit_length` ([#1488](https://github.com/JarryShaw/PyPCAPKit/issues/1488), [#1497](https://github.com/JarryShaw/PyPCAPKit/pull/1497)), and the bytes an untyped `ListField` unpacked ([#1489](https://github.com/JarryShaw/PyPCAPKit/issues/1489), [#1494](https://github.com/JarryShaw/PyPCAPKit/pull/1494)). An unnamed `ListField`, `PayloadField` or `SchemaField` gets a default name ([#1491](https://github.com/JarryShaw/PyPCAPKit/issues/1491), [#1493](https://github.com/JarryShaw/PyPCAPKit/pull/1493)). - each non-final `Info` subclass is finalised on its own state ([#1490](https://github.com/JarryShaw/PyPCAPKit/issues/1490), [#1496](https://github.com/JarryShaw/PyPCAPKit/pull/1496)), and its constructor reads only the class's own annotations ([#1692](https://github.com/JarryShaw/PyPCAPKit/issues/1692), [#1702](https://github.com/JarryShaw/PyPCAPKit/pull/1702)). - copying or pickling a stream proxy no longer recurses ([#1524](https://github.com/JarryShaw/PyPCAPKit/issues/1524), [#1544](https://github.com/JarryShaw/PyPCAPKit/pull/1544)). +- `@prepare` reads `length` and `packet` by keyword as well as by position, so `Schema.unpack(data)` and `Schema.unpack(data, length=2)` work where they raised `IndexError` ([#444](https://github.com/JarryShaw/PyPCAPKit/issues/444), [#450](https://github.com/JarryShaw/PyPCAPKit/pull/450)). +- `ListField` unpacks a schema item from the per-item copy its callback and `length_callback` configured, not from the unconfigured original ([#433](https://github.com/JarryShaw/PyPCAPKit/issues/433), [#453](https://github.com/JarryShaw/PyPCAPKit/pull/453)). +- an `*IntField` given a `signed=` that contradicts its fixed sign, such as `UInt8Field(signed=True)`, raises `FieldValueError`; the argument used to be discarded silently ([#545](https://github.com/JarryShaw/PyPCAPKit/issues/545), [#549](https://github.com/JarryShaw/PyPCAPKit/pull/549)). +- negative lengths raise `ProtocolError` rather than a bare stdlib error: `FieldBase.length` tells a malformed template from a negative resolved length ([#825](https://github.com/JarryShaw/PyPCAPKit/issues/825), [#827](https://github.com/JarryShaw/PyPCAPKit/pull/827)), and `NumberField` checks a negative resolved `length` and a negative `bit_length` ([#828](https://github.com/JarryShaw/PyPCAPKit/issues/828), [#829](https://github.com/JarryShaw/PyPCAPKit/pull/829), [#831](https://github.com/JarryShaw/PyPCAPKit/issues/831), [#834](https://github.com/JarryShaw/PyPCAPKit/pull/834)). +- `ModuleDescriptor.klass` raises `ProtocolError` for a class name that resolves to nothing, or one never given, rather than a bare `AttributeError`; the two registry modules share one `NULL` sentinel compared by identity, where each had its own string ([#832](https://github.com/JarryShaw/PyPCAPKit/issues/832), [#833](https://github.com/JarryShaw/PyPCAPKit/issues/833), [#835](https://github.com/JarryShaw/PyPCAPKit/pull/835)). ### pcapkit.dumpkit @@ -116,7 +123,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), ` - conflicting TCP overlaps resolve first-write-wins, per [RFC 9293](https://datatracker.ietf.org/doc/html/rfc9293) section 3.10, where they had silently resolved last-write-wins ([#443](https://github.com/JarryShaw/PyPCAPKit/issues/443), [#478](https://github.com/JarryShaw/PyPCAPKit/pull/478)). A deliberate, narrow behaviour break: a conforming retransmission carries identical bytes. IP fragment reassembly keeps last-write-wins, because [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) specifies the opposite resolution, and records the disagreement instead ([#482](https://github.com/JarryShaw/PyPCAPKit/pull/482)). - renames with no compatibility alias left behind: `HoleDiscriptor` is spelled `HoleDescriptor` and its package alias `TCP_HoleDiscriptor` is `TCP_HoleDescriptor` ([#350](https://github.com/JarryShaw/PyPCAPKit/pull/350)). - subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` ([#514](https://github.com/JarryShaw/PyPCAPKit/issues/514)). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias, which every built-in does (hence the public classes had **0** subclasses against the `*Base` classes' 9, 5, 2 and 3). **This breaks out-of-tree code that subclasses one of the four and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Two silent failures are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` (passing `name=` to a `Reassembly` subclass silently ignored the key, since `protocol=` is the real one), and so does `Dumper`'s `ext=` without `fmt=`. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass gained a class-level `registry` property mirroring `EnumSchema.registry`, and a `Dumper` subclass no longer touches the filesystem while its `class` statement runs. `Engine`'s keyword is `engine=` rather than the `name=` first shipped, because `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s parameters on Python 3.10 and raise `TypeError` before the hook runs. There is no `name=` alias: a keyword that works on some interpreters and not others is the trap being removed. -- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (all 13 derive from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its two siblings check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched. +- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (all 13 derive from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its two siblings check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched ([#1020](https://github.com/JarryShaw/PyPCAPKit/pull/1020)). - extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output ([#420](https://github.com/JarryShaw/PyPCAPKit/pull/420)). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% (IP reassembly submits a datagram for every frame, fragmented or not) ([#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow dumper handed each record to a `Frame` constructor that re-dissected the whole stack to return the bytes it had just been given; options are no longer parsed twice either ([#427](https://github.com/JarryShaw/PyPCAPKit/pull/427)). - **a breaking change to** `Extractor.engine` and `Extractor.record_header`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. The built-in engines subclass `EngineBase` directly, so none was ever an `Engine`, which adds only the `__init_subclass__` registration hook. Two things are visible to a type checker. Code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). - the documentation of the engine base classes: the member stubs (`name`, `module`, `registry`, `extractor`, `unsupported_reason`, `run`, `read_frame`, `close` and the rest) moved from `Engine` to `EngineBase`, where they are defined, and `Engine` now documents only the `__init_subclass__` registration hook it adds, so the `EngineBase[_P]` that `Extractor.engine` returns leads to an API. Cross-references to `Engine.run`, `Engine.read_frame`, `Engine.close` and `Engine.unsupported_reason` now point at `EngineBase`. Exporting `EngineBase` in `__all__` is left to [#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016) ([#1024](https://github.com/JarryShaw/PyPCAPKit/issues/1024)). @@ -158,7 +165,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18), ` - ESP parsing and construction [[RFC 4303](https://datatracker.ietf.org/doc/html/rfc4303)], with optional payload decryption and ICV verification through `cryptography` (`pip install pypcapkit[crypto]`) ([#378](https://github.com/JarryShaw/PyPCAPKit/pull/378)). Keys reach the dissector through a new caller-state channel, `pcapkit.corekit.context`, surfaced as the `context=` keyword on `extract()` and `Extractor`, carrying an `esp.SecurityAssociation`. Nothing here raises: with no association the SPI and sequence number are still reported and the ciphertext is left opaque with `status=NO_SA`, and a failed decryption or ICV check is recorded on the parsed result the same way. Extended Sequence Numbers, TFC padding and anti-replay are not implemented, and an unsupported cipher or MAC is refused with a clear error rather than half-processed. - SCTP as a transport protocol [[RFC 9260](https://datatracker.ietf.org/doc/html/rfc9260)]: all 13 chunk types, 8 chunk parameters and 13 error causes, with the CRC32c both recorded and verifiable (it covers the SCTP packet alone, with no IP pseudo-header). Upper layers register on the DATA chunk's Payload Protocol Identifier through `register_sctp`, not on a port ([#379](https://github.com/JarryShaw/PyPCAPKit/pull/379)). -- NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) ([#251](https://github.com/JarryShaw/PyPCAPKit/discussions/251), [#417](https://github.com/JarryShaw/PyPCAPKit/pull/417)). Decoding is generic by ASN.1 shape rather than per-procedure, so all 81 elementary procedures and 438 protocol IEs work and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` is deliberately excluded from the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module. +- NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) ([#251](https://github.com/JarryShaw/PyPCAPKit/discussions/251), [#417](https://github.com/JarryShaw/PyPCAPKit/pull/417)). Decoding is generic by ASN.1 shape rather than per-procedure, so all 81 elementary procedures and 438 protocol IEs work and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` was at first left out of the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module -- until [#910](https://github.com/JarryShaw/PyPCAPKit/issues/910) counted it among `all`'s core addons, so `all` installs it (the `all` entry under *Project infrastructure*). - the Mobility Header registry, completed ([#383](https://github.com/JarryShaw/PyPCAPKit/pull/383), [#437](https://github.com/JarryShaw/PyPCAPKit/pull/437)): the [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options, then all 24 registered message data types, all 71 registered options (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families) and all 4 CGA extensions. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the four extensions round-trip end to end. - dispatch entries for dissectors that existed but were reachable from no registry ([#436](https://github.com/JarryShaw/PyPCAPKit/pull/436)): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, `L2TPv2` on UDP 1701 and OSPF at `TransType` 89. `VLAN` became an abstract base with `C_Tag` (802.1Q) and `S_Tag` (802.1ad) as concrete subclasses, so a Q-in-Q frame no longer collapses into one opaque `Raw`; `L2TP` likewise became a base, with `L2TPv2` carrying the [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) implementation. - `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other four families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) needs; [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) turned out to need a missing class instead (115 is [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3 over IP, not [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661); see the corresponding **Fixed** entry below), so its worked example names `L2TPv3` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). @@ -207,7 +214,7 @@ Layer-limited extraction: `layer='link'` no longer sets the termination flag on - the last four call sites where a `_make_*` method converts an address itself, ahead of the schema, so the same `bool`-as-`int` mistake reached them, unguarded by the schema-layer fix above: `ARP._make_proto_resolve`, `IPv6_Route._make_data_type_rpl` (which also derived its RPL compression lengths from the laundered value, corrupting the address list), the `dst` parameter of `IPv6_Route.make`, and the latent `OSPF._make_id_numbers`. All four now go through `parse_ip_address`, the helper [#539](https://github.com/JarryShaw/PyPCAPKit/pull/539) and [#552](https://github.com/JarryShaw/PyPCAPKit/issues/552) route their own sites through, completing the programme [#481](https://github.com/JarryShaw/PyPCAPKit/pull/481) and [#508](https://github.com/JarryShaw/PyPCAPKit/issues/508) started. Pinning `version=6` on the two `IPv6_Route` sites is a second, smaller behaviour change: both converted a plain integer through the family-inferring `ipaddress.ip_address`, so `ip=[258]` packed a 4-octet IPv4 address (`0.0.1.2`) inside an IPv6-only header; it now packs the 16-octet `::102` ([#540](https://github.com/JarryShaw/PyPCAPKit/issues/540)). - PCAP and PCAP-NG output and parsing: `bytes(frame)` returned the *next* frame's octets, `files=True` wrote names like `Frame 1..json`, and a PCAP-NG `timestamp_epoch` was shifted by the reading host's timezone ([#403](https://github.com/JarryShaw/PyPCAPKit/pull/403)); seven further parser defects ([#341](https://github.com/JarryShaw/PyPCAPKit/issues/341)--[#347](https://github.com/JarryShaw/PyPCAPKit/issues/347), [#371](https://github.com/JarryShaw/PyPCAPKit/pull/371)) and, on the write path, four more block-parsing ones ([#388](https://github.com/JarryShaw/PyPCAPKit/pull/388)); `BitField` packed every named bit as set ([#359](https://github.com/JarryShaw/PyPCAPKit/issues/359), [#374](https://github.com/JarryShaw/PyPCAPKit/pull/374)); the extension-header walk failed to advance past the last IPv6 extension header ([#348](https://github.com/JarryShaw/PyPCAPKit/issues/348), [#373](https://github.com/JarryShaw/PyPCAPKit/pull/373)); and IPv6 fragment offsets went unscaled, with reassembly keyed on the flow label -- optional, and routinely zero, so distinct datagrams collapsed together -- rather than on the fragment identification ([#389](https://github.com/JarryShaw/PyPCAPKit/pull/389)). - `TCP._make_mptcp_addaddr` could not build an `ADD_ADDR` option end to end: its `kind=`/`length=` arguments were rejected with `UnknownFieldWarning` and dropped, and `.pack()` then raised `KeyError: 'length'` from `port`'s own condition, `pkt['length'] in (10, 22)`. The cause was one layer up: `MPTCP`, the base of every Multipath TCP subtype schema, declared `kind` and `length` only under `typing.TYPE_CHECKING` rather than as real fields, unlike `Option`. That dropped `kind=`/`length=` for every `_make_mptcp_*` constructor, so `MPTCP` now declares both for real ([#541](https://github.com/JarryShaw/PyPCAPKit/issues/541)). The missing fields broke parsing too: a subtype schema's leading field read the `kind` octet itself, an off-by-two in field alignment rather than a wire-format change (a correct sender's octets were always right). Spec-correct `ADD_ADDR` and `MP_PRIO` options failed with `FieldError: TCP: [OptNo 30] 3 invalid IP version` and `KeyError: 'length'`; both parse now. -- two dropped-keyword/wrong-cast defects flagged in review during this release and never filed until now: HIP's `_make_param_encrypted` passed `cipher=` to a schema with no such field, so an AES-cipher `ENCRYPTED` parameter built through `make` packed without its IV; and IPv6-Route's `RPL.post_process`, which runs on every `Schema.pack`, assumed `self.addresses` was still the concatenated `bytes` a parse leaves, and raised slicing the `list[bytes]` a `make`-built multi-address header holds ([#556](https://github.com/JarryShaw/PyPCAPKit/issues/556)). +- two dropped-keyword/wrong-cast defects flagged in review during this release and filed only as [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556): HIP's `_make_param_encrypted` passed `cipher=` to a schema with no such field, so an AES-cipher `ENCRYPTED` parameter built through `make` packed without its IV; and IPv6-Route's `RPL.post_process`, which runs on every `Schema.pack`, assumed `self.addresses` was still the concatenated `bytes` a parse leaves, and raised slicing the `list[bytes]` a `make`-built multi-address header holds ([#556](https://github.com/JarryShaw/PyPCAPKit/issues/556)). - two more defects [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) exposed by letting construction reach code that had never run. `MPTCP.subtype` was still `typing.TYPE_CHECKING`-only, so `TCP(options=[(Enum_Option.Multipath_TCP, ...)])` raised `AttributeError: ... has no attribute 'subtype'` for every subtype but `MP_JOIN`: the convenience constructor reads the schema straight back through `_read_mptcp_*` with no byte round trip, so `_MPTCP.post_process`, the only code that set `subtype`, never ran. Fixed on the construction path (`TCP._make_mode_mp`) rather than by adding a third real field the way `kind`/`length` got in [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541): `subtype` is already packed as 4 bits of each subtype's own `test` bitfield, and a second field for the same bits would double-encode them or need a "derive, don't pack" field kind this library lacks ([#566](https://github.com/JarryShaw/PyPCAPKit/issues/566)). Separately, `_make_mptcp_capable` wrote `length=20 if rkey is None else 32` where [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 gives 12 and 20, and `MPTCPCapable.rkey`'s condition (`pkt['length'] != 32`) dropped the receiver's key for exactly the length the maker used for "key present": a key-absent MP_CAPABLE could not be built, and a key-present one lost its key on the wire. Both, and the guard in `_read_mptcp_capable`, now agree on 12/20. **This changes MP_CAPABLE's packed output**: a 20-octet key-present option built or parsed under the old code becomes 12 octets with no key, or 20 octets with the key present, depending on which the caller meant ([#567](https://github.com/JarryShaw/PyPCAPKit/issues/567)). - TCP's MP_CAPABLE now parses and builds all five lengths of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.1 -- 4 (SYN, no key), 12, 20, 22 (adds the Data-Level Length) and 24 (adds the Checksum) -- where `_read_mptcp_capable` raised `ProtocolError: TCP: [OptNo 30] invalid format` for anything but 12 and 20. In the schema `skey` is now present only for `length > 4` and `rkey` for `length > 12`, and two fields are new: `dl_len` (16 bits, `length > 20`) and `checksum` (2 octets, `length > 22`). `_make_mptcp_capable` takes `dl_len` and `checksum` too, where it used to drop them silently through `**kwargs`, and picks the length from how many of `skey`, `rkey`, `dl_len` and `checksum` are not `None`. A field given without the one before it raises `ProtocolError`. The data model `MPTCPCapable` gains `dl_len` and `checksum`, and `skey` becomes `Optional[int]`: code that constructs it directly must now pass both new fields, and `to_dict()` carries two more keys ([#1042](https://github.com/JarryShaw/PyPCAPKit/issues/1042)). - IPv6-Route's RPL routing data was five octets wide at the front where [RFC 6554](https://datatracker.ietf.org/doc/html/rfc6554) section 3 gives four, with three further defects behind it. `CmprI`, `CmprE` and `Pad` are each 4 bits sharing a 32-bit word with a 20-bit `Reserved`, but the schema declared `cmpr_i` and `cmpr_e` as whole octets, so a constructed two-address header packed to 41 octets while its `Hdr Ext Len` of 5 declared 48. Correcting the width is a **wire-format change** and reshapes the schema: `RPL(cmpr_i=..., cmpr_e=...)` is now `RPL(cmpr={'cmpr_i': ..., 'cmpr_e': ...})`, beside the existing `pad={'pad_len': ...}`. Behind it, the reader's `header.length % 16` guard read `Hdr Ext Len` as an octet count and assumed 16-octet addresses, which an SRH carries only when `CmprI` and `CmprE` are both 0 -- the unit confusion [#487](https://github.com/JarryShaw/PyPCAPKit/issues/487) fixed for Source Route and Type 2, left by [#489](https://github.com/JarryShaw/PyPCAPKit/pull/489) for want of a working RPL round trip to validate against. It is replaced by section 4.2's address-count arithmetic, `n = (((Hdr Ext Len * 8) - Pad - (16 - CmprE)) / (16 - CmprI)) + 1`, which the reader now requires to be non-negative and whole. `RPL.post_process` subtracted `pad_len` a second time from a buffer whose length callback had already taken it off, losing one address per `16 - CmprI` octets of padding, and set `ip` only when it had parsed octets, so `IPv6_Route(type=..., data={'ip': [...]})` raised a bare `AttributeError: 'RPL' object has no attribute 'ip'`. And `_make_data_type_rpl` computed `Pad` as `8 - length % 8` without the outer `% 8`, so an aligned address vector got a full 8 octets of padding where section 3 requires `Pad` to be 0 when `CmprI` and `CmprE` are both 0. The narrower `cmpr_i` also removes a latent divide-by-zero (a whole octet could hold 16, making `16 - CmprI` zero). `ipv6-route-type/RPL_Source_Route_Header` round-trips and its `EXPECTED_FAILURES` entry is deleted; none of this has been checked against a real RPL capture ([#564](https://github.com/JarryShaw/PyPCAPKit/issues/564)). @@ -221,8 +228,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - as a consequence of the above, the unhandled `MemoryError` at `pcapkit/protocols/protocol.py:1411` on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack` eleven lines earlier spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). - documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, whereas a `SwitchField` always resolves to a concrete field ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- the one assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, red on `mainline` since [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) merged. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced; [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- `main` went red the moment [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)'s `ljust()` landed, because that test still pinned the head-padded short read: `Reserved_79` declaring `length=12` over 6 real octets now reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) left the file alone because [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it at the time ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621)). +- the one assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, red on `main` since [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) merged. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - the HIP `SOLUTION` builder sized the parameter with `4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)`, an invalid shorthand for two fields. [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5) gives `Length` as `4 + RHASH_len / 4` over a `Random #I` and a `Puzzle solution #J` of `RHASH_len / 8` octets **each**, and `/ 4` equals twice `/ 8` only because `RHASH_len` is a whole number of octets. For an arbitrary `bit_length()` the builder emitted an *odd* contents width, which its own reader refused: `_read_param_solution`'s `(schema.len - 4) % 2` guard exists because `SolutionParameter` splits that width into two equal halves. `random=0x1` with `solution=0xfff` declared `len=7` and raised `ProtocolError: HIPv2: [ParamNo 321] invalid format` on input the library had produced. The undersized length was worse: both fields take their width from that `len`, so `solution=0xfff` was packed into one octet and came back as `0xff`, silently. Fixed to `4 + 2 * math.ceil(max(...) / 8)`, the form the sibling `PUZZLE` builder uses, which is even by construction. Loosening the reader was rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). - `TCP.read` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. One dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). - **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading** `frame.len` **or** `frame.cap_len` **from a** `.pcap` **gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). @@ -236,7 +242,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register` was a bare `cls.__enum__[code] = schema`, and is the schema half of 14 public registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's two branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The seven code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of the three takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its 327 registry writes (one being `R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690)) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). - every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). A second route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the eight unwrapped `length - N` spans and eight `__option_padding__` fields: `_TextField.__call__` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; two others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Three bare exceptions in `SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The five unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the fifth and last registrar with no overwrite guard: `Option.register` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the seven parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the three decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and eight short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). - HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- two independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). A second defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), leaving 43 entries. `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. -- `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at four sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for all four input classes. All four now raise `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so a fifth bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787)). +- `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at four sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for all four input classes. All four now raise `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so a fifth bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787), [#789](https://github.com/JarryShaw/PyPCAPKit/pull/789)). - a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: `b'MESSAGE=hello'` plus three NULs returned `MESSAGE == 'hello\x00\x00\x00'`. The padding-only-line guard from [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) catches padding only on a line of its own, which needs the preceding field to have ended with a real newline; without one `readline()` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 octets, so at most the trailing three octets of a terminator-less line are stripped, with a `SchemaWarning`, and anything past that is left as data. The gate reads the *raw* line, since a real trailing newline, or any other raw last octet including ASCII whitespace, proves the padding is zero ([#794](https://github.com/JarryShaw/PyPCAPKit/issues/794)). `examples/generators/pcapng.py`'s `_journal_entry` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s `not line.strip(b'\x00')` guard and the mechanism ([#791](https://github.com/JarryShaw/PyPCAPKit/issues/791)). - `httpv2.HTTP.read` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack`/ `FieldBase.length` machinery shared by ten schema modules, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). - `SystemdJournalExportBlock.post_process` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`; a missing or non-`b'\n'` octet ends the entry with a `SchemaWarning` instead of resynchronising at the wrong offset. `test_journal_fields_following_a_binary_field_are_not_discarded` fails against the unfixed code with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). @@ -244,7 +250,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - **a breaking change to** four `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). All four now peek at what `.get()` would consult and mint only once the value is unresolvable; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and 6 of the other 7 namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: 14 of 23 captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves all 14 byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). - `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the first two left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). - `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and nine genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: TCP:80/8080 traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). -- **a breaking change to** 8 more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 1 of 2 (`AppType` is PR 2, [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) below), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. The first five convert 12 unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; the three with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a round-2 review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling (*"get will not have sufficient information to create new ones"*: `Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (a second, independent mint site) now build an unregistered member. `FEATCode`'s crawler now declares all 15 real FEAT-code values from the live IANA table instead of minting 10 of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are PR 2's. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s two paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). +- **a breaking change to** 8 more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 1 of 2 (`AppType` is PR 2, [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) below), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. The first five convert 12 unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; the three with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a round-2 review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (a second, independent mint site) now build an unregistered member. `FEATCode`'s crawler now declares all 15 real FEAT-code values from the live IANA table instead of minting 10 of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are PR 2's. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s two paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). - `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. All seven `Raises:` clauses already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Two observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). - `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Two more guards named the wrong option: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). - `httpv1.HTTP` refused every HTTP/1 message with no header fields, which [[RFC 9112 Section 2.1](https://datatracker.ietf.org/doc/html/rfc9112#section-2.1)] allows (`*( field-line CRLF )`): `b'GET / HTTP/1.0\r\n\r\n'` raised `ProtocolError: HTTP: invalid format`, because `_read_http_header` treated a header with no CRLF as malformed. So `HTTPv1(method='GET', uri='/', http_version='1.1')` failed to read back the bytes its own `make` built, and `HTTP._guess_version` answered `unknown HTTP version`, since `_test_start_line` mirrored the same check. A CRLF-less header is now a start line with zero fields, in the parser and the predicate alike. That check was also what refused the HTTP/2 connection preface, whose header is exactly `PRI * HTTP/2.0`; it is now refused by name instead, to the same message ([#1041](https://github.com/JarryShaw/PyPCAPKit/issues/1041)). @@ -297,6 +303,14 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - IPv4: a Total Length of 0 is read as TSO, and one inside the header as bogus ([#1547](https://github.com/JarryShaw/PyPCAPKit/issues/1547), [#1554](https://github.com/JarryShaw/PyPCAPKit/pull/1554)). - IPv6: a jumbogram's payload is dissected up to its Jumbo Payload Length ([#1434](https://github.com/JarryShaw/PyPCAPKit/issues/1434), [#1440](https://github.com/JarryShaw/PyPCAPKit/pull/1440)), a jumbogram [RFC 2675](https://datatracker.ietf.org/doc/html/rfc2675) calls an error is rejected ([#1687](https://github.com/JarryShaw/PyPCAPKit/issues/1687), [#1713](https://github.com/JarryShaw/PyPCAPKit/pull/1713)), and so is a header whose Version is not 6 ([#1680](https://github.com/JarryShaw/PyPCAPKit/issues/1680), [#1723](https://github.com/JarryShaw/PyPCAPKit/pull/1723)). - the warnings of a next layer replaced by `Raw` are dropped ([#1580](https://github.com/JarryShaw/PyPCAPKit/issues/1580), [#1588](https://github.com/JarryShaw/PyPCAPKit/pull/1588)). +- IPv6-Route's three data-type readers name the routing type in their `ProtocolError`, where each message read `[TypeNo ]` ([#442](https://github.com/JarryShaw/PyPCAPKit/issues/442)), and `HTTP(..., version=1)` parses from the frame's data rather than the drained file, where it raised `ValueError` ([#447](https://github.com/JarryShaw/PyPCAPKit/issues/447), [#451](https://github.com/JarryShaw/PyPCAPKit/pull/451)). +- `Schema.pack` accepts a `tuple` for a `ListField`, as the data models declare, so HIP and MH options that hold one round-trip ([#476](https://github.com/JarryShaw/PyPCAPKit/issues/476), [#480](https://github.com/JarryShaw/PyPCAPKit/pull/480)). +- a Mobility Header CGA extension reports its length with its 4-octet header, where every parsed one read 2 octets short ([#512](https://github.com/JarryShaw/PyPCAPKit/issues/512), [#517](https://github.com/JarryShaw/PyPCAPKit/issues/517), [#528](https://github.com/JarryShaw/PyPCAPKit/pull/528)). +- TCP's SACK option enforces the `8n + 2` length [RFC 2018](https://datatracker.ietf.org/doc/html/rfc2018) gives it, as its docstring promised ([#519](https://github.com/JarryShaw/PyPCAPKit/issues/519), [#535](https://github.com/JarryShaw/PyPCAPKit/pull/535)). +- `IPv4.from_data()` rebuilds a parsed datagram: `Schema.pack` checks a payload against `ProtocolBase`, and option padding ends in an `EOOL` option rather than a bare wire code ([#506](https://github.com/JarryShaw/PyPCAPKit/issues/506), [#536](https://github.com/JarryShaw/PyPCAPKit/pull/536)). +- IPv4's SEC option sets the termination bit its own reader requires, sizes the bitmap from the bit count, and SID is 16 bits wide ([#534](https://github.com/JarryShaw/PyPCAPKit/issues/534), [#537](https://github.com/JarryShaw/PyPCAPKit/issues/537), [#543](https://github.com/JarryShaw/PyPCAPKit/pull/543)). +- the MPTCP option header reads `length` from bit 8, per [RFC 8684 Section 3](https://datatracker.ietf.org/doc/html/rfc8684#section-3), rather than bit 1 ([#553](https://github.com/JarryShaw/PyPCAPKit/issues/553), [#558](https://github.com/JarryShaw/PyPCAPKit/pull/558)). +- four Mobility Header enumerations defined inline from their RFCs (`FastBindingAcknowledgmentStatus`, `IPv6AddressPrefixCode`, `LocalizedRoutingStatus`, `LMAAddressCode`) raise `EnumValueError` for an unassigned value rather than minting `Unassigned_N` ([#880](https://github.com/JarryShaw/PyPCAPKit/issues/880), [#889](https://github.com/JarryShaw/PyPCAPKit/pull/889)), and all four inherit `get` and its `default` from `EnumLookup` rather than overriding it ([#935](https://github.com/JarryShaw/PyPCAPKit/issues/935), [#940](https://github.com/JarryShaw/PyPCAPKit/pull/940)). ### pcapkit.toolkit @@ -315,14 +329,14 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - scapy's `packet2dict` dumped nested packets as `{}` and mutated its input ([#1287](https://github.com/JarryShaw/PyPCAPKit/pull/1287)); the dpkt reassembly adapters build their data from the record's own octets rather than dpkt's re-serialised bytes, and leave the packet alone ([#1297](https://github.com/JarryShaw/PyPCAPKit/issues/1297), [#1335](https://github.com/JarryShaw/PyPCAPKit/pull/1335)). - the TCP adapters: pyshark accepts tshark's `True`/`False` TCP flags ([#1514](https://github.com/JarryShaw/PyPCAPKit/issues/1514), [#1527](https://github.com/JarryShaw/PyPCAPKit/pull/1527)); a TCP segment whose header the snapshot cut is traced and reassembled ([#1518](https://github.com/JarryShaw/PyPCAPKit/issues/1518), [#1556](https://github.com/JarryShaw/PyPCAPKit/pull/1556)); dpkt reads SYN, FIN and RST whatever the NS bit ([#1618](https://github.com/JarryShaw/PyPCAPKit/issues/1618), [#1621](https://github.com/JarryShaw/PyPCAPKit/pull/1621)); and the scapy adapters stop at the IP datagram's declared length ([#1584](https://github.com/JarryShaw/PyPCAPKit/issues/1584), [#1589](https://github.com/JarryShaw/PyPCAPKit/pull/1589)). - timestamps: dpkt and pypcapfile hand over the correctly rounded timestamp ([#1668](https://github.com/JarryShaw/PyPCAPKit/issues/1668), [#1690](https://github.com/JarryShaw/PyPCAPKit/pull/1690)), and a copied `DecimalTimestamp` keeps `orig_len` under `_pydecimal` ([#1647](https://github.com/JarryShaw/PyPCAPKit/issues/1647), [#1653](https://github.com/JarryShaw/PyPCAPKit/pull/1653)). -- the third-party engines are best effort: the code each adapter carried only to match the default engine is reverted ([#1718](https://github.com/JarryShaw/PyPCAPKit/pull/1718), [#1721](https://github.com/JarryShaw/PyPCAPKit/pull/1721), [#1724](https://github.com/JarryShaw/PyPCAPKit/pull/1724), [#1727](https://github.com/JarryShaw/PyPCAPKit/pull/1727), closing [#1708](https://github.com/JarryShaw/PyPCAPKit/issues/1708) through [#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). What each engine leaves out, such as TCP behind a tunnel or an IPv6 extension header, is documented rather than warned about; the pypcapfile engine still warns once per capture that it leaves TCP over IPv6 out ([#1513](https://github.com/JarryShaw/PyPCAPKit/issues/1513), [#1535](https://github.com/JarryShaw/PyPCAPKit/pull/1535)). +- the third-party engines are best effort: the code each adapter carried only to match the default engine is reverted ([#1718](https://github.com/JarryShaw/PyPCAPKit/pull/1718), [#1721](https://github.com/JarryShaw/PyPCAPKit/pull/1721), [#1724](https://github.com/JarryShaw/PyPCAPKit/pull/1724), [#1727](https://github.com/JarryShaw/PyPCAPKit/pull/1727), closing [#1708](https://github.com/JarryShaw/PyPCAPKit/issues/1708) through [#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). What each engine leaves out, such as TCP behind a tunnel or an IPv6 extension header, is documented rather than warned about. The pypcapfile engine keeps its warning, once per capture, that it leaves TCP over IPv6 out ([#1513](https://github.com/JarryShaw/PyPCAPKit/issues/1513), [#1535](https://github.com/JarryShaw/PyPCAPKit/pull/1535)). ### pcapkit.utilities #### Added - `pcapkit.utilities.logging` as a real interface: `get_logger()` for per-module children, `configure()` to set level, handler, stream, format or propagation at runtime, `reset()` to return to library-neutral, and `ensure_output()`. Seventeen modules now log under their own `__name__`, so a consumer can silence `pcapkit.foundation.registry` while keeping `pcapkit.foundation.extraction` ([#384](https://github.com/JarryShaw/PyPCAPKit/pull/384)). -- `pcapkit.utilities.config`, one module for every tunable limit, each read once from a `PCAPKIT_*` environment variable: the frame-layer and extension-header limits, the dumpkit fallback, the charset-detection cache, the read chunk and the three zero-padding guards. A bad value warns with `ConfigWarning` and falls back to the default; `-1` lifts a limit. **The old homes are removed**, with no re-export ([#1726](https://github.com/JarryShaw/PyPCAPKit/issues/1726), [#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). +- `pcapkit.utilities.config`, one module for every tunable limit, each read once from a `PCAPKIT_*` environment variable: the frame-layer and extension-header limits, the pypcapfile engine's TCP-over-IPv6 warning limit, the dumpkit fallback, the charset-detection cache, the read chunk and the three zero-padding guards. A bad value warns with `ConfigWarning` and falls back to the default; `-1` lifts a limit. The homes these limits had in the prereleases keep no re-export ([#1726](https://github.com/JarryShaw/PyPCAPKit/issues/1726), [#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). #### Changed @@ -334,6 +348,9 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - two more gaps the [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) keyword audit turned up, neither covered by a test: `StreamEOFError`'s docstring did not say that `@prepare` always raises it with `quiet=True` (the end-of-stream convention `StructError` follows via its own `eof=True`), so nothing pinned that silence against regression; and `register_extractor_engine`'s real keyword, `name`, was not itself under test, only its corrected docstring, so a rename could put the two out of step again ([#577](https://github.com/JarryShaw/PyPCAPKit/pull/577)). - `hold_warnings` is in `pcapkit.utilities.warnings.__all__` ([#1602](https://github.com/JarryShaw/PyPCAPKit/issues/1602), [#1603](https://github.com/JarryShaw/PyPCAPKit/pull/1603)). - the default threading excepthook is the fallback before Python 3.10, which has no `threading.__excepthook__` ([#1691](https://github.com/JarryShaw/PyPCAPKit/issues/1691), [#1701](https://github.com/JarryShaw/PyPCAPKit/pull/1701)). +- `stacklevel()` returns a level relative to its caller, as `warnings.warn` and `logging` count it, so a warning or a `BaseError` log record names the caller's frame at any stack depth rather than a frame inside `logging` or pcapkit ([#397](https://github.com/JarryShaw/PyPCAPKit/pull/397)). +- re-executing `pcapkit.utilities.logging` under `PCAPKIT_DEVMODE` keeps one stderr handler, where each `importlib.reload` added another ([#391](https://github.com/JarryShaw/PyPCAPKit/pull/391)). +- a loud `BaseError` no longer sets the process-global `sys.tracebacklimit`, which truncated every later traceback, pcapkit's or not; lazily installed `sys.excepthook` and `threading.excepthook` keep its output terse instead ([#983](https://github.com/JarryShaw/PyPCAPKit/pull/983)). ### pcapkit.vendor @@ -341,11 +358,16 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - thirteen `re.sub` sites under `pcapkit/vendor/` passed `re.MULTILINE` as the fourth *positional* argument, which is `count`, not `flags`: substitution capped at 8 matches, and a `DeprecationWarning` on Python 3.13+ (a future `TypeError`). Found by an AST sweep rather than grep, since 8 of the 13 put the flag on a continuation line. The flag moves to `flags=` at all 13 sites (`default.py` plus one each in `hip/eddsa_curve.py`, `http/frame.py`, `http/method.py`, `http/status_code.py`, `ipv4/router_alert.py`, `ipv6/option.py`, `ipv6/router_alert.py`, `ipv6/tagger_id.py`, `reg/ethertype.py`, `tcp/flags.py`, `tcp/mp_tcp_option.py` and `tcp/option.py`). Behaviour preserving: the shared pattern (`r'\r*\n'`) has no MULTILINE-sensitive anchor, so no `const/` regeneration was needed ([#796](https://github.com/JarryShaw/PyPCAPKit/issues/796)). - `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)): 12 tests across three files, all `RecursionError` (`tests/const/test_const_enum_lookup.py`; two `test_const_enum_no_mint.py` methods, two registries each; 7 of the 12 in `tests/protocols/misc/test_pcapng_unit.py`). `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a two-line `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not the two *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled 5 of 82 regenerations, missing these two. Both crawlers now collapse to the single returning line the other 101 registries use, and the two const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process`'s shape legitimately breaks. The guard sees only 35 of 96 crawlers; the other 61 are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). -- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling (*"only discard changes made by a non-zero sub-vendor... for cron jobs, at most warn about failed sub-vendors, but never fail the entire job"*), three pieces land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). +- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about failed sub-vendors without failing as a whole -- three pieces land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). - three follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: one unpinned invariant and two inaccurate pieces of prose/lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: a test asserts that a `KeyboardInterrupt` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose `_dest_path` is an instance-method-only override fails the *class* call `_snapshot_and_restore` makes but not the *instance* call inside `__init__`, so that target runs unprotected and **succeeds silently**. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**: the belief that R1720 never applies to `try`/`except`/`else` was a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does), and under the `Makefile`'s flags, which CI runs, pylint flags the `else:`, which is deliberate because it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, since `copy2` reopens the path) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). - `TLSKeyLabel` tracked the Mozilla NSS key-log wiki, but `draft-ietf-opsawg-pcapng-06` S4.7 now points at `[I-D.ietf-tls-keylogfile]` (draft-05), published as [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850), "The SSLKEYLOGFILE Format for TLS", whose S4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, `TLSKeyLabel` was missing `ECH_SECRET` and `ECH_CONFIG` ([RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) SS2.3/4.2), so `TLSKeyLabel('ECH_SECRET')` raised `ValueError` on a legitimate Decryption Secrets Block; both are added, `ECH_SECRET` keeping the file's `# nosec B105` bandit convention for `*_SECRET` members. `RSA` has no row in [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850)'s registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since removing a public member is breaking, with a docstring comment marking it the historical exception. `pcapkit/vendor/pcapng/secrets_type.py`'s stale `# NSS Key Log Format` comment is repointed at RFC 9850, and `TLSKeyLabel`'s docstring cites it. A new test pins the member set against RFC 9850 plus the `RSA` exception. Closes [#882](https://github.com/JarryShaw/PyPCAPKit/issues/882) ([#883](https://github.com/JarryShaw/PyPCAPKit/pull/883)). - both spellings of an IANA citation render as `:rfc:` roles in the generated `pcapkit.const` docstrings ([#1664](https://github.com/JarryShaw/PyPCAPKit/issues/1664), [#1673](https://github.com/JarryShaw/PyPCAPKit/pull/1673)). - `MAX_RETRY` and `CI_MODE` stay in `pcapkit.vendor.default`, read from `PCAPKIT_VENDOR_RETRY` and `PCAPKIT_CI_MODE`; a bad value warns with `ConfigWarning` and falls back, where 1.4.x crashed on a non-integer and read `0` as `1` ([#1745](https://github.com/JarryShaw/PyPCAPKit/pull/1745)). +- the IPX socket crawler no longer scrapes Wikipedia, whose page dropped the socket table, so a regeneration stops failing on it ([#507](https://github.com/JarryShaw/PyPCAPKit/issues/507), [#511](https://github.com/JarryShaw/PyPCAPKit/pull/511)). +- the crawlers send a descriptive `User-Agent` built from the package metadata, which Wikipedia requires, and the PCAP-NG crawlers read draft `-03`, the one revision that reproduces their constant files ([#518](https://github.com/JarryShaw/PyPCAPKit/issues/518), [#524](https://github.com/JarryShaw/PyPCAPKit/pull/524)). +- `pcapkit.vendor` and `pcapkit.vendor.mh` list `MH_HandoverInitiateStatus` in `__all__`, where both named `MH_HandoverACKStatus` twice, so a bare `pcapkit-vendor` run reaches its crawler ([#531](https://github.com/JarryShaw/PyPCAPKit/pull/531)). +- with no terminal on stdin, a failed fetch re-raises rather than waiting for an operator to save the page, a wait that printed without end under a closed or piped stdin ([#522](https://github.com/JarryShaw/PyPCAPKit/issues/522), [#542](https://github.com/JarryShaw/PyPCAPKit/pull/542)). +- each crawler derives its `const/` path from the `pcapkit.vendor` root, so a crawler nested two levels down writes to the right file ([#741](https://github.com/JarryShaw/PyPCAPKit/pull/741)). ### Project infrastructure @@ -367,7 +389,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - `tests/vendor/test_vendor_reg_apptype_generator_unit.py` gated on `mypy` with no `MODULE_PROVIDERS` entry, so `_top_level_providers()` died on a bare `KeyError: 'mypy'` once the gate became visible -- a further instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop, like [#766](https://github.com/JarryShaw/PyPCAPKit/issues/766) above. `MODULE_PROVIDERS['mypy'] = ('mypy',)` fixes the lookup, and the three bare `MODULE_PROVIDERS[...]` subscripts route through that function, which raises an `AssertionError` naming the module, key and table on any future gap. `mypy` is deliberately not added to a `pyproject.toml` extra: no extra carries it, it is a `Pipfile` dev-dependency that `lint.yml` installs and already runs whole-package, and `lint.yml` runs no pytest job for `pytest_jobs()` to see, so the fix is a `DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']` entry on `test`, `engine-tests` and `gate`. `dependency_gate_gaps()` goes 15 to 18 `(flag, job)` pairs, all three new and expected; ([#779](https://github.com/JarryShaw/PyPCAPKit/issues/779)). - `engine='pyshark'` had no CI job driving it through a real `tshark`-parsed capture. `pypcap-parity` installs `pyshark` but its `apt-get install` named no `tshark` package, so `PyShark.unsupported_reason()` declined the engine, the extractor fell back to the built-in parser, and the one real-extraction assertion, `assertGreater(extractor.length, 0)`, passed against the fallback's output: a vacuous pass invisible in every skip count. `pypcap-parity` now installs `tshark` with the debconf pre-seed `engine-tests` carries, and the test proves which engine ran (no `EngineWarning`, `extractor._exnam == 'pyshark'`, a `PyShark` instance), as the PyPCAP parity test does. Only `tests/integration/test_engine_runtime.py` and `test_engine_parity.py` change; the length assertion is kept ([#846](https://github.com/JarryShaw/PyPCAPKit/pull/846)). - `engine-tests` ran one Python-version leg per interpreter regardless of which engines that interpreter could install, so `pyproject.toml`'s `PyPCAPFile = ["pypcapfile; python_version < '3.12'"]` marker resolved to *nothing* on 3.12--3.14, yet the `Engines Python 3.12/3.13/3.14` legs passed and, once required, gated merges while exercising an engine that was never installed. Rebuilt as a Python x engine matrix (6 interpreters x 6 engines, 36 cells after 10 `include` overrides) with three outcomes: **26** `supported` cells that install through the real extra and must pass; **6** `unsupported` cells (`PyPCAPFile` on 3.12--3.15, `PyShark` on 3.14/3.15) installed by raw `pip install` to bypass the marker and assert the engine's own `PYTHON_CEILING` decline; and **4** `not-installable` cells (`PyPCAP` on 3.12--3.15) where a genuine C-extension build failure is logged and the test step skipped. The install step never passes silently in either direction. 3.15 is `continue-on-error` throughout. Job names change shape (`Engines Python X` becomes `Engines Python X (engine)`), so the branch-protection ruleset's required-check list was updated in the same window, lest a required check that stopped existing block every merge ([#849](https://github.com/JarryShaw/PyPCAPKit/pull/849)). -- Ruleset `23497679`'s `required_status_checks` names 22 exact check contexts, and GitHub rulesets match literally with no wildcard, so the list needs hand-editing on every matrix change. Five of the 22, the old `Engines Python ` names, are dead since [#849](https://github.com/JarryShaw/PyPCAPKit/pull/849); the five `Compat Python 3.10`-`3.14` names stay live, emitted by `python-compatibility.yml`, and are untouched. Adds one job, `required-checks` (context `Required checks passed`), depending on `[test, integration, engine-tests, pypcap-parity]` with `if: always() && inputs.gate-only != true` and an explicit per-dependency check that fails loudly, naming the job, on anything but `success`: a bare `needs:` list would leave it *skipped* rather than failed when a dependency failed, and GitHub's docs list a skipped required check as passing. That one context is meant to replace 17 of the ruleset's 22 entries (`test`, `integration`, `Engines`, `pypcap-parity`) once the ruleset is updated separately, a repository setting this PR does not touch; the five `Compat` names are excluded from the replacement. One file, `.github/workflows/unit-tests.yml`, 167 lines added; no library or test code ([#856](https://github.com/JarryShaw/PyPCAPKit/pull/856)). +- Ruleset `23497679`'s `required_status_checks` names 22 exact check contexts, and GitHub rulesets match literally with no wildcard, so the list needs hand-editing on every matrix change. Five of the 22, the old `Engines Python ` names, are dead since [#849](https://github.com/JarryShaw/PyPCAPKit/pull/849); the five `Compat Python 3.10`-`3.14` names stay live, emitted by `python-compatibility.yml`, and are untouched. Adds one job, `required-checks` (context `Required checks passed`), depending on `[test, integration, engine-tests, pypcap-parity]` with `if: always() && inputs.gate-only != true` and an explicit per-dependency check that fails loudly, naming the job, on anything but `success`: a bare `needs:` list would leave it *skipped* rather than failed when a dependency failed, and GitHub's docs list a skipped required check as passing. That one context is meant to replace 17 of the ruleset's 22 entries (`test`, `integration`, `Engines`, `pypcap-parity`) once the ruleset is updated separately, as a repository setting outside the change; the five `Compat` names are excluded from the replacement. One file, `.github/workflows/unit-tests.yml`, 167 lines added; no library or test code ([#856](https://github.com/JarryShaw/PyPCAPKit/pull/856)). - the prose the post-[#719](https://github.com/JarryShaw/PyPCAPKit/issues/719) defect wave added or touched, checked against the code; no code changes. `MesgIDOption.seconds` in the Mobility Header schema said seconds since 1970 where it holds an [RFC 1305](https://datatracker.ietf.org/doc/html/rfc1305) NTP count from 1900; two `HTTP._guess_version` comments predated the HTTP/2 gate; and the `_parse_next_layer` note miscounted the captured octets. `Extractor._open_output` now says only the PCAP engine passes header fields, `Extractor.run` and the PCAP-NG engine's `run` list the `FormatError` for PCAP output from a PCAP-NG input, and the extraction and engine pages document the PCAP-output path ([#1179](https://github.com/JarryShaw/PyPCAPKit/pull/1179), [#1180](https://github.com/JarryShaw/PyPCAPKit/pull/1180), [#1186](https://github.com/JarryShaw/PyPCAPKit/pull/1186), [#1103](https://github.com/JarryShaw/PyPCAPKit/issues/1103)). - **a breaking change to** the `all` extra: it carries the core addons only -- `cli`, `crypto` and `NGAP`. The three third-party engines 1.4.x bundled in it (`dpkt`, `scapy` and `pyshark`) and the vendor crawler's `requests` and `beautifulsoup4` are no longer in it; the engines keep their own `DPKT`, `Scapy` and `PyShark` extras and the crawler its `vendor` extra, and `pypcapfile` has a new `PyPCAPFile` extra of its own. A new `dev` extra carries what CI needs ([#910](https://github.com/JarryShaw/PyPCAPKit/issues/910), [#914](https://github.com/JarryShaw/PyPCAPKit/pull/914)). - the `Scapy` and `dev` extras require `scapy>=2.6.0` ([#1572](https://github.com/JarryShaw/PyPCAPKit/issues/1572), [#1578](https://github.com/JarryShaw/PyPCAPKit/pull/1578)), and the `docs` extra `Sphinx>=7.3`, the first release with `versionremoved` ([#1743](https://github.com/JarryShaw/PyPCAPKit/pull/1743)). @@ -379,7 +401,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - `examples/generators/options.py`'s `TCP_BASE` spelled three header fields with names `TCP.make` does not declare, so every option fixture in `examples/captures/options-tcp.pcap` was built from values the generator had not asked for. `make` ends in `**kwargs` and reads nothing out of it, so an undeclared keyword is accepted and discarded with no warning. `'seq': 1` was `seq_no`, so all 25 captured frames carried sequence number 0; `'urgent_pointer'` was `urgent` (requested value and default both 0); and `'ack_flag': False` was `ack` while `'ack': 0` bound to `ack` itself, the acknowledgement *flag* rather than the number (`ack_no`), so that pair was wrong both ways. Each is renamed to the parameter `make` declares, keeping the intended values. `TCP(**TCP_BASE).info.seq` was 0 against a declared 1 and is now 1, and `options-tcp.pcap` changes in exactly 25 bytes (the low octet of each frame's sequence number), with no case status or `EXPECTED_FAILURES` entry moving. The module's six other base mappings were audited: five pass nothing their `make` does not declare, and HIP's `extension` is declared by `HIP.read`, this library's documented way of forwarding a read-path keyword, not a sixth instance ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). - `util/bump_version.py` left `CITATION.cff` naming the previous release. Nothing else maintains that file, so every bump since [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) would have stranded the `version` and `date-released` that GitHub's "Cite this repository" button, citation managers, Zenodo and dependency inventories read. Both fields now move with `__version__`, together, since the file's header says both describe the newest *published* release and moving only one would assert that 1.5.0b5 was released on the day 1.5.0b4 was. The date is UTC, because seven of the thirty most recent bumps were made late evening US-Eastern, where a local date is a day behind the publish; the bump triggers `create-release.yml`, the median gap to the PyPI upload is three minutes, and the UTC dates agree 30 times of 30. The rewrite is line-oriented, so the comment header, key ordering and each field's quoting survive (`cff-version` and a `references` entry's own `version` are anchored out at column zero), and the result is checked with `cffconvert --validate`. An absent file is reported on stderr and skipped rather than failing the vendor cron before its `git commit`, which would discard a whole registry crawl for a documentation file; a file with no `version` field raises before anything is written, since reporting success while rewriting nothing is the staleness being fixed. The script gains a `main()` guard (it ran the whole bump at import, hence no testable surface), and the `import pcapkit` fallback in its version reader, which returned `"1.5.0b4'\n"` (closing quote and newline, which `packaging` rejects), is fixed; it never worked and went unnoticed because the only caller installs the package first. A new gate asserts the committed file still names the packaged version, covering hand-made version changes that never run this script (40 of the 159 commits that moved `__version__` on `main`, 11 of the latest 25) ([#625](https://github.com/JarryShaw/PyPCAPKit/pull/625)). - `LICENSE`'s copyright notice began the term at 2018, a year after the work it covers. The repository's first commit is `c57f7d0b7`, dated 2017-11-07, so the notice understated the term and contradicted `docs/source/conf.py`, whose Sphinx footer is `f'2017-{datetime.date.today().year}, Jarry Shaw'`. The wrong start year survived every maintenance pass because each looked only at the other end of the range: `bc836cfa2` (2020-05-31) introduced `2018-2020` when the BSD-3-Clause text replaced MPL 2.0, and hand bumps to `2018-2022`, `2018-2023` and `2018-2026` ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)) each copied `2018` forward. 2017 is **restored** rather than newly asserted: the initial MIT `LICENSE` read `Copyright (c) 2017 Jarry Shaw`, and the GPL v3 era carried `Copyright (C) 2017 Jarry Shaw` in its appendix; it was lost on 2018-12-08, when `4f3e00d5f` relicensed to Apache 2.0 under `Copyright 2018 Jarry Shaw` and `1c69341dc` replaced that with MPL 2.0 text with no author notice. The end year is **dropped** rather than automated: the notice ships inside the sdist and wheel, so its text is fixed at build time in every copy already downloaded and cannot be computed like the docs footer; copyright subsists from creation whether or not a notice names the current year; and BSD-3-Clause's canonical form is a singular `Copyright (c) `. A range was six years of hand maintenance buying nothing, and automation was considered and rejected, since automating a value that need not be current is worse than not carrying it. `conf.py` is untouched (a self-maintaining footer showing a range is conventional and correct). The licence body is unaltered: one hunk at line 3, and the `DAMAGE.` that [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) repaired from `DAMAGE.s` is still clean ([#630](https://github.com/JarryShaw/PyPCAPKit/pull/630)). -- `CITATION.cff` shipped in no source distribution. `MANIFEST.in` has an `include` each for `README.md`, `LICENSE` and `CHANGELOG.md` but none for the citation file, and its two `global-include` patterns (`*.rst`, `*.py`) cannot match a `.cff`. Removing all three `include` lines and rebuilding shows `README.md` and `LICENSE` shipping anyway (setuptools adds the latter from `license_files`, recorded as `License-File: LICENSE` in `PKG-INFO`) while `CHANGELOG.md` vanishes, so only `CHANGELOG.md` is load-bearing, and a citation file, which no packaging default covers, is in the same position. The gap is invisible from the web UI, because GitHub renders "Cite this repository" from the repository; citation managers, Zenodo and dependency inventories read the published artifact, the surface that was missing it. [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) flagged the omission and [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) did not address it. It matters now because [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) taught `util/bump_version.py` to keep the file's `version` and `date-released` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets `RepositoryCitationTests` in `tests/project/test_bump_version.py` (the gate [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) added for hand-authored bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is not shipped in the source distribution". `python -m build --sdist` gives 860 archive entries before and 861 after (`pypcapkit-1.5.0b4/CITATION.cff` the only difference), the shipped copy is byte-identical, and `twine check --strict` passes on both ([#631](https://github.com/JarryShaw/PyPCAPKit/pull/631)). +- `CITATION.cff` shipped in no source distribution. `MANIFEST.in` has an `include` each for `README.md`, `LICENSE` and `CHANGELOG.md` but none for the citation file, and its two `global-include` patterns (`*.rst`, `*.py`) cannot match a `.cff`. Removing all three `include` lines and rebuilding shows `README.md` and `LICENSE` shipping anyway (setuptools adds the latter from `license_files`, recorded as `License-File: LICENSE` in `PKG-INFO`) while `CHANGELOG.md` vanishes, so only `CHANGELOG.md` is load-bearing, and a citation file, which no packaging default covers, is in the same position. The gap is invisible from the web UI, because GitHub renders "Cite this repository" from the repository; citation managers, Zenodo and dependency inventories read the published artifact, the surface that was missing it. [#615](https://github.com/JarryShaw/PyPCAPKit/pull/615) flagged the omission and [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) did not address it. It matters because [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) taught `util/bump_version.py` to keep the file's `version` and `date-released` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets `RepositoryCitationTests` in `tests/project/test_bump_version.py` (the gate [#625](https://github.com/JarryShaw/PyPCAPKit/pull/625) added for hand-authored bumps) run against an unpacked sdist, where it skipped itself with "CITATION.cff is not shipped in the source distribution". `python -m build --sdist` gives 860 archive entries before and 861 after (`pypcapkit-1.5.0b4/CITATION.cff` the only difference), the shipped copy is byte-identical, and `twine check --strict` passes on both ([#631](https://github.com/JarryShaw/PyPCAPKit/pull/631)). - two inaccurate claims in the [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) entry above, caught in review after [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) had merged; both concern the history of `LICENSE`, which is untouched here. 2017 was and remains the right answer, so this is a changelog-prose correction. The entry said the file "until then was stock MPL text carrying no author notice at all, so 2018 is the first start year the project ever asserted and it was already a year late when it was written". That was true of the single blob `bc836cfa2` replaced and wrong as a description of the era: the pre-BSD file was MIT, then GPL v3, then Apache 2.0, then MPL 2.0; the MIT and GPL texts both carried an author notice naming **2017**; and `2018` first appeared in the Apache notice of 2018-12-08 as a single year, current when written. The claim also contradicted the entry's own dating of the first commit to 2017-11-07. The history supports a stronger argument than the one written, which is why this is corrected rather than deleted: 2017 is the year asserted from the first commit, and [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) restores it. The `BSD 3-Clause License` title line is reattributed from opensource.org's template to choosealicense.com's, whose first line is that title and which carries the `Copyright (c) [year], [fullname]` comma this file uses. The error came from generalising the pre-BSD era off two blobs that were the same file (`bc836cfa2^:LICENSE` and `1c69341dc:LICENSE`); enumerating all 14 commits that touched `LICENSE` caught it ([#638](https://github.com/JarryShaw/PyPCAPKit/pull/638)). - the release workflow published to PyPI and to Anaconda with nothing a human had to approve. `create-release.yml`'s `pypi` job had `environment: release` commented out while the `id-token: write` from the same upstream snippet was re-added live below it, so the block read as disabled as a unit when only its gate still was; the `conda` job had no `environment:` at all. Neither needed a tag: the workflow also fires on completion of `Vendor Update`, so a scheduled registry crawl that bumped the version was enough to publish to two public indexes, a median three minutes from bump to upload. Four jobs reach outside the run and all four are now gated: `github` on `github-release` (the GitHub Release and the `v*` tag), `tag` on `conda-tag` (the commit pushed to `main`), `pypi` on `pypi`, `conda` on `anaconda`. One environment per credential rather than a shared `release`, because approval is granted to an environment and not a job, so one `release` approved once would release both indexes together. PyPI has to be answerable alone: an accepted version number cannot be reused, and `skip-existing: true` makes the re-upload *succeed* having published nothing, so an unintended publish permanently consumes the number the intended release wanted. The workflow half is only half the fix: an `environment:` naming one that does not exist is created implicitly with no protection rules and the job proceeds unapproved, so this is inert until each of the four exists in repository settings with a required reviewer, and each one's "Deployment branches and tags" must stay unrestricted or the `tags: v*` trigger fails outright instead of pausing. `github-pages` is the standing example, carrying no protection rule since 2021. The new test asserts the general rule, that no job running a publishing action or a `git push` may omit an `environment:`, so a publishing job added later without a gate fails rather than ships ([#641](https://github.com/JarryShaw/PyPCAPKit/issues/641)). - three test comments stated an exact count of committed captures under `examples/captures/`, a number that drifts whenever the tracked set changes and in one case was wrong when written: `tests/_tiers.py` said "six," `tests/test_tier_guard.py` "the moment a seventh capture is committed," and `tests/integration/_helpers.py` "four," which undercounted on the day it was written. All three now describe the invariant (`_tiers.py`: "the moment somebody commits another capture... or stops committing one"; `_helpers.py`: "...are fixtures -- some of them committed --..."; `test_tier_guard.py` drops its ordinal). Prose-only, verified with `coverage.parser.PythonParser` that no executable statement moved ([#700](https://github.com/JarryShaw/PyPCAPKit/issues/700)). @@ -388,6 +410,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - `examples/generators/options.py` wrote every HTTP/2 sample frame on stream 0, which [RFC 9113](https://datatracker.ietf.org/doc/html/rfc9113) forbids for `DATA`, `HEADERS` and the other stream-level types, so six of the ten frames in `options-transport.pcap` decoded as `TCP:Raw`. `_httpv2_build` now puts `SETTINGS`, `PING` and `GOAWAY` on stream 0 and every other type on stream 1, with `PUSH_PROMISE` promising stream 2, and all ten decode as `TCP:HTTP/2`; no other generated capture changes ([#1156](https://github.com/JarryShaw/PyPCAPKit/issues/1156)). - `make_samples.py` builds fixtures from the tree it runs in, not the shared checkout ([#1343](https://github.com/JarryShaw/PyPCAPKit/issues/1343), [#1388](https://github.com/JarryShaw/PyPCAPKit/pull/1388)), and retries the Wireshark sample downloads on 429, 5xx and timeouts ([#1658](https://github.com/JarryShaw/PyPCAPKit/issues/1658), [#1659](https://github.com/JarryShaw/PyPCAPKit/pull/1659)). - the Conda Update workflow tags and uploads each bump exactly once ([#1632](https://github.com/JarryShaw/PyPCAPKit/issues/1632), [#1641](https://github.com/JarryShaw/PyPCAPKit/pull/1641)), stays green when a merge races its push ([#1649](https://github.com/JarryShaw/PyPCAPKit/issues/1649), [#1669](https://github.com/JarryShaw/PyPCAPKit/pull/1669)), and keeps the release's name and notes when it uploads ([#1657](https://github.com/JarryShaw/PyPCAPKit/issues/1657), [#1678](https://github.com/JarryShaw/PyPCAPKit/pull/1678)). Vendor Update commits the new modules a crawl adds ([#1685](https://github.com/JarryShaw/PyPCAPKit/issues/1685), [#1688](https://github.com/JarryShaw/PyPCAPKit/pull/1688)). +- package `__all__` lists match what each package holds: six disagreed with it, and `from pcapkit.protocols.data import *` raised `AttributeError` on the unbound `HOPOPT_QuickStartOption` ([#515](https://github.com/JarryShaw/PyPCAPKit/issues/515), [#527](https://github.com/JarryShaw/PyPCAPKit/pull/527)); the public attributes several packages left out are listed ([#533](https://github.com/JarryShaw/PyPCAPKit/issues/533), [#544](https://github.com/JarryShaw/PyPCAPKit/pull/544)). --- diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 5092122bbc..3939825dc4 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -45,7 +45,7 @@ carries only the version being released; this page is the whole history. and a logger that no longer hijacks the consumer's. :doc:`Full changelog `. * **1.4.1** (2026-08-22) -- Release engineering only -- no library behaviour - changed: a new unit-test CI workflow now gates the packaging, vendor-cron + changed: a new unit-test CI workflow gates the packaging, vendor-cron and documentation workflows, the vendor cron refuses to publish when the registry update itself failed, and unit-test compatibility on Python 3.10 was fixed. :doc:`Full changelog `. @@ -57,12 +57,12 @@ carries only the version being released; this page is the whole history. mis-decoded every TCP and UDP frame under IPv4/IPv6 as raw payload -- skip 1.3.4 entirely. :doc:`Full changelog `. * **1.3.4** (2024-11-14) -- Fixed IPv6 Routing header length handling and - extension-header chain extraction (#218), but introduced the + extension-header chain extraction (:issue:`218`), but introduced the transport-layer regression fixed in 1.3.5 -- upgrade past this version rather than to it. :doc:`Full changelog `. * **1.3.3** (2024-11-03) -- Fixed PCAP reading from the CLI and library - (#240), a source install broken by ``setup.py`` (#221), and PCAP output - during flow tracing (#180); 1.3.2 was never released. :doc:`Full changelog + (:issue:`240`), a source install broken by ``setup.py`` (:issue:`221`), and PCAP output + during flow tracing (:issue:`180`); 1.3.2 was never released. :doc:`Full changelog `. * **1.3.1** (2023-12-21) -- Renamed ``register_port()`` to ``register_apptype()`` with no alias left behind, and revised the @@ -72,13 +72,13 @@ carries only the version being released; this page is the whole history. metaclasses, adding ``ModuleDescriptor`` so a registry entry can name a module and class imported lazily on first use. :doc:`Full changelog `. -* **1.2.2** (2023-09-14) -- Fixed extraction failing outright (#166). +* **1.2.2** (2023-09-14) -- Fixed extraction failing outright (:issue:`166`). :doc:`Full changelog `. * **1.2.1** (2023-08-27) -- Fixed bugs in ``SeekableReader`` introduced one day earlier in 1.2.0. :doc:`Full changelog `. * **1.2.0** (2023-08-26) -- Added streaming input through ``SeekableReader``, so a capture can be read from ``stdin`` or a pipe - (#156), and fixed reassembly, broken since 0.16.3 (#155). :doc:`Full + (:issue:`156`), and fixed reassembly, broken since 0.16.3 (:issue:`155`). :doc:`Full changelog `. * **1.1.1** (2023-08-15) -- Fixed ``AppType`` enumeration lookups, ``typing.TypeAlias`` compatibility on older interpreters, the flow-tracing @@ -93,7 +93,7 @@ carries only the version being released; this page is the whole history. resolve ports to service names. :doc:`Full changelog `. * **1.0.2** (2023-06-05) -- Added Mobility Header parsing and construction [:rfc:`6275`], and fixed ``pcapkit.extract`` failing on interpreters where - ``decimal.localcontext`` takes no keyword arguments (#139). :doc:`Full + ``decimal.localcontext`` takes no keyword arguments (:issue:`139`). :doc:`Full changelog `. * **1.0.1** (2023-05-14) -- Changed ``Info`` and ``Schema`` subclasses to finalise at class creation rather than on each instantiation, a straight @@ -103,11 +103,11 @@ carries only the version being released; this page is the whole history. backend became pluggable (with PCAP-NG support and the DPKT, Scapy and PyShark engines), and roughly 55,000 lines changed across 617 files. :doc:`Full changelog `. -* **0.16.3** (2022-10-31) -- Fixed import crashes on some interpreters (#114, - #116), the reassembly property caches, and README rendering for PyPI; the +* **0.16.3** (2022-10-31) -- Fixed import crashes on some interpreters (:issue:`114`, + :issue:`116`), the reassembly property caches, and README rendering for PyPI; the build chain was revised. :doc:`Full changelog `. * **0.16.2** (2022-08-03) -- Fixed ``Info`` ``__init__`` generation for - classes without annotations (#113), and the IP reassembly algorithm (#82). + classes without annotations (:issue:`113`), and the IP reassembly algorithm (:issue:`82`). :doc:`Full changelog `. * **0.16.1** (2022-06-07) -- Changed warnings to raise through ``pcapkit.utilities.warnings`` instead of ``warnings.warn`` directly, and @@ -119,12 +119,12 @@ carries only the version being released; this page is the whole history. ``pcapkit.foundation.registry`` for subscribing protocols by name. :doc:`Full changelog `. * **0.15.5** (2020-10-11) -- Added a fallback encoder so ``DictDumper`` no - longer raises on data it cannot encode (#65). :doc:`Full changelog + longer raises on data it cannot encode (:issue:`65`). :doc:`Full changelog `. * **0.15.4** (2020-08-28) -- Fixed the missing ``stacklevel`` attribute on - ``pcapkit.utilities`` (#58); Travis CI was set up. :doc:`Full changelog + ``pcapkit.utilities`` (:issue:`58`); Travis CI was set up. :doc:`Full changelog `. -* **0.15.3** (2020-08-17) -- Fixed ARP parsing (#55), and added logging +* **0.15.3** (2020-08-17) -- Fixed ARP parsing (:issue:`55`), and added logging through a ``pcapkit`` logger. :doc:`Full changelog `. * **0.15.2** (2020-06-27) -- Added a register interface on the protocol classes and on ``pcapkit.foundation.analysis``, so both extend without @@ -135,7 +135,7 @@ carries only the version being released; this page is the whole history. changelog `. * **0.15.0** (2020-06-07) -- Merged the parsing and construction logic so a protocol is no longer implemented twice, added full API documentation, and - fixed the ``DictDumper`` dependency (#37, #40). :doc:`Full changelog + fixed the ``DictDumper`` dependency (:issue:`37`, :issue:`40`). :doc:`Full changelog `. * **0.14.5** (2019-10-24) -- Added a CLI for the vendor crawlers, and deferred vendor imports so the crawler dependencies are not needed just to @@ -143,7 +143,7 @@ carries only the version being released; this page is the whole history. * **0.14.4** (2019-09-01) -- Shipped the vendor crawler scripts in the distribution, and split the CLI and vendor dependencies into separate ``setup.py`` extras. :doc:`Full changelog `. -* **0.14.3** (2019-08-08) -- Fixed reported bugs (#29, #30) and a Wikipedia +* **0.14.3** (2019-08-08) -- Fixed reported bugs (:issue:`29`, :pr:`30`) and a Wikipedia block that broke a vendor crawler; the constant enumerations and sample output were regenerated. :doc:`Full changelog `. * **0.14.2** (2019-03-28) -- Fixed the exception classes and the TCP diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index 91ad8066c9..7b963f05a5 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -4,12 +4,12 @@ The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, -and a defect programme run through the issue tracker, citing 611 issues and pull +and a defect programme run through the issue tracker, citing 669 issues and pull requests from :pr:`340` to :pr:`1754`. -Preceded by ``1.5.0a1`` (2026-09-15), ``1.5.0b1`` and ``1.5.0b2`` (both -2026-09-18), ``1.5.0b3`` (2026-09-19) and ``1.5.0b4`` to ``1.5.0b12`` (2026-09-20 to -2026-10-13), all published as prereleases and so +Preceded by ``1.5.0a1`` (2026-09-15), ``1.5.0b1`` to ``1.5.0b3`` (all 2026-09-19) +and ``1.5.0b4`` to ``1.5.0b12`` (2026-09-20 to 2026-10-13), dated in UTC and all +published as prereleases and so resolved only by ``pip install --pre``. ``1.5.0b1`` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because ``twine check`` found a Sphinx-only ``:mod:`` role in ``README.rst``, @@ -68,12 +68,13 @@ Changed Aliases show in ``repr()`` and ``str()``, and can be registered per transport (:issue:`807`, :pr:`837`). * **a breaking change to** ``TransportProtocol``: its base moves from ``aenum.IntFlag`` - to ``aenum.IntEnum``, so members no longer compose with ``|`` (:issue:`808`, - :pr:`836`). + to ``aenum.IntEnum``, so ``|`` between two members gives a plain ``int`` rather + than a composite member (:issue:`808`, :pr:`836`). * **a breaking change to** ``TLSKeyLabel``: the hand-written class in ``pcapkit.protocols.misc.pcapng`` becomes the generated - ``pcapkit.const.pcapng.TLSKeyLabel``, crawled from the IANA registry :rfc:`9850` - creates (:issue:`886`, :pr:`890`), and the old path is removed (:issue:`1744`, + ``pcapkit.const.pcapng.tls_key_label.TLSKeyLabel`` (``PCAPNG_TLSKeyLabel`` in + ``pcapkit.const.pcapng``), crawled from the IANA registry :rfc:`9850` creates + (:issue:`886`, :pr:`890`), and the old path is removed (:issue:`1744`, :pr:`1754`). * **a breaking change to** ``Method.get``: the method token is case-sensitive, per :rfc:`9110#section-9.1`, so ``Method.get('get')`` no longer resolves to ``GET`` @@ -176,9 +177,11 @@ Fixed several transports by its lowest set bit, and ``tcp`` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since ``AppType.__eq__`` compares on ``port`` alone. - ``TransportProtocol`` is an ``aenum.IntFlag`` and a member carries the whole set - IANA assigned the service, so ``tcp | udp`` is an ordinary value to pass back in. - Over every multi-transport member this misrouted two: port 888 (``accessbuilder`` + ``TransportProtocol`` was still an ``aenum.IntFlag`` and a member's ``proto`` + carried the whole set IANA assigned the service, so ``tcp | udp`` was an ordinary + value to pass back in; the ``AppType.proto`` and ``TransportProtocol`` entries above + record how :issue:`806` and :issue:`808` later retired both. Over every + multi-transport member this misrouted two: port 888 (``accessbuilder`` resolved to TCP's ``cddbp``) and port 999 (``puprouter`` resolved to TCP's ``garcon``). It now raises ``ProtocolError``, a ``ValueError`` subclass, so the documented contract holds. No library caller passes a composite @@ -213,14 +216,17 @@ Fixed * ``LinkType(209).name`` returned ``'IPMB_LINUX'``, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current ``I2C_LINUX = 209`` and ``aenum`` gives a value to - the member defined first. ``pcapkit/vendor/reg/linktype.py`` now sinks any row - whose note mentions "legacy" after every current row, so the current name is - defined first; today that affects value 209 alone. ``IPMB_LINUX`` stays a - reachable alias (``LinkType['IPMB_LINUX']`` still resolves to 209). Not - breaking: only the canonical name for the shared value moves (:pr:`848`). + the member defined first. ``pcapkit/vendor/reg/linktype.py`` sinks a row whose + note mentions "legacy" after every current row, so the current name is defined + first; in the registry crawled for this release that affects value 209 alone. + ``IPMB_LINUX`` stays a reachable alias (``LinkType['IPMB_LINUX']`` still resolves + to 209). Not breaking: only the canonical name for the shared value moves + (:pr:`848`). The rule sinks only a row whose value another row also holds, + counting values inside ranges, so a current row that merely mentions a legacy + name stays first (:issue:`852`, :pr:`854`). * **a breaking change to** 82 more const registries' ``_missing_``: applies the - owner's mint/unmint ruling (settled on :pr:`847`, confirmed on :issue:`775` -- *"a - final concrete assigned name -> mint; a notation for the readers -> unmint"*) across + owner's mint/unmint ruling (settled on :pr:`847`, confirmed on :issue:`775`: a + final, concrete assigned name mints, and a notation for readers does not) across the 89 ``_missing_`` files still calling ``extend_enum``. 80 registries convert wholly (164 branches), each minting a bare status word (``Unassigned``, ``Reserved [...]``, ``Reserved for Private/Experimental Use``, @@ -273,8 +279,8 @@ Fixed both mint sites -- ``_missing_``'s 766 range branches and ``get()``'s second mint (``PORT_{port}_{transport}`` -> ``'unknown'``) -- now build through a new ``_unregistered_member`` override that reconstructs ``svc``/``port``/``proto``, per - the owner's ruling that *"only IANA registered ones are legit values... get will not - have sufficient information to create new ones"*. All 766+1 branches convert, + the owner's ruling that only IANA-registered values are legitimate and that + ``get()`` lacks the information to create new ones. All 766+1 branches convert, including 8 naming a real service assigned to a whole port span (7 distinct names: ``x11`` twice, ``active-net``, ``satvid-datalnk``, ``vrml-multi-use``, ``ircu``, ``swx``, ``flex-lm``); unlike ``FEATCode``'s fix (:pr:`869`, above) there is no @@ -289,8 +295,8 @@ Fixed ``4`` resolved to ``SCTP.http`` on ``main`` and now **silently** resolves to ``DCCP.unknown``, because a real member sits at ``4`` under both numberings; a bare ``8`` used to mint ``DCCP.PORT_80_dccp`` and now raises ``ValueError``. The - composed-value case is deliberately untested, per the owner's ruling that it is "an - obsoleted path from a breaking change" (:pr:`874`). + composed-value case is deliberately untested, per the owner's ruling that a breaking + change already retired that path (:pr:`874`). * closes the remaining scope of :issue:`775`: ``EtherType`` (52 branches) and ``Socket`` (1), the last registries whose ``_missing_`` minted a permanent member, now return a non-registering ``cls._unregistered_member(value, name)``. The issue's @@ -318,6 +324,14 @@ Fixed registry row names (:issue:`1443`, :pr:`1444`). * the ``AppType`` registry's dict union is defined only on interpreters whose ``dict`` has one (:issue:`1694`, :pr:`1703`). +* NGAP's ``ProcedureCode`` and ``ProtocolIE`` are generated into + ``pcapkit.const.ngap`` from ``pycrate``'s compiled NGAP specification, with no + network fetch. An unseen value gets a non-registering member, where every unknown + key used to mint a member at the shared value ``-1`` and so aliased every other + (:issue:`880`, :pr:`899`). +* ``FEATCode.get`` matches case-insensitively, as :rfc:`5797#section-2` defines FEAT + codes; the registry-wide case-sensitivity audit found no other registry that + needed to fold (:issue:`903`, :pr:`913`). pcapkit.corekit --------------- @@ -566,7 +580,7 @@ Fixed treats a taken value as an alias request, so ``register(existing_value, 'TOTALLY_NEW_NAME')`` returned the existing member, minted nothing and raised nothing, contradicting its docstring. - ``pcapkit.corekit.enums.EnumRegistry`` is a new mixin implementing ``get``, + ``pcapkit.corekit.enum.EnumRegistry`` is a new mixin implementing ``get``, ``get_all``, ``register``, ``register_alias``, ``register_aliases`` and ``_unregistered_member`` once, per the maintainer's ruling on :issue:`842`; it converts the first six registries with no bespoke ``__new__`` @@ -616,9 +630,9 @@ Fixed * **a breaking change to** ``EnumRegistry.get()``: its docstring claimed "It never mints", but both ``cls(default)`` call sites reached ``_missing_`` for a ``default`` inside a still-minting registry's range, growing the registry as a side effect of - resolving ``default`` rather than ``key``. Per the owner's ruling (*"Only register - can mint. get should not mint unless it falls through the _missing_'s minted - ranges"*), both sites now resolve ``default`` through a plain ``_value2member_map_`` + resolving ``default`` rather than ``key``. Per the owner's ruling that only + ``register()`` mints, and ``get()`` only through the ranges ``_missing_`` mints, + both sites now resolve ``default`` through a plain ``_value2member_map_`` lookup; ``key`` resolution is unchanged, so ``EtherType.get(0x0888)`` still mints ``Xyplex_0x0888`` through its own ``_missing_``, as :issue:`775`'s deliberate exception permits. A ``default`` naming no registered member now falls through to @@ -663,6 +677,24 @@ Fixed :pr:`1496`), and its constructor reads only the class's own annotations (:issue:`1692`, :pr:`1702`). * copying or pickling a stream proxy no longer recurses (:issue:`1524`, :pr:`1544`). +* ``@prepare`` reads ``length`` and ``packet`` by keyword as well as by position, so + ``Schema.unpack(data)`` and ``Schema.unpack(data, length=2)`` work where they + raised ``IndexError`` (:issue:`444`, :pr:`450`). +* ``ListField`` unpacks a schema item from the per-item copy its callback and + ``length_callback`` configured, not from the unconfigured original + (:issue:`433`, :pr:`453`). +* an ``*IntField`` given a ``signed=`` that contradicts its fixed sign, such as + ``UInt8Field(signed=True)``, raises ``FieldValueError``; the argument used to be + discarded silently (:issue:`545`, :pr:`549`). +* negative lengths raise ``ProtocolError`` rather than a bare stdlib error: + ``FieldBase.length`` tells a malformed template from a negative resolved length + (:issue:`825`, :pr:`827`), and ``NumberField`` checks a negative resolved + ``length`` and a negative ``bit_length`` (:issue:`828`, :pr:`829`, :issue:`831`, + :pr:`834`). +* ``ModuleDescriptor.klass`` raises ``ProtocolError`` for a class name that resolves + to nothing, or one never given, rather than a bare ``AttributeError``; the two + registry modules share one ``NULL`` sentinel compared by identity, where each had + its own string (:issue:`832`, :issue:`833`, :pr:`835`). pcapkit.dumpkit --------------- @@ -822,7 +854,7 @@ Changed class instead. The error messages are unchanged. The built-ins never pass through a registrar; ``Extractor._register_internal_engine`` and its two siblings check the base for internal registration, and are what the :issue:`513` regression test now - exercises. ``register_protocol`` is untouched. + exercises. ``register_protocol`` is untouched (:pr:`1020`). * extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output (:pr:`420`). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% @@ -1072,8 +1104,10 @@ Added procedures and 438 protocol IEs work and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. ``pycrate`` - is deliberately excluded from the ``all`` extra -- it is LGPL-2.1+ where this - package is BSD-3-Clause, and lands some 238 MB to obtain one module. + was at first left out of the ``all`` extra -- it is LGPL-2.1+ where this package + is BSD-3-Clause, and lands some 238 MB to obtain one module -- until :issue:`910` + counted it among ``all``'s core addons, so ``all`` installs it (the ``all`` entry + under *Project infrastructure*). * the Mobility Header registry, completed (:pr:`383`, :pr:`437`): the :rfc:`5568` fast-handover messages and options, then all 24 registered message data types, all 71 registered options (with nested sub-option registries for the flow @@ -1345,7 +1379,7 @@ Fixed ``FieldError: TCP: [OptNo 30] 3 invalid IP version`` and ``KeyError: 'length'``; both parse now. * two dropped-keyword/wrong-cast defects flagged in review during this release and - never filed until now: HIP's ``_make_param_encrypted`` passed ``cipher=`` to a + filed only as :issue:`556`: HIP's ``_make_param_encrypted`` passed ``cipher=`` to a schema with no such field, so an AES-cipher ``ENCRYPTED`` parameter built through ``make`` packed without its IV; and IPv6-Route's ``RPL.post_process``, which runs on every ``Schema.pack``, assumed ``self.addresses`` was still the concatenated @@ -1528,20 +1562,17 @@ Fixed for the narrower reason the note now gives: ``ConditionalField``'s ``length`` forwards to the wrapped field without consulting the condition, whereas a ``SwitchField`` always resolves to a concrete field (:issue:`603`). -* the one assertion :issue:`604` left pinning the old padding side, red on ``mainline`` +* the one assertion :issue:`604` left pinning the old padding side, red on ``main`` since :pr:`621` merged. ``TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length`` expected a truncated TCP option's ``data`` as the synthesised zero octets *followed - by* the real ones, as ``rjust()`` produced; :pr:`621` made the padding ``ljust()`` - everywhere but could not retarget this file, since :pr:`612` owned it. The real - octets now come first, and the docstring is corrected. Test-only; the sibling case in + by* the real ones, as ``rjust()`` produced: ``Reserved_79`` declaring ``length=12`` + over 6 real octets reports ``aabbccddeeff00000000`` where the test expected + ``00000000aabbccddeeff``. :pr:`621` made the padding ``ljust()`` everywhere but + could not retarget this file, since :pr:`612` owned it. The real octets now come + first, and the docstring is corrected. Test-only; the sibling case in ``tests/protocols/internet/test_ipv4_unit.py`` was retargeted in :pr:`621` (:issue:`604`). -* ``main`` went red the moment :issue:`604`'s ``ljust()`` landed, because that test - still pinned the head-padded short read: ``Reserved_79`` declaring ``length=12`` over - 6 real octets now reports ``aabbccddeeff00000000`` where the test expected - ``00000000aabbccddeeff``. :pr:`621` left the file alone because :pr:`612` owned it - at the time (:issue:`604`, :pr:`621`). * the HIP ``SOLUTION`` builder sized the parameter with ``4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)``, an invalid shorthand for two fields. :rfc:`7401#section-5.2.5` gives ``Length`` as @@ -1905,7 +1936,8 @@ Fixed as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to ``None`` and yielded ``Raw``; it now folds case and warns with ``RegistryWarning`` for a genuinely unregistered name, and ``__init__`` - assigns through that property instead of writing ``_protocol`` past it (:issue:`787`). + assigns through that property instead of writing ``_protocol`` past it (:issue:`787`, + :pr:`789`). * a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: ``b'MESSAGE=hello'`` plus three NULs @@ -2015,8 +2047,8 @@ Fixed (``PCAPNG._make_pcapng_options``) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. ``FEATCode``, ``Command`` and ``Method`` mint the literal wire value as its - own name rather than any placeholder; per the owner's ruling (*"get will not have - sufficient information to create new ones"*: ``Command`` needs + own name rather than any placeholder; per the owner's ruling that ``get()`` lacks + the information to create new members (``Command`` needs ``feat``/``desc``/``type``/``conf`` and ``Method`` needs ``safe``/``idempotent``, which a bare wire string lacks), both ``_missing_`` and each class's own ``get()`` (a second, independent mint site) now build an unregistered member. ``FEATCode``'s @@ -2327,6 +2359,31 @@ Fixed (:issue:`1680`, :pr:`1723`). * the warnings of a next layer replaced by ``Raw`` are dropped (:issue:`1580`, :pr:`1588`). +* IPv6-Route's three data-type readers name the routing type in their + ``ProtocolError``, where each message read ``[TypeNo ]`` + (:issue:`442`), and ``HTTP(..., version=1)`` parses from the frame's data rather + than the drained file, where it raised ``ValueError`` (:issue:`447`, :pr:`451`). +* ``Schema.pack`` accepts a ``tuple`` for a ``ListField``, as the data models + declare, so HIP and MH options that hold one round-trip (:issue:`476`, + :pr:`480`). +* a Mobility Header CGA extension reports its length with its 4-octet header, where + every parsed one read 2 octets short (:issue:`512`, :issue:`517`, :pr:`528`). +* TCP's SACK option enforces the ``8n + 2`` length :rfc:`2018` gives it, as its + docstring promised (:issue:`519`, :pr:`535`). +* ``IPv4.from_data()`` rebuilds a parsed datagram: ``Schema.pack`` checks a payload + against ``ProtocolBase``, and option padding ends in an ``EOOL`` option rather + than a bare wire code (:issue:`506`, :pr:`536`). +* IPv4's SEC option sets the termination bit its own reader requires, sizes the + bitmap from the bit count, and SID is 16 bits wide (:issue:`534`, :issue:`537`, + :pr:`543`). +* the MPTCP option header reads ``length`` from bit 8, per :rfc:`8684#section-3`, + rather than bit 1 (:issue:`553`, :pr:`558`). +* four Mobility Header enumerations defined inline from their RFCs + (``FastBindingAcknowledgmentStatus``, ``IPv6AddressPrefixCode``, + ``LocalizedRoutingStatus``, ``LMAAddressCode``) raise ``EnumValueError`` for an + unassigned value rather than minting ``Unassigned_N`` (:issue:`880`, :pr:`889`), + and all four inherit ``get`` and its ``default`` from ``EnumLookup`` rather than + overriding it (:issue:`935`, :pr:`940`). pcapkit.toolkit --------------- @@ -2425,8 +2482,8 @@ Fixed match the default engine is reverted (:pr:`1718`, :pr:`1721`, :pr:`1724`, :pr:`1727`, closing :issue:`1708` through :pr:`1743`). What each engine leaves out, such as TCP behind a tunnel or an IPv6 extension header, is documented rather - than warned about; the pypcapfile engine still warns once per - capture that it leaves TCP over IPv6 out (:issue:`1513`, :pr:`1535`). + than warned about. The pypcapfile engine keeps its warning, once per capture, + that it leaves TCP over IPv6 out (:issue:`1513`, :pr:`1535`). pcapkit.utilities ----------------- @@ -2442,10 +2499,11 @@ Added (:pr:`384`). * ``pcapkit.utilities.config``, one module for every tunable limit, each read once from a ``PCAPKIT_*`` environment variable: the frame-layer and extension-header - limits, the dumpkit fallback, the charset-detection cache, the read chunk and the - three zero-padding guards. A bad value warns with ``ConfigWarning`` and falls back - to the default; ``-1`` lifts a limit. **The old homes are removed**, with no - re-export (:issue:`1726`, :pr:`1743`). + limits, the pypcapfile engine's TCP-over-IPv6 warning limit, the dumpkit fallback, + the charset-detection cache, the read chunk and the three zero-padding guards. A + bad value warns with ``ConfigWarning`` and falls back to the default; ``-1`` lifts + a limit. The homes these limits had in the prereleases keep no re-export + (:issue:`1726`, :pr:`1743`). Changed ~~~~~~~ @@ -2483,6 +2541,16 @@ Fixed :pr:`1603`). * the default threading excepthook is the fallback before Python 3.10, which has no ``threading.__excepthook__`` (:issue:`1691`, :pr:`1701`). +* ``stacklevel()`` returns a level relative to its caller, as ``warnings.warn`` and + ``logging`` count it, so a warning or a ``BaseError`` log record names the + caller's frame at any stack depth rather than a frame inside ``logging`` or + pcapkit (:pr:`397`). +* re-executing ``pcapkit.utilities.logging`` under ``PCAPKIT_DEVMODE`` keeps one + stderr handler, where each ``importlib.reload`` added another (:pr:`391`). +* a loud ``BaseError`` no longer sets the process-global ``sys.tracebacklimit``, + which truncated every later traceback, pcapkit's or not; lazily installed + ``sys.excepthook`` and ``threading.excepthook`` keep its output terse instead + (:pr:`983`). pcapkit.vendor -------------- @@ -2524,11 +2592,11 @@ Fixed * ``python -m pcapkit.vendor `` could not fail: ``run()`` swallowed every crawler exception as a filterable ``VendorRuntimeWarning`` and ``main()`` always returned ``0``, so a broken crawler regenerating nothing - looked like success to a human and to the cron job. Per the owner's ruling - (*"only discard changes made by a non-zero sub-vendor... for cron jobs, at - most warn about failed sub-vendors, but never fail the entire job"*), three - pieces land. **Exit code** (``pcapkit/vendor/__main__.py``): ``run()`` - returns whether its target succeeded; ``main()`` attempts every target and + looked like success to a human and to the cron job. Per the owner's ruling -- + discard only what a failing sub-vendor changed, and let a cron job warn about + failed sub-vendors without failing as a whole -- three pieces land. + **Exit code** (``pcapkit/vendor/__main__.py``): ``run()`` returns whether its + target succeeded; ``main()`` attempts every target and returns ``1`` if any raised, writing the failing target's qualified name and exception ``repr`` to stderr unconditionally via ``print()`` rather than the filterable warning. **Per-target snapshot and restore**: a new @@ -2586,6 +2654,19 @@ Fixed ``PCAPKIT_VENDOR_RETRY`` and ``PCAPKIT_CI_MODE``; a bad value warns with ``ConfigWarning`` and falls back, where 1.4.x crashed on a non-integer and read ``0`` as ``1`` (:pr:`1745`). +* the IPX socket crawler no longer scrapes Wikipedia, whose page dropped the socket + table, so a regeneration stops failing on it (:issue:`507`, :pr:`511`). +* the crawlers send a descriptive ``User-Agent`` built from the package metadata, + which Wikipedia requires, and the PCAP-NG crawlers read draft ``-03``, the one + revision that reproduces their constant files (:issue:`518`, :pr:`524`). +* ``pcapkit.vendor`` and ``pcapkit.vendor.mh`` list ``MH_HandoverInitiateStatus`` + in ``__all__``, where both named ``MH_HandoverACKStatus`` twice, so a bare + ``pcapkit-vendor`` run reaches its crawler (:pr:`531`). +* with no terminal on stdin, a failed fetch re-raises rather than waiting for an + operator to save the page, a wait that printed without end under a closed or + piped stdin (:issue:`522`, :pr:`542`). +* each crawler derives its ``const/`` path from the ``pcapkit.vendor`` root, so a + crawler nested two levels down writes to the right file (:pr:`741`). Project infrastructure ---------------------- @@ -2772,8 +2853,8 @@ Changed list would leave it *skipped* rather than failed when a dependency failed, and GitHub's docs list a skipped required check as passing. That one context is meant to replace 17 of the ruleset's 22 entries (``test``, ``integration``, ``Engines``, - ``pypcap-parity``) once the ruleset is updated separately, a repository setting this - PR does not touch; the five ``Compat`` names are excluded from the replacement. One + ``pypcap-parity``) once the ruleset is updated separately, as a repository setting + outside the change; the five ``Compat`` names are excluded from the replacement. One file, ``.github/workflows/unit-tests.yml``, 167 lines added; no library or test code (:pr:`856`). * the prose the post-:issue:`719` defect wave added or touched, checked against the @@ -2888,7 +2969,7 @@ Fixed covers, is in the same position. The gap is invisible from the web UI, because GitHub renders "Cite this repository" from the repository; citation managers, Zenodo and dependency inventories read the published artifact, the surface that was missing it. - :pr:`615` flagged the omission and :pr:`619` did not address it. It matters now because + :pr:`615` flagged the omission and :pr:`619` did not address it. It matters because :pr:`625` taught ``util/bump_version.py`` to keep the file's ``version`` and ``date-released`` in step, and a release exercising that path would publish an sdist omitting the artefact under test. It also lets ``RepositoryCitationTests`` in @@ -2981,3 +3062,7 @@ Fixed (:issue:`1649`, :pr:`1669`), and keeps the release's name and notes when it uploads (:issue:`1657`, :pr:`1678`). Vendor Update commits the new modules a crawl adds (:issue:`1685`, :pr:`1688`). +* package ``__all__`` lists match what each package holds: six disagreed with it, + and ``from pcapkit.protocols.data import *`` raised ``AttributeError`` on the + unbound ``HOPOPT_QuickStartOption`` (:issue:`515`, :pr:`527`); the public + attributes several packages left out are listed (:issue:`533`, :pr:`544`). diff --git a/docs/source/contributing/conventions/process.rst b/docs/source/contributing/conventions/process.rst index f2f17131c7..c21c42e542 100644 --- a/docs/source/contributing/conventions/process.rst +++ b/docs/source/contributing/conventions/process.rst @@ -101,7 +101,7 @@ commands down instead of a figure that will be stale by the next merge: The grouping scheme was settled on :issue:`918`: **a section per top-level module, with** ``Added``/``Changed``/``Fixed`` **nested inside each** -- module granularity, not per-file and not per-subpackage. The file carries **10** module-level sections holding -269 entries, and no entry carries an inline kind label:: +292 entries, and no entry carries an inline kind label:: $ grep -cE '^\* \*\*(Added|Changed|Fixed)\*\*' docs/source/changelog/1.5.0.rst 0