Skip to content

docs(foundation): restore Extractor._vfunc's annotation (#1839) - #1867

Merged
JarryShaw merged 4 commits into
mainfrom
docs/1839-vfunc-annotation
Oct 17, 2026
Merged

JarryShaw merged 4 commits into
mainfrom
docs/1839-vfunc-annotation

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

  • Uncomment the _vfunc annotation under its #: comment, so Sphinx attaches
    the description and the type the extraction page asks for. Typed
    Callable[[Extractor, Any], Any] rather than VerboseHandler, whose
    Packet union makes mypy reject the default and engine callbacks; mypy
    output on the extractor and the engines is unchanged.
  • Add a test that the comment attaches, read the way Sphinx's parser reads
    it, plus a cross-check against Sphinx itself where it is installed.

Closes #1839

- Uncomment the `_vfunc` annotation under its `#:` comment, so Sphinx attaches
  the description and the type the extraction page asks for. Typed
  `Callable[[Extractor, Any], Any]` rather than `VerboseHandler`, whose
  `Packet` union makes mypy reject the default and engine callbacks; mypy
  output on the extractor and the engines is unchanged.
- Add a test that the comment attaches, read the way Sphinx's parser reads
  it, plus a cross-check against Sphinx itself where it is installed.

Closes #1839
@JarryShaw JarryShaw added docs Pull requests that change documentation only (docs: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 16, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 74efba3ce. A Sonnet cross-review (author: Opus) confirmed:

  • ModuleAnalyzer.find_attr_docs() now attaches "Verbose callback function." to Extractor._vfunc. That is its only difference from main, and the # NOTE: block does not leak into the doc text.
  • A minimal Sphinx build renders _vfunc: Callable[[Extractor, Any], Any] with its description.
  • Both new tests fail on main and pass here, and tests/project passes.
  • The annotation matches every assignment and call site: ext._vfunc(ext, frame). mypy output with the repo's mypy.ini is identical to main, and VerboseHandler would add errors at the sites the NOTE names.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 16, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

The Callable signature should be more clear. Avoid using Any is we have a specific format of this function's signature.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Understood: _vfunc will carry the callback's actual signature instead of Any. That means the VerboseHandler alias, with the extractor and the frame as parameters and no return value. The engines' default callbacks will be typed to match, rather than the annotation being loosened to suit them. A new commit is coming on this branch.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 17, 2026
…1839)

- Type `VerboseHandler` as `Callable[['Extractor', Packet], object]` and
  annotate `_vfunc` with it. The return is `object`, not `None`: every
  caller discards it, and `None` would make mypy reject a user callback
  that returns a value.
- Narrow the frame to the engine's own packet type with `cast` inside the
  default callback and each engine's callback, so they accept the alias's
  `Packet` union. Runtime output is unchanged.
- Pin the annotation and the alias's signature in the #1839 test.

Refs #1839
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 17, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Pushed e7d2465dd. VerboseHandler is now Callable[[Extractor, Packet], object], and _vfunc is annotated with it. Each engine's callback takes the Packet union and narrows with cast to its own packet type. mypy shows no new errors on foundation/.

The return is object rather than None deliberately. With None, mypy rejects a user callback that returns a value, such as def cb(e, f): return 1 or sys.stdout.write, for both Extractor(verbose=...) and extract(verbose=...). main accepts those today. object is just as precise, since the result is always discarded, and rejects nothing users write now. If you want None regardless, it is a one-word change.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Correction to my comment above. sys.stdout.write is a bad example: it takes one argument, so mypy rejects it as a verbose callback on every variant, main included, and it would fail at runtime too.

The point still holds for real two-argument callbacks. A re-review measured def cb(e, f) -> int, lambda e, f: 1 and lambda e, f: sys.stdout.write(str(f)): all are accepted on main (Any) and with object, and all are rejected with None. A follow-up commit fixes the same wrong example in the source NOTE.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 17, 2026
- Replace `sys.stdout.write`, which takes one argument and so is no verbose
  callback at all, with `lambda e, f: sys.stdout.write(str(f))`: it takes the
  extractor and the frame and returns a value, so `object` accepts it where
  `None` would not.

Refs #1839
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 817208c8a. The Sonnet re-review (author: Opus) of e7d2465dd checked the following and asked for only one change.

  • Return type. It measured that object accepts every two-argument callback main accepts, and that None rejects those that return a value.
  • Casts. It checked each engine's cast against what that engine passes to _vfunc. Wrong targets are caught downstream for dpkt, scapy, pyshark and pypcap.
  • mypy and tests. mypy output is unchanged, also with --follow-untyped-imports. The new signature test fails on the first commit and catches mutation to None or Any.
  • Runtime. verbose=True output is byte-identical for the PCAP, PCAP-NG, scapy and dpkt engines.

The one change was the NOTE's sys.stdout.write example, which takes one argument. 817208c8a replaces it with lambda e, f: sys.stdout.write(str(f)). That commit is comment-only (AST unchanged), and the worker re-checked it with mypy: accepted with object, rejected with None.

Non-blocking: _vfunc now renders its full union, with three same-named Packet entries. Qualifying them is noted for the conf.py work. The engines replacing a user's callback is now #1874.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 17, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

For the typing, in that case, I think Any would be a better one for object.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Understood. The callback's return type becomes Any rather than object. That matches what main has today, and it still accepts every two-argument callback.

A worker will change VerboseHandler, the NOTE that explains the choice, and the signature test that currently pins object. It will then rebase #1880, which is stacked on this branch, onto the new head. The review label resets when the new head is pushed.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 17, 2026
- Type `VerboseHandler`'s return as `Any` rather than `object`, per the
  owner's preference on #1867: every caller discards the result, and `Any`
  accepts any two-argument callback, whatever it returns.
- Rewrite the NOTE above the alias to give that reason.
- Pin `Any` in the signature test; the exact comparison still fails on a
  return of `None` or `object`.

Refs #1839
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 17, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 5b0e9fdbe. The Sonnet reviewer re-checked the Any change made per your ruling.

  • VerboseHandler is Callable[['Extractor', Packet], Any], and Any is imported in the same TYPE_CHECKING block.
  • The NOTE is accurate: every _vfunc(...) call discards its result.
  • The signature test fails if Any is replaced with object, None or int.
  • mypy on extraction.py is clean. It accepts def cb(e, f) -> int, lambda e, f: 1 and lambda e, f: sys.stdout.write(str(f)), and rejects only a one-argument callback. As a control, the same callbacks are rejected when the return type is None.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 90.05% (unit tier, Python 3.15, 5b0e9fdbe, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 2 20 2 96.77%
pcapkit/const 19595 881 2618 867 92.11%
pcapkit/corekit 2462 69 780 35 96.55%
pcapkit/dumpkit 276 3 100 3 98.40%
pcapkit/foundation 2914 164 1016 34 93.59%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17406 224 4580 189 98.10%
pcapkit/toolkit 872 177 298 8 77.18%
pcapkit/utilities 512 2 158 4 99.10%
pcapkit/vendor 4407 2342 1010 157 43.27%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 7b40f7a into main Oct 17, 2026
57 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 17, 2026
@JarryShaw
JarryShaw deleted the docs/1839-vfunc-annotation branch October 17, 2026 13:19
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Pull requests that change documentation only (docs: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

docs(foundation): Extractor._vfunc's annotation is commented out, so its documented attribute renders empty

1 participant