chore: drop stale build leftovers and move the examples' READMEs to Markdown (#1773)

.gitignore

@@ -18,7 +18,7 @@ examples/captures/*
 # PCAP-NG packet blocks their captured octets back. Regenerating them on every such
 # change is work with no reader, and the tier guard in `tests/_tiers.py` asks git
 # which captures are tracked rather than hardcoding a list, so dropping them here
-# is the whole change -- see `examples/legacy_smoke/README.rst` for how to rebuild
+# is the whole change -- see `examples/legacy_smoke/README.md` for how to rebuild
 # them locally.
 !examples/captures/in.pcap
 !examples/captures/dhcp.pcapng

MANIFEST.in

@@ -2,14 +2,15 @@ global-exclude .DS_Store
 
 global-include *.rst
 global-include *.py
+# Neither global-include reaches Markdown, which the examples' READMEs are (#719).
+recursive-include examples *.md
 
 prune .eggs
 prune .github
 prune .venv
 prune docs
 prune examples/captures
 prune temp
-prune test
 
 # Belt-and-braces, both of them: setuptools ships these two whether or not this
 # file mentions them, so neither line is load-bearing. An earlier version of this

Makefile

@@ -193,8 +193,10 @@ mypy:
 bandit:
 	$(RUN) bandit $(BANDIT_FLAGS) pcapkit
 
+# Profile one extraction and render its call graph; the tools the recipe needs
+# are listed in examples/legacy_smoke/README.md.
 profile:
-	$(MAKE) -C test profile
+	$(MAKE) -C examples/legacy_smoke profile
 
 mypy-types:
 	pipenv run mypy --install-types

docs/source/contributing/conventions/documentation.rst

@@ -289,10 +289,10 @@ Format and Mechanics
    ruled this on :issue:`719`, correcting a blanket *always* ``.rst`` that had been in
    circulation: the Sphinx documentation is reST, and the other documents --
    the READMEs included -- are Markdown where that applies. ``CONTRIBUTING.md``'s own
-   *Documentation* section records the same split. One trap arrived with the ruling:
-   :file:`MANIFEST.in` reaches the two READMEs under :file:`examples/` through
-   ``global-include *.rst`` and has no ``*.md`` equivalent, so renaming one without
-   adding coverage drops it from every source distribution.
+   *Documentation* section records the same split. Neither ``global-include`` in
+   :file:`MANIFEST.in` reaches Markdown, so the two READMEs under :file:`examples/`
+   ship through ``recursive-include examples *.md``, and any other Markdown file
+   ships only if setuptools or an ``include`` line adds it.
 *  **Prose names the real defining module, not a re-export.** One strand of the
    consistency sweep on :doc:`the roadmap </contributing/pep>`. The sentinels are the
    worked example: all four are defined in :mod:`pcapkit.corekit.sentinels`, and

examples/benchmark/benchmark.py

@@ -22,7 +22,7 @@
 :data:`ROUNDS` timed extractions of the same capture per engine, timed with
 :func:`time.perf_counter_ns`, first sample discarded as a warm-up, reported as
 milliseconds per packet. The deliberate departures are listed in
-:file:`README.rst` under "Departures from the legacy methodology".
+:file:`README.md` under "Departures from the legacy methodology".
 
 Two properties matter more than the numbers themselves.
 

examples/legacy_smoke/README.md

@@ -0,0 +1,126 @@
+# Demonstration Scripts
+
+A tour of `pcapkit` by example: each script here does one thing and prints or dumps
+the result, so you can get a thorough view of the library either on how to use it or
+on what it can do. They are demos rather than tests -- `pytest` does not collect
+this directory (`testpaths = ["tests"]`) and nothing in CI runs them, so they are
+also where API drift goes unnoticed the longest. Run them after changing anything
+public.
+
+## Running them
+
+Every script reads its capture with a path relative to its own directory, so **run
+them from inside** `examples/legacy_smoke/`, not from the repository root:
+
+```shell
+cd examples/legacy_smoke
+python test_basic.py
+```
+
+The captures live in `../captures/`. Only `in.pcap` and `dhcp.pcapng` are
+committed; the rest are generated, so build them once before running anything else:
+
+```shell
+python ../generators/make_samples.py     # or: make samples, from the repository root
+```
+
+## The scripts
+
+| Script | What it demonstrates |
+| --- | --- |
+| [test_basic](test_basic.py) | the smallest useful call -- extract `in.pcap` to a tree-view text file |
+| [test_extractor](test_extractor.py) | `pcapkit.extract` dumping the same capture as plist, JSON and tree |
+| [test_file](test_file.py) | reading from an already-open binary file object rather than a path |
+| [test_api](test_api.py) | most of the keyword surface at once: per-frame files, reassembly and flow tracing |
+| [test_tcp](test_tcp.py) | extraction of TCP packets |
+| [test_ipv6](test_ipv6.py) | extraction of IPv6 packets, one output file per frame |
+| [test_pcapng](test_pcapng.py) | extraction of a PCAP-NG capture, which uses a different engine |
+| [test_http](test_http.py) | iterating frames manually and testing `pcapkit.HTTP in frame` |
+| [test_reassembly](test_reassembly.py) | TCP payload reassembly, writing the reassembled datagrams out |
+| [test_ip_reasm](test_ip_reasm.py) | IPv4 fragment reassembly |
+| [test_ipv6_reasm](test_ipv6_reasm.py) | IPv6 fragment reassembly |
+| [test_analyse](test_analyse.py) | analysis of the application layer *after* reassembly, printing the recovered HTTP messages |
+| [test_trace](test_trace.py) | tracing TCP flows and dumping the flow index |
+| [test_engine](test_engine.py) | the same capture through each extraction engine, side by side |
+| [test_time](test_time.py) | timing each engine, reported as milliseconds per packet |
+| [test_profile](test_profile.py) | cProfile stats for one extraction; `make profile` renders the call graph |
+| [test_perf](test_perf.py) | the same comparison under the `pyperf` benchmark harness |
+| [test_stream](test_stream.py) | extracting a live capture streamed from `tcpdump` on stdin |
+| [test_stream_askpass](test_stream_askpass.py) | the same, with `sudo -A` for a non-interactive password |
+
+`_engine_support.py` is not a demo -- it is the shared helper the engine demos use
+to work out whether an engine can run on this host.
+
+## What needs more than pcapkit
+
+Most of these run with nothing but `pcapkit`. These do not:
+
+- **test_engine, test_time, test_perf** exercise the `dpkt`, `scapy` and
+  `pyshark` engines. Those are optional packages, and `pyshark` additionally
+  drives Wireshark's `tshark` binary. An engine that is missing is reported as
+  skipped, with the reason, and the rest still run -- see `_engine_support.py`. Two
+  things worth knowing:
+
+  - `pyshark` 0.6 does not work on Python 3.14 at all. It asks
+    `asyncio.get_event_loop_policy().get_event_loop()` for a loop on a thread that
+    has none, which 3.14 no longer creates implicitly, so every extraction through it
+    raises `RuntimeError: There is no current event loop in thread 'MainThread'`
+    before `tshark` is even reached. That is a pyshark bug, not a pcapkit one, and
+    there is nothing to configure around it.
+  - When an engine's *package* is missing, `pcapkit.extract` does not fail. It warns
+    and falls back to pcapkit's own parser, so the extraction succeeds under the wrong
+    engine. The demos name the driver that actually ran, so a fallback is visible
+    rather than being reported as a healthy result.
+
+- **test_perf** needs the `pyperf` harness, which `pcapkit` does not depend on:
+  `python -m pip install pyperf`. Without it the script says so and stops.
+  `test_time.py` does the same comparison with no extra dependency.
+- **test_profile**'s `make profile` target needs `gprof2dot`, `graphviz` and
+  `snakeviz` to render the call graph. The script itself only needs `pcapkit`.
+- **test_stream, test_stream_askpass** run `sudo tcpdump` against a live interface,
+  so they need root and cannot run unattended. `INTERFACE` at the top of each is
+  `en0` (macOS); on Linux it is usually `eth0` or `wlan0`. They buffer the live
+  capture into a temporary file -- never into `../captures/`, which holds generated
+  captures the test suite pins.
+
+## Regenerating the reference reports
+
+Four files in `../captures/` are reports rather than inputs -- `out.json`,
+`out.plist`, `out.txt` and `pcapng.txt` -- and they are produced by two of the
+scripts here. **Git does not track them**, so a fresh clone does not have them. Build
+them the way you would any other generated capture, and again after a change to how
+`pcapkit` renders one:
+
+```shell
+cd examples/legacy_smoke && make fixtures
+```
+
+That is the whole recipe. It runs, equivalently:
+
+```shell
+TZ=UTC python test_extractor.py     # ../captures/out.json, out.plist, out.txt  <- in.pcap
+TZ=UTC python test_pcapng.py        # ../captures/pcapng.txt                    <- dhcp.pcapng
+```
+
+Both *inputs* are committed, so this works on a fresh clone with no `make samples`
+first. Use `make fixtures PYTHON=../../.venv/bin/python` to pick a specific
+interpreter.
+
+These four were tracked once, and #685 is why they are not any more. Nothing reads a
+report back -- no test asserts on one, and the docs only quote them as prose -- so a
+parser change that alters how a frame renders leaves the committed copy quietly wrong,
+with no failure anywhere to say so. `pcapng.txt` sat recording `packet -> NIL` for
+four blocks that had started carrying their captured octets, and the staleness was
+found by reading the file rather than by anything breaking. Generating them on demand
+means the copy on your disk describes the tree you are standing in, which was the only
+thing that made them worth looking at.
+
+`TZ=UTC` **is not decoration.** `pcapkit` renders frame timestamps in the host's
+local zone, so an unpinned run rewrites every timestamp line in all four files with
+wherever it happened to be run -- which is how these reports came to disagree with
+each other while they were still tracked, the PCAP three having been generated at
+UTC-05:00 and `pcapng.txt` at UTC+08:00. Pinning UTC makes the output reproducible on
+any host, so two runs can be diffed against each other. It also happens to be the only
+zone in which PCAP-NG's `timestamp_epoch` is the true UNIX epoch:
+`PCAPNG._read_timestamp` adds the zone's UTC offset to the value it returns, so a
+report generated anywhere else bakes in that offset.

setup.py

@@ -47,7 +47,7 @@ def refactor(path: "str") -> "None":
             )
         except subprocess.CalledProcessError as error:
             logger.error(
-                "Failed to perform assignment expression backport compiling. "
+                "Failed to perform f-string backport compiling. "
                 "Please consider manually install `bpc-f2format` and try again."
             )
             sys.exit(error.returncode)
@@ -70,7 +70,7 @@ def refactor(path: "str") -> "None":
             )
         except subprocess.CalledProcessError as error:
             logger.error(
-                "Failed to perform assignment expression backport compiling. "
+                "Failed to perform positional-only parameter backport compiling. "
                 "Please consider manually install `bpc-poseur` and try again."
             )
             sys.exit(error.returncode)
@@ -156,24 +156,9 @@ def run(self) -> "None":
         "setuptools version is too old to support PyBPC conversion in install."
     )
 
-try:
-    from setuptools.command.bdist_wheel import bdist_wheel
-
-    class pcapkit_bdist_wheel(bdist_wheel):
-        """Modified bdist_wheel to run PyBPC conversion."""
-
-        def run(self) -> "None":
-            super(pcapkit_bdist_wheel, self).run()
-            logger.info("running bdist_wheel")
-
-            # PyBPC compatibility enforcement
-            refactor(os.path.join(self.dist_dir, "pcapkit"))  # type: ignore[arg-type]
-
-    cmdclass["bdist_wheel"] = pcapkit_bdist_wheel
-except ImportError:
-    logger.warning(
-        "setuptools version is too old to support PyBPC conversion in bdist_wheel."
-    )
+# No bdist_wheel override: ``bdist_wheel`` runs ``build_py`` and ``install``
+# above, so the tree it archives is already converted before the wheel is
+# written, and ``dist_dir`` holds only the finished wheels (#1773).
 
 setup(
     cmdclass=cmdclass,