Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions tests/_dependency_gates.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
every ``test_*.py`` looking at :func:`unittest.skipUnless` decorators on
classes *and* on methods. A text search finds the method-level ones and
misses the class-level ones entirely, which is most of them: ``HAS_EMOJI``'s
14 methods are gated by two class decorators.
methods, for one, are gated by class decorators alone.
2. **What the flag requires**, from :func:`flag_requirements` -- the module
names its defining expression probes, read out of the source rather than
listed here. A flag defined in one module and imported by another
Expand Down Expand Up @@ -117,14 +117,14 @@

#: The workflow holding every blocking job that runs :program:`pytest`.
#: :file:`experimental.yml` runs the unit tier as well, on legs nothing waits on
#: (#1732), and is not read here. Five other workflows install ``.[all]``
#: (#1732), and is not read here. Other workflows install ``.[all]``
#: somewhere -- a docs build, a conda recipe, a vendor crawl, the lint pass,
#: and the release packaging workflow -- and none of them invokes the suite;
#: :file:`python-compatibility.yml` installs a bare ``.`` and only compiles and
#: imports, and :file:`codeql-analysis.yml` installs nothing explicitly at all
#: (CodeQL's own autobuild step). That is exactly why this module keys on *jobs that
#: run pytest* rather than on install lines anywhere in the workflow tree: three
#: of those five install ``.[all,dev]`` -- carrying ``pypcapfile``, ``pyshark``
#: run pytest* rather than on install lines anywhere in the workflow tree: some
#: of those install ``.[all,dev]`` -- carrying ``pypcapfile``, ``pyshark``
#: and ``scapy`` through the ``dev`` extra #910 added when it narrowed ``all``
#: to core addons only -- so a guard reading those lines would satisfy nearly
#: every flag here vacuously.
Expand Down Expand Up @@ -1174,9 +1174,9 @@ def flag_requirements() -> 'dict[tuple[str, str], frozenset[str]]':
"""``(module, flag)`` -> the modules that flag needs importable.

Resolved per file rather than per flag name, which matters for exactly one
flag and matters a lot: ``HAS_RUNTIME`` names the four core dependencies in
flag and matters a lot: ``HAS_RUNTIME`` names the core dependencies in
every module but one, and in
:file:`tests/foundation/engines/test_runtime_engines.py` it names those four
:file:`tests/foundation/engines/test_runtime_engines.py` it names those
plus ``dpkt``, ``scapy`` and ``pyshark``. Keyed on the name alone, that one
module's requirements would be demanded of every other module that gates on
it, or they would excuse it.
Expand Down Expand Up @@ -1563,8 +1563,8 @@ def dependency_gate_gaps(workflow: 'Optional[pathlib.Path]' = None) -> 'tuple[Ga

Aggregated per ``(flag, job)`` rather than per gate: one missing extra
darkens every gate on that flag at once, and a reader wants "the ``test``
job reaches 14 ``HAS_CRYPTO`` methods and installs no ``cryptography``",
not fourteen copies of it.
job reaches N ``HAS_CRYPTO`` methods and installs no ``cryptography``",
not one copy of it per method.

"""
requirements = flag_requirements()
Expand Down
12 changes: 6 additions & 6 deletions tests/_support.py
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ def load_module(module_name: str, relative_path: str,
Both halves of that write to :data:`sys.modules` -- the stub parents, and
``module_name`` itself -- so both are taken back off again when ``test``
finishes, via :func:`restore_modules_after`. Before issue #674 they were not,
and five test modules left a bare ``pcapkit`` bound for whatever ran next.
and test modules left a bare ``pcapkit`` bound for whatever ran next.

Args:
module_name: Dotted name to execute the file under, e.g.
Expand Down Expand Up @@ -405,12 +405,12 @@ def load_module(module_name: str, relative_path: str,
def bootstrap_core_modules(test: 'Optional[unittest.TestCase]' = None) -> dict[str, object]:
"""Execute the core :mod:`pcapkit` modules from source, in dependency order.

The seven modules the unit tier needs before it can test anything else, each
The modules the unit tier needs before it can test anything else, each
loaded by :func:`load_module` and so each restored when ``test`` finishes.
The restore is arranged here as well as in :func:`load_module`, and
deliberately: :func:`restore_modules_after` registers one cleanup per test
however many times it is called, so the snapshot kept is the one taken here,
before the first of the seven loads wrote anything.
before the first of those loads wrote anything.

Args:
test: The running test. Found from the calling frames when omitted, as in
Expand Down Expand Up @@ -465,7 +465,7 @@ def install_fake_protocol_module(test: 'unittest.TestCase') -> type:
which is what puts :data:`sys.modules` back afterwards. Required
rather than optional so that a future caller cannot install the
stand-in without arranging for its removal -- the mistake is a
:exc:`TypeError` at the call rather than twelve unrelated failures
:exc:`TypeError` at the call rather than a run of unrelated failures
in another directory.

Returns:
Expand Down Expand Up @@ -694,7 +694,7 @@ def _running_test_case() -> 'Optional[unittest.TestCase]':
has to be *handed* the running test is a helper every call site can forget to
hand it to, and forgetting is silent. Issue #674 is the bill for that --
:func:`bootstrap_core_modules` and :func:`load_module` left stub ``pcapkit``
entries in :data:`sys.modules` at five call sites, not one of which looked
entries in :data:`sys.modules` at call sites none of which looked
wrong, and the resulting failures landed in a different directory.

The *nearest* enclosing ``self`` wins, and that is the right one rather than
Expand Down Expand Up @@ -871,7 +871,7 @@ def isolate_modules(test: 'unittest.TestCase',
difference is the whole of issue #660: purging on the way *in* protects the
test that does it and nothing else, so a test that replaces
``pcapkit.protocols.protocol`` with a stand-in and then finishes leaves that
stand-in bound for whatever runs next. Twelve tests in
stand-in bound for whatever runs next. Tests in
:mod:`tests.project.test_public_api` and
:mod:`tests.project.test_documentation_claims` failed with ``TypeError: type
'ProtocolBase' is not subscriptable`` on exactly that, and only when the
Expand Down
9 changes: 5 additions & 4 deletions tests/_tiers.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
on demand and gitignored. A machine that has run ``make samples`` has them all,
so the test passes locally and then fails on a fresh CI checkout with a
missing-file error that blames the fixture rather than the tier rule. It has
happened three times (GitHub pull requests #372 and #384, and once more), and
happened more than once (GitHub pull requests #372 and #384 among them), and
each time it cost a CI round-trip to work out.

So this module answers, cheaply and without needing the fixtures themselves:
Expand All @@ -49,9 +49,10 @@
* :func:`is_roundtrip_sweep` -- is this unit-tier module one of the round-trip
sweeps, which only the ``latest`` leg of the ``test`` job runs (#1732)?

:file:`tests/conftest.py` drives the first four at collection time, and
:func:`tests._support.sample_path` consults :func:`check_unit_tier_read` on
every call. Both paths no-op when git cannot answer, so an unpacked source
:file:`tests/conftest.py` runs :func:`is_unit_tier` and :func:`audit_module` --
and through it :func:`committed_captures` and :func:`explain` -- at collection
time, and :func:`tests._support.sample_path` consults :func:`check_unit_tier_read`
on every call. Both paths no-op when git cannot answer, so an unpacked source
tarball still runs its tests.

:func:`fixture_tier_paths` is what the ``integration`` job of
Expand Down
18 changes: 9 additions & 9 deletions tests/cli/test_main.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@
""":mod:`pcapkit.__main__`, against stand-ins for everything it imports.

The command line tool is tested without the library behind it: :mod:`pcapkit`,
:mod:`pcapkit.foundation.extraction`, :mod:`pcapkit.interface`, three
:mod:`pcapkit.utilities` modules and :mod:`emoji` are all replaced with
:mod:`pcapkit.foundation.extraction`, :mod:`pcapkit.interface`, the
:mod:`pcapkit.utilities` modules it imports and :mod:`emoji` are all replaced with
stand-ins, so what is under test is the argument wiring and nothing else. The
real thing runs in :mod:`tests.integration.test_cli_subprocess`.

:data:`sys.modules` is process-global, so binding a stand-in over a real module
name is a write that outlives the test unless something undoes it -- and these
stand-ins are *emptier* than what they replace, which is the quiet kind. Issue
#688: this file wrote nine names and put none of them back -- the seven stand-ins
#688: this file wrote those names and put none of them back -- the stand-ins
above, the bare ``pcapkit.foundation`` parent they hang from, and
``pcapkit.__main__`` itself. Pairing it with :mod:`tests.project.test_public_api`
gave ten errors: nine ``ImportError: cannot import name 'show_flag_values' from
'pcapkit.utilities.compat'`` and one for ``SeekError``, because the real library's
next import found a ``pcapkit.utilities.compat`` carrying exactly one name.
gave a run of ``ImportError: cannot import name 'show_flag_values' from
'pcapkit.utilities.compat'`` errors and one for ``SeekError``, because the real
library's next import found a ``pcapkit.utilities.compat`` carrying exactly one name.

That is the same defect as issues #660 and #674, and it is fixed the same way:
:func:`tests._support.isolate_modules` in ``setUp``, which purges the region on
Expand Down Expand Up @@ -65,7 +65,7 @@

#: How many module names :func:`summarise_names` spells out before it starts
#: counting instead. Bounded because the interesting list is long: on a warm
#: module table the :mod:`pcapkit` region holds some three hundred names, so an
#: module table the :mod:`pcapkit` region holds hundreds of names, so an
#: unbounded report of what a leak did to it is twelve kilobytes of failure
#: message that nobody reads -- measured, at 12089 characters.
NAMES_IN_FAILURE_MESSAGE = 8
Expand Down Expand Up @@ -107,8 +107,8 @@ def setUp(self) -> None:
def assert_module_table_restored(self, before: 'dict[str, types.ModuleType]') -> None:
"""The :data:`ISOLATED` region of :data:`sys.modules` is as it was found.

The regression assertion for issue #688, and exact in all three
directions a restore can be wrong: a name this test added, a name it
The regression assertion for issue #688, and exact in every
direction a restore can be wrong: a name this test added, a name it
dropped, and a name it rebound to something else. Absence matters as much
as presence -- ``pcapkit.utilities.compat`` did not exist before a test
here ran on a cold table, so it must not exist after one either, and a
Expand Down
14 changes: 7 additions & 7 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ def pytest_sessionstart(session: 'pytest.Session') -> 'None':
region is empty at the pin and restored to empty after every test, which
makes each of the tests that does not purge for itself -- :mod:`tests.project`
is the bulk of them -- pay a fresh ``import pcapkit`` it used to get from the
warm table. Measured over the 96 tests of ``tests/project/``: 1.59s with no
warm table. Measured over ``tests/project/``: 1.59s with no
guard at all, **9.34s** guarded from a cold table, **0.35s** guarded from a
warm one. The last is the fastest of the three because the import it would
otherwise do inside the first test has moved here instead.
Expand Down Expand Up @@ -161,8 +161,8 @@ def pytest_sessionstart(session: 'pytest.Session') -> 'None':
in ``TCP``/``UDP`` and :attr:`AppType.__registries__
<pcapkit.const.reg.apptype.apptype.AppType.__registries__>`. The next direct
import of the submodule then built a second ``TCP`` and ``UDP``, and
:mod:`tests.project.test_conventions_doc_claims` counted both generations:
``127 != 129`` registries on the 3.10-3.13 unit legs of PR #1560.
:mod:`tests.project.test_conventions_doc_claims` counted both generations, and
its registry count came out wrong on the 3.10-3.13 unit legs of PR #1560.

Failure here is not an error. A checkout without the runtime dependencies
installed cannot import the package at all -- which is a supported way to run
Expand Down Expand Up @@ -244,9 +244,9 @@ def restore_module_table() -> 'Iterator[None]':
:class:`~typing.Generic` stayed bound after the test that installed it, and
the next test to import :mod:`pcapkit` died on ``TypeError: type
'ProtocolBase' is not subscriptable`` at
``pcapkit/protocols/misc/pcap/frame.py:69`` -- twelve tests in
``pcapkit/protocols/misc/pcap/frame.py:69`` -- a run of tests in
:mod:`tests.project`, and only when the polluting file happened to be
collected first. Three separate files turned out to leak this way, one of
collected first. Several files turned out to leak this way, one of
which -- :mod:`tests.cli.test_main` -- did not import :mod:`tests._support`
at all.

Expand All @@ -259,8 +259,8 @@ def restore_module_table() -> 'Iterator[None]':
that will be written, which is the difference between the failure being
unlikely and being impossible.

Two of the three were fixed at their call sites when this guard landed, and
the third was not: :mod:`tests.cli.test_main` kept its own purge loop, which
The others were fixed at their call sites when this guard landed, but
:mod:`tests.cli.test_main` was not: it kept its own purge loop, which
purges and restores nothing, and this fixture went on quietly healing it for
every test. That is issue #688 -- it took an audit rather than a failing run
to find, because a guard that repairs a leak also hides it. It is now fixed
Expand Down
Loading
Loading