Skip to content

Capture a stack when tests/ui wedges, instead of killing the run blind - #48

Merged
sensitron merged 1 commit into
mainfrom
diag/issue-39-hang-diagnostics
Sep 5, 2026
Merged

sensitron merged 1 commit into
mainfrom
diag/issue-39-hang-diagnostics

Conversation

@sensitron

Copy link
Copy Markdown
Collaborator

tests/ui has occasionally wedged instead of failing — no output, no crash, no
progress — and nobody had ever captured a stack from a hung run. The three prior
sightings were killed by a 120 s outer timeout before any diagnostic tool was
armed, so each one was recorded as "a hang" and abandoned.

This adds the tool that was missing: tests/ui/conftest.py arms
faulthandler.dump_traceback_later() around every test item when
MATTELOOP_TEST_HANG_DUMP_SECONDS is set, so a wedged run writes a thread dump
instead of dying silently. It mirrors pytest's own built-in faulthandler_timeout
plugin, but is switched by an environment variable rather than an ini option, and
writes to a plain file rather than the duped stderr fd — so a measurement harness
can turn it on for one run without touching pyproject.toml, and the dump
survives however the caller redirects the subprocess's own output.

Off by default. The normal suite is unaffected: uv run pytest -q gives
1779 passed, 4 skipped, matching the baseline on main exactly.

It already paid for itself

Run overnight on Linux (QT_QPA_PLATFORM=offscreen), 122 runs over two hours,
the switch found 7 hangs, all with an identical stack, all parked in
test_preview_controller.py::test_cancel_keeps_modal_dialog_open_until_the_safe_checkpoint
— inside a qtbot.waitUntil whose own 5 s timeout never fired, because the Qt
event loop it pumps never came back. Plus two segfaults in a structurally
similar cancel test and one clean QThread already deleted traceback in
SourceController.shutdown().

Full findings, stacks and the platform caveat (measured on Linux, originally
seen on Windows) are posted on #39.

Scope

Deliberately just the switch. The measurement loop script and the 5.5 MB of raw
dumps stayed on a throwaway branch — they were scaffolding for one night, not
something the repository should carry. The switch is the part that will help the
next time this happens.

Refs #39 and not Closes: this makes the hang observable, it does not fix it.

Verification

uv run ruff check . · uv run mypy src · uv run python scripts/check_guardrails.py ·
QT_QPA_PLATFORM=offscreen uv run pytest -q — all green on Linux.

🤖 Generated with Claude Code

Issue #39: tests/ui occasionally hangs during a run -- no output, no
crash, no progress -- instead of failing outright, and nobody has ever
captured a stack from a hung run. The three prior sightings were killed
by a 120s outer timeout before any diagnostic tool was even armed.

tests/ui/conftest.py now arms faulthandler.dump_traceback_later() around
every test item when MATTELOOP_TEST_HANG_DUMP_SECONDS is set, the same
technique matteloop.logs already uses for native crashes in the shipped
app. It mirrors pytest's own built-in faulthandler_timeout plugin
(_pytest/faulthandler.py: pytest_configure/pytest_runtest_protocol), but
is switched by an environment variable instead of an ini option so a
single measurement run can turn it on without touching pyproject.toml,
and it writes to a plain file instead of the duped stderr fd so the dump
survives however a caller redirects the subprocess's own stdout/stderr.

The dump file is opened once for the whole pytest session and kept open,
not reopened per test: closing it right after cancelling a just-armed
timer reproducibly segfaulted the interpreter here when the dump raced
the close (reproduced at an artificially tiny 0.01s threshold). Matches
the built-in plugin's own lifetime (dup'd once in pytest_configure,
closed once in pytest_unconfigure).

Off by default -- the normal suite runs unaffected, verified by a full
green `uv run pytest -q` (1779 passed, 4 skipped, matching the existing
baseline) with the switch untouched.

Refs #39

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5KtfjmhMjKKKPyE8brrHt
@sensitron
sensitron merged commit c51b33f into main Sep 5, 2026
11 checks passed
@sensitron
sensitron deleted the diag/issue-39-hang-diagnostics branch September 5, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant