Skip to content

fix(tests): finish the Windows portability tail — path separators, signal handlers, skills paths - #77

Open
mjerris wants to merge 7 commits into
mainfrom
fix/win-test-portability-tail
Open

fix(tests): finish the Windows portability tail — path separators, signal handlers, skills paths#77
mjerris wants to merge 7 commits into
mainfrom
fix/win-test-portability-tail

Conversation

@mjerris

@mjerris mjerris commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Finishes the Windows portability work: the 15 TEST failures left after PR #76 took the count from 36 → 15. Measured from Multi-OS run 30261304144 (windows-latest).

Three defect classes. One is a product bug, not a test bug.

1. PRODUCT — RelayClient.run() was broken for every Windows user (7 failures)

_run_forever() called loop.add_signal_handler(SIGINT, ...) unguarded, as its first statement:

signalwire\signalwire\relay\client.py:684: in _run_forever
    loop.add_signal_handler(
...\asyncio\events.py:582: in add_signal_handler
    raise NotImplementedError
E   NotImplementedError

Loop-level signal handling is a Unix-only asyncio capability — both Windows event loops raise NotImplementedError unconditionally. So the exception escaped before connect() was ever reached: RelayClient.run() could not establish a RELAY connection on Windows at all. This is a genuine platform limitation of asyncio, but calling it unguarded from product code is our defect — a Windows user hits it, not just the test harness.

Fixed with a NotImplementedError guard that degrades gracefully: Ctrl+C still stops the client (asyncio.run() surfaces KeyboardInterrupt, which run() already suppresses); only the graceful _shutdown() handshake is lost. Also replaced the odd __import__("signal") inline with a module-level import.

Per the bar set by #76 — where it skipped, it also added platform-independent coverage — this ships a regression test that forces the exact Windows condition on every OS (patching the loop method to raise what Windows raises) instead of a skipif(win32) that would leave Windows uncovered. Proven on macOS: against the unfixed product the new test fails with the same NotImplementedError at client.py:684 as the Windows traceback.

The 7th relay failure was a different mechanism (Request timeout for signalwire.connect). The ping-loop test patched _EXECUTE_TIMEOUT=0.01 around client.connect(), putting the auth round-trip under a 10 ms deadline. The handshake needs several event-loop turns (the mock queues the reply, then _recv_task must be scheduled to drain it), and 10 ms is at/below Windows' asyncio timer granularity (~15.6 ms clock tick), so connect() could expire before the loop ever ran the recv task. Scoped the patch to the pings the test is actually about. Reproduced on POSIX with a deadline sweep — the debug log shows Response for unknown/expired request, i.e. the reply does arrive but the deadline already passed:

_EXECUTE_TIMEOUT=1.0      failures= 0/20  connected OK
_EXECUTE_TIMEOUT=0.01     failures= 0/20  connected OK
_EXECUTE_TIMEOUT=0.0      failures=20/20  RELAY error -1: Request timeout for signalwire.connect

2. TESTS — POSIX-separator expectations (3 failures)

test_schema_utils (2) and test_registry (1) compared product output against hardcoded POSIX literals. The product is separator-correct in all three cases — it returns str(Path(...)), composes with os.path.join(), and splits on os.pathsep. Only the expectations were POSIX-only, so these are test bugs, and a skip would be illegitimate here.

Each expectation is now built the way the product builds the value (compare Path objects / computed joins). Verified under Windows path semantics (ntpath / PureWindowsPath) from POSIX — the old literals match only on POSIX, the new expectations match on both, so Windows keeps real coverage:

== manual_search: product composes os.path.join(os.getcwd(), 'schema.json')
  posix    product='/current/schema.json'   old_literal_match=True  new_computed_match=True
  windows  product='\current\schema.json'   old_literal_match=False new_computed_match=True

3. TESTS — claude_skills paths (5 failures)

  • WinError 267 (invalid directory) — the test passed Path("/tmp") as the subprocess cwd. /tmp doesn't exist on Windows, so the spawn failed before the timeout could fire (assert '[command timed out:' in '[command error: sleep 10]'). Switched to pytest's tmp_path. The command was also sleep 10, which isn't a Windows shell builtin and exits immediately, so the timeout contract wouldn't be exercised even with a valid cwd — replaced with a portable python -c sleep. The test now takes 1.01 s, confirming it genuinely blocks and hits the timeout rather than passing on a spawn error.
  • 8.3 short-path mismatches (3)setup() canonicalizes skills_path via .resolve(), which expands the Windows 8.3 temp dir to its long form (RUNNER~1runneradmin). The tests compared an unresolved tempfile path to resolved product output. Both sides now resolve. Correct product behavior; test-side bug, as the short-path shape suggested.
  • Removed all 15 hardcoded /tmp uses from this file per the standing project rule; apart from the cwd above they were inert placeholders (_make_skill never calls setup(), so skills_path was unvalidated).

Disposition summary

No test was blanket-skipped, and no skipif(win32) was added. Every one of the 15 is either a product fix (7) or an expectation corrected to match platform-correct product behavior (8).

Verification

bash scripts/run-ci.shexit 0, all gates PASS:

[TEST] python -m pytest tests/unit/ (STRICT-MOCKS: MOCK_RELAY_STRICT=1) ... PASS
[FMT] ruff format (local: apply; CI: --check) ... PASS
[LINT] ruff check zero findings ... PASS
[TYPECHECK] mypy zero findings ... PASS
[DRIFT] python_signatures.json unchanged after regen ... PASS
[SPEC-PARITY] implemented routes == canonical spec ... PASS
[REST-COVERAGE] every implemented REST route covered success+error ... PASS

tests/unit5710 passed, 3 skipped, 0 failed (5699 before; +1 new regression test). Working tree clean after the FMT gate, so no auto-applied formatting is left uncommitted.

Windows itself still needs CI confirmation: Multi-OS runs on cron + workflow_dispatch only, so it does not run on this PR. A dispatch against this branch is noted in the thread.

Sequencing

Branched from origin/main (d782db8); the diff is 5 files and contains no content from #75 or #76.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GFKJhLvfV8yGrASwqxdgaf

mjerris and others added 7 commits July 27, 2026 01:30
The multi-OS PACKAGE-SMOKE step invoked bare `python3`, which on the macOS runner
does NOT resolve to the interpreter actions/setup-python provisioned. Nightly run
30238061313, job macos-latest:

    [FAIL] build: exit 1
        /Library/Frameworks/Python.framework/Versions/3.12/bin/python3:
        No module named build

Mechanism. `setup-python@v6` with python-version "3.12" provisions
/Users/runner/hostedtoolcache/Python/3.12.10/arm64 (the step's own log confirms
pythonLocation), and the earlier `pip install` step installed `build` THERE. But
the macOS image ships a pre-installed framework Python whose bin dir sits ahead of
the toolcache for the name `python3`, so bare `python3` selected
/Library/Frameworks/Python.framework/Versions/3.12/bin/python3 — a DIFFERENT
interpreter, without `build`.

The same run is the proof: the TEST step immediately above uses `python -m pytest`
(the setup-python shim) and passed, as did every `pip install`. Only the one step
spelled `python3` missed.

package_smoke.py is NOT at fault — it correctly uses sys.executable, so it
faithfully used whichever interpreter this line handed it, and it reported that
path verbatim in the error. Fixed by invoking `python` (setup-python's shim).

NOT fixed by `pip install build`: installing into the wrong interpreter would mask
the resolution bug rather than repair it, and would leave the step running an
interpreter nobody selected.

Sweep of the sibling workflows that call setup-python: doc-audit.yml had the only
other bare-`python3` `run:` lines (2). They are ubuntu-only, where setup-python
front-loads its dir so both names currently resolve to the toolcache — latent, not
live — but switched to the shim for consistency so the trap cannot activate if that
job ever gains a macOS runner. nightly.yml and live-smoke.yml call setup-python but
contain no bare `python3`, and are ubuntu-only.

Verified: `grep -n '\bpython3\b' .github/workflows/*.yml` now matches only
explanatory comments, no executable line. actionlint on both changed files reports
the same 6 pre-existing shellcheck info/style findings as before the change (all at
doc-audit.yml's untouched Summary step) — no new findings.

Known-latent, deliberately NOT changed here (needs an owner call): scripts/run-ci.sh
uses bare `python3` in 51 places. Both workflows that invoke it are ubuntu-only, and
that script also runs on developer machines where `python3` is the correct name and
`python` may not exist — so a blanket rename is a behavior change beyond this fix.
Corrects my own previous commit on this branch, which mis-attributed the failure to
interpreter resolution. Dispatching the workflow disproved that fix: with `python`
instead of `python3` the macOS job failed IDENTICALLY, one word different —

    /Library/Frameworks/Python.framework/Versions/3.12/bin/python: No module named build

i.e. the SAME framework interpreter, reached via the other name. So on the
macOS/arm64 runner BOTH names resolve to the pre-installed framework Python;
setup-python does not win PATH there at all.

The real root cause, from the same log: `pip` is that framework Python's pip —
every dependency reports installing to
`/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages`.
The job is therefore CONSISTENTLY one interpreter, and there was never a mismatch
to fix. `build` is simply not installed, because **it is not a declared dependency
anywhere** — not in requirements-dev.txt, not in requirements.txt, not in
pyproject.toml. The gate relied on the runner image happening to ship it.

AGENT_RULES §7: a tool a gate needs is DECLARED, not assumed present. So:
requirements-dev.txt gains `build>=1.0.0`, and the step above already installs that
file. This is not "pip install build into the wrong interpreter" (which the brief
rightly forbids as masking a resolution bug) — there is no wrong interpreter here,
and declaring a real, undeclared dev dependency is the fix rather than the mask.

Note this gate has never passed for python on any OS: multi-os.yml is the ONLY
place PACKAGE-SMOKE runs for this port (nightly.yml and scripts/run-ci.sh do not
invoke it), so the "ubuntu ships build" path was never exercised either.

`python` (not `python3`) is KEPT, on its own merits: it is the same name `pip`
above pairs with, so the step provably runs the interpreter the deps went into, and
it stays correct if the image's PATH precedence ever changes. The workflow comment
is rewritten to state this true mechanism instead of the interpreter-mismatch story.

The doc-audit.yml python3→python change from the previous commit also stands — that
job is ubuntu-only and green either way; the shim is the consistent choice.

Windows remains red on this workflow for an unrelated, separately-assigned reason
(RED 3: four test-portability defects in the TEST step — WinError 32 on an unclosed
sqlite temp file, a hardcoded "/tmp/custom" assertion, a POSIX 0o755 mode check, and
cp1252 UnicodeEncodeErrors). That step fails before PACKAGE-SMOKE runs, so the
Windows job cannot confirm this fix either way.
…rmission bits, UTF-8 encoding

Four distinct cross-platform defects, all measured from nightly Multi-OS run
30238061313 (job windows-latest, step TEST: 36 failed / 2 errors / 5671 passed).
Two turned out to be product bugs, not test bugs.

1. sqlite handle outlives the temp file (PermissionError [WinError 32])
   PRODUCT BUG. search/index_builder.py validate_index() closed its connection
   only on the success path; the "Missing tables" early return and the except
   both leaked it. Windows refuses to delete a file with a live handle, so the
   fixture teardown's os.remove() raised. Fixed with contextlib.closing (NOT
   `with sqlite3.connect(...)` — a Connection context manager commits but does
   not close). Audited all 14 sqlite3.connect sites in search/ and found two
   more unguarded on their error paths: migration.get_index_info() and
   search_service._get_model_name(). Also fixed the test-side leak in
   test_search_engine.py, where NamedTemporaryFile(delete=False) held its own
   handle open while os.unlink ran inside the `with` block.

2. Hardcoded POSIX path assertion
   str(Path) renders with the platform separator, so `== "/tmp/custom"` can
   never hold on Windows (it yields "\tmp\custom"). Now compares Path objects.
   Also removes the /tmp usage itself per the project rule (tmp_path instead).
   Same fix in test_init_project.py::test_main_custom_dir, which failed the
   same way via `'/tmp/custom' in 'D:\tmp\custom\testproject'`.

3. POSIX permission bits
   Windows has no execute bit, so st_mode never carries 0o755. The two
   observable-mode assertions are skipif(win32) with the reason recorded, and
   the contract they were checking is now ALSO asserted platform-independently
   (that chmod 0o755 is requested, and only for deploy.sh). POSIX coverage is
   unchanged — nothing was deleted or weakened.

4. UnicodeEncodeError: 'charmap'
   PRODUCT BUG. cli/dokku.py:1964 wrote generated files with write_text() and
   no encoding, i.e. the platform default (cp1252 on Windows). The templates
   embed box-drawing rules (U+2500/U+2550, 59-char runs — matching the log's
   "position 130-188") and arrows, which cp1252 cannot represent. This caused 8
   direct failures plus 4 more surfacing as generate() returning False
   ("Failed to generate project: 'charmap' codec can't encode..."). Fixed at
   _write_file plus the two app.json reads. cli/init_project.py had the same
   latent defect — 98 cp1252-hostile characters across 29 unguarded write_text
   sites — fixed there too before it bites.

Proven on POSIX (not merely "looks right"):
- Defect 1: 3 new tests assert the platform-independent invariant (every
  connection opened is closed, via a connect spy). Verified they FAIL against
  the unfixed product code and pass with it.
- Defect 4: reproduced the Windows failure class on macOS with
  `LC_ALL=C python -X utf8=0`, which yields
  "'ascii' codec can't encode characters in position 2-80" — the same position
  range as CI's charmap error. The 3 new UTF-8 round-trip tests fail without
  the fix; with it, all 210 cli tests pass even under that hostile locale.
- Defects 2 and 3 are structural (Path comparison / platform skip) and remain
  pending real confirmation on the Windows runner.

Full local gate run: `bash scripts/run-ci.sh` → CI PASS, exit 0 (37 gates).
Unit suite 5621 passed / 100 skipped on macOS.

Note: the Windows TEST step failing is what kept PACKAGE-SMOKE from running at
all on that job (TEST: failure -> PACKAGE-SMOKE: skipped). If TEST now passes,
PACKAGE-SMOKE will execute on Windows for the first time and may surface
unrelated failures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFKJhLvfV8yGrASwqxdgaf
The new test_deploy_script_round_trips_as_utf8 asserted
read_bytes().decode('utf-8') == read_text(encoding='utf-8'). Those two views
legitimately differ on Windows: text-mode writes translate \n -> \r\n and
read_text translates it back (universal newlines), so the assertion compared
raw CRLF against normalized LF and failed on the runner -- while the encoding
fix it was guarding was working correctly (every U+2550/U+2192/U+2705/U+1F310
round-tripped intact).

Line endings are not what the test is about. It now compares against the source
DEPLOY_SCRIPT_TEMPLATE line-by-line and asserts the non-ASCII character set
survives byte-for-byte, which is the actual regression being guarded. Still
verified to fail against the unfixed product code (all 3 tests in the class fail
under LC_ALL=C -X utf8=0 without the encoding fix).

Caught by Multi-OS run 30260346853 (windows-latest), which this branch
dispatched -- Windows TEST went 36 failed -> 16 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFKJhLvfV8yGrASwqxdgaf
…gnal handlers, skills paths

Closes the 15 Windows TEST failures left by PR #76 (which took the count
36 -> 15), measured from Multi-OS run 30261304144 / windows-latest.

One of the three defect classes is a PRODUCT bug, not a test bug.

1. PRODUCT — RelayClient.run() was broken for every Windows user (7 tests)

   _run_forever() called loop.add_signal_handler(SIGINT, ...) unguarded on its
   FIRST statement. Loop-level signal handling is a Unix-only asyncio
   capability: both Windows event loops raise NotImplementedError
   unconditionally, so the exception escaped before connect() was ever reached
   — RelayClient.run() could not establish a RELAY connection on Windows at
   all. Guarded with a NotImplementedError fallback that degrades to
   KeyboardInterrupt-driven shutdown (Ctrl+C still stops the client; only the
   graceful _shutdown() handshake is lost). Also replaced the
   __import__("signal") inline with a module-level import.

   Covered by a new platform-independent regression test that forces the exact
   Windows condition (patching the loop method to raise what Windows raises)
   rather than skipping on win32, so the contract is exercised on every OS.
   Verified to FAIL against the unfixed product on macOS with the same
   NotImplementedError at client.py:684 as the Windows traceback.

   The 7th relay failure was a separate mechanism: the ping-loop test patched
   _EXECUTE_TIMEOUT=0.01 around client.connect(), putting the auth round-trip
   under a 10ms deadline. The handshake needs several event-loop turns, and
   10ms is at/below the Windows asyncio timer granularity (~15.6ms clock tick),
   so connect() could time out before the loop ran the recv task. Scoped the
   patch to the pings the test is actually about. A deadline sweep on POSIX
   reproduces the identical "Request timeout for signalwire.connect" error
   deterministically (20/20) once the deadline drops below the turns required.

2. TESTS — POSIX-separator expectations (3 tests)

   test_schema_utils (2) and test_registry (1) compared product output to
   hardcoded POSIX literals. The product is separator-correct in all three
   cases: it returns str(Path(...)), composes with os.path.join(), and splits
   on os.pathsep. Build the expectation the same way the product builds the
   value (compare Path objects / computed joins) instead of a POSIX-only
   spelling. Confirmed under Windows path semantics (ntpath/PureWindowsPath) on
   POSIX: the old literals match only on POSIX, the new expectations match on
   both — so Windows keeps real coverage rather than losing it to a skip.

3. TESTS — claude_skills paths (5 tests)

   - WinError 267 ("directory name is invalid"): the test passed Path("/tmp") as
     the subprocess cwd. /tmp does not exist on Windows, so the spawn failed
     before the timeout could fire. Switched to pytest tmp_path. The command was
     also `sleep 10`, which is not a Windows shell builtin and exits
     immediately; replaced with a portable python -c sleep so the timeout path
     is genuinely exercised (test now takes ~1.01s, i.e. it really blocks).
   - 8.3 short-path mismatches (3 tests): setup() canonicalizes skills_path via
     .resolve(), which expands the Windows 8.3 temp dir to its long form
     (RUNNER~1 -> runneradmin). The tests compared an unresolved tempfile path
     to resolved product output. Resolve both sides.
   - Removed all 15 hardcoded /tmp uses from this file (a standing project rule
     bans /tmp outright); they were inert placeholders except where noted above.

No test was blanket-skipped: every disposition is either a product fix or an
expectation corrected to match platform-correct product behavior.

Verification: bash scripts/run-ci.sh exit 0, all gates PASS (TEST, FMT, LINT,
TYPECHECK, DRIFT, SPEC-PARITY, REST-COVERAGE, ...); tests/unit 5710 passed,
3 skipped, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFKJhLvfV8yGrASwqxdgaf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant