Test Harness: interactive RC1 validation server for Sendspin client apps #84

Build sendspin-validate, an interactive validation server that Sendspin client developers run on Linux or macOS to check their app against spec 1.0.0-rc1. It lives in Sendspin/conformance as a second console script, and runs with no clone through uvx.

What it does

• Connection modes: a client can connect in either direction. The validator advertises itself and browses for clients over mDNS, and also accepts --connect.
• Unpaired and paired:
  • It checks unpaired access in both its on and off states.
  • It walks through every pairing method the client advertises: pairing_psk, the dynamic code and the static code.
  • After pairing it checks reconnect, credential mismatch, unpair and persistence.
• Machine checks: it validates every message the client sends, and every response it owes, against the spec. Each failure cites the spec file and line.
• Prompts: it tells the user what to do in their app, for example "paste the pairing token" or "open the pairing window".
• Synthesized audio: it generates test audio itself, so there is no licensing question. It plays a clip in every format the client advertises. The human answers randomized challenge questions such as how many beeps they heard and which side, so a check can't be rubber-stamped.
• Display checks: it covers the artwork, metadata, controller, color, visualizer and source roles, again with randomized challenges.
• Report: a report.json and a report.html, which separate machine-checked results from results a human attested to.

Decisions

• Plain CLI for v1. The prompts sit behind an interface, so a UI can be added later.
• Pairing input is pasted text only.
• Fault injection is opt-in.
• v1 targets RC1 only.
• The validator is self-contained. It bundles a patched aiosendspin until upstream releases the hooks it needs.

Work

• feat(conformance): bundle aiosendspin 10.0.2 for the client validator conformance#155 Bundle aiosendspin 10.0.2 for the client validator
• ci: run the matrix only when harness or site code changes conformance#156 Run the matrix only when harness or site code changes, so validator PRs only run the light CI
• build: commit uv.lock and install with uv sync --locked conformance#158 Commit uv.lock so installs resolve reproducibly
• feat(conformance): patch a frame observer into the bundled aiosendspin conformance#159 Patch a frame observer into the bundled aiosendspin
• feat(validator): add the sendspin-validate skeleton with handshake and hello checks conformance#163 Phase 1 skeleton: the CLI, both connection modes, the recorder, the report, and the handshake and hello checks. It adds the 0002 non-compliance observer patch.
• feat(validator): add the unpaired, pairing_psk and post-pairing suites conformance#164 Phase 2a: the unpaired, pairing_psk and post-pairing checks
• Phase 2b: the dynamic and static pairing codes, cancel and timeout
• Phase 3: the audio format sweep and the human playback checks
• Phase 4: the controller, metadata, artwork, color, visualizer and source suites
• Phase 5: opt-in fault injection, and a PyPI release
• Offer the bundled aiosendspin patches upstream to Sendspin/aiosendspin: 0001 frame observer and 0002 non-compliance observer so far

Related: #70