[akol1]

What this changes

Adds three files under tools/qwen27/. qwen38_27b_oracle.json pins four single-host receipts (temperature 0, seed 1234, no drafter, one stream) with the checkpoint revision, and records for chat-256 the one value a pair is accepted at and why it parts from the oracle. pair_accept.py run sends each fixture twice, requires identical replies and the oracle's token_sha, accepts that recorded value for its case, and prints 3/4 … 0 unexpected on a pair; pair_accept.py warm holds a two-turn conversation and requires the second turn to report cached prompt tokens. pair_supervise.sh up|down|status|watch starts rank 1 first and rank 0 second, refuses two installs whose binaries differ unless ALLOW_MISMATCH=1, waits for both /health ports, and in watch stops and restarts both ranks whenever either stops answering. Every location is environment overridable, and the script prints that environment from its usage text.

Refs #1236, the tracker for tensor-parallel ranks: these are the harness and receipts this stack was accepted with, and they do not cover the mixed Metal/CUDA case that issue is about. Refs #808: it asked for two-Mac prompt-splitting speed with drafts and images; this stack splits rounds and refuses both, so nothing here is a retry of that report. Stacked on #1432, #1433, #1434, #1435; the diff against main carries them too, so review this pull request's own commit, 53a0d278.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: 8c36359d plus this commit (53a0d278); no MLX or PyTorch; macOS 27.0.1 on two Apple M4 Pro Macs (Mac16,11), Metal only (applegpu_g16, no tensor units), Apple metal version 32023.921; checkpoint TensorFold/Qwen3.8-27B-MLX-4bit at revision 22d8d538154e0e5b6f8dcb1fcc74a60a0798104c, 4-bit affine with group 64, on both Macs; the two Macs joined by a Thunderbolt bridge. The three files are not compiled and need no checkpoint to read; they were exercised against sha256 2dda0e50fad3ca10487c215820f5cd9acee83d3914ab474955659333d9e81bdf, which is this stack's native code (later commits add tests, tools and docs only).
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: pair_accept.py run --repeat 2 against one Mac matches all four oracle fixtures on both repeats, chat-256 at a06fab348fbe; against the pair it matches three, twice each, and reports KNOWN chat-256 token_sha=58daf9743077 is the pair value recorded for this case, not the oracle's a06fab348fbe, ending pair_accept: 3/4 cases match the oracle, 1 recorded as different, 0 unexpected. pair_accept.py warm on the one-Mac arm prints warm: turn 1 prefill marked 0; turn 2 cached=158 and PASS warm: the next turn resumed 158 cached prompt tokens. The supervisor confirmed matching installs (binaries match: 2dda0e50fad3), held rank 0's port until the handshake (rank 0's port refused 2 probes while the pair handshake completed), survived kill -9 on rank 0 (restarting both (restart 1), same shas afterwards), and on TERM stopped both ranks with no serve process or listener left on either host.
• Decode speed before and after, from tools/bench_openai.py: n/a — these tools time nothing. The pair/lifecycle pull request carries the done … tok/s= lines from the same binary: 16.8 to 17.1 tok/s for the pair against 12.3 to 13.6 tok/s for one Mac.
• Prompt speed before and after: n/a — the fixtures' prompts are 14 and 31 tokens, and warm adds a 187-token conversation over two turns without timing a cold long prompt; the pair/lifecycle pull request owns those numbers.
• Tests run, with pass, fail and skip counts:zig build test test-golden at this tree: 7116 passed, 45 skipped, 0 failed of 7161, with 535 of 535 build steps; the 45 skips are the image-codec and pixel cases that need libjpeg-turbo 3.1.4.1 or libwebp 1.6.0, plus SKIP qwen27-tp-check: no --model DIR. The three files add no CI test; their own runs at this tree: pair_accept.py run --repeat 2 sent 8 requests per arm — one Mac 4 of 4 cases matched on both repeats (0 failed), the pair 3 matched and 1 recorded and 0 unexpected (0 failed) — pair_accept.py warm passed once with 158 cached tokens, and python3 -m py_compile tools/qwen27/pair_accept.py plus sh -n tools/qwen27/pair_supervise.sh were clean.
• Not run, and why: the supervisor's SSH path ran between two Macs over one Thunderbolt bridge and no other transport; CUDA, HIP/RDNA, M1-M3/M5 and Windows are absent here; pair_accept.py needs a serving pair, so CI covers none of these three files; and the oracle is four short fixtures, not a corpus, so it is a regression pin rather than a release-wide benchmark.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. — these tools serve no tokens; but the oracle pins "drafter": false because the pair refuses a drafter, so they cannot check a drafted reply and the drafted path keeps its existing single-host tests.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. — the pair refuses a prompt cache and a second stream by name, so warm checks resume on the one-Mac arm only and concurrency is not exercised; those refusals are the lifecycle pull request's receipt.
• No precision traded for speed. If the bits change, the description says which sums change and why. — no arithmetic here; the oracle records the pair's one differing value and its two contenders instead of widening a tolerance.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. — no kernel in this commit; the pair's four are registered in the first pull request.
• Prompt processing is no slower than the last release. — n/a: tooling, no prompt path touched; no cold long-prompt timing was taken.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. — ran Metal on two M4 Pro Macs over Thunderbolt; no CUDA, no HIP, no M1-M3/M5, no Windows (see Not run).
• New tests fail before the change, pass after it, and skip cleanly without their dependency. — no CI test is added; the confidence is the runs below, and the files do not exist before this commit.
• Comments and docstrings are one line. No measurements or history in the source. — the Python modules follow tools/bench_openai.py's shape, and the supervisor opens with two comment lines and prints its environment from its usage text instead of a header block; the lean check reports 428 problems, 428 on the base, 0 new.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines. — the files name no machine, user path, address or serial; the peer, model, settings and binaries all arrive as environment variables, and the only fixed address is the usual 127.0.0.1 loopback default.

[ashhart]

• The oracle. It says "A paired run must reproduce every token_sha here", but pair_known then accepts 58daf9743077 for chat-256. Please remove the exemption.
• stop_peer and down kill every process matching tensorfold-native serve.*--speed-up on both Macs, which can include someone else's GLM or Flash Next pair. Kill only the pids the script started, and check them.
• The default SSH_OPTS turns off StrictHostKeyChecking on a script that starts binaries remotely. Please keep the normal host-key check.