[swbiggart]

Update: The branch has been updated following review, and the existing hardware and host receipts have been updated. The original description and measurements below are retained for reference; the current code and linked receipts supersede them.

Problem and change

We do not have a network switch: our four GB10 hosts form a physical ring with dual-PCI-domain RoCE links between adjacent ranks. This was the fastest ring implementation we measured. We are not claiming a direct performance comparison against the target branch or a switched/full-mesh setup.

Add one default-off opt-in, TF_DS_NEIGHBOR_RDMA=1, that selects the optimized paths together:

• Pair gathers use xor1 only.
• Full/Markov gathers use xor1 then xor3, preserving global rank order.
• Equal-width quarter gathers collect directly into the final layout.
• Parity diagonal exchanges use remote-only xor1 then xor3 to reach rank xor2.

These paths fit the available links and remove unnecessary stages/copies. They reuse existing RDMA kernels and retain the 512 KiB cutoff. Larger diagonal traffic uses two-edge NCCL forwarding; other size/layout fallbacks remain. No arithmetic or drafting-policy changes and no separate experimental switches.

Unset/0 retains upstream inference paths and creates no neighbor RDMA resources. One TP4 startup byte all-gather runs in either mode to reject mixed enablement. All ranks must use the same binary. The opt-in requires two adjacent-edge rail lists and GID 3; it cannot be combined with TF_RDMA_DEVICES. Local failures propagate to both stages; this does not add a distributed recovery guarantee.

Base and tested code

Target: bertholomus/TensorFold:deepseek-v41-zig-tp4 at 59402086381a42c74d53053443224eec2790b4c5.

Hardware-tested engine commit: a9225d72ec63b19867c72eba1cca48a49648805c. Later commits only add qualification receipts/documentation. This integrates the DeepSeek release branch, not canonical ashhart/TensorFold:main's newer transport layer. No unrelated model/server/forced-tool patches are included.

Tests on the consolidated implementation

The exact commit was built for Linux/aarch64 with Zig 0.17.0 and CUDA fatbins, then tested on all four hosts. A weight-free C-ABI adapter and build targets were added externally; all 423 tracked production files under zig/src were verified unchanged. Native serving used the unmodified production code.

• 56 rank/shape cases passed: independent byte oracle, guards, changing-input CUDA graph replay, mixed-operation graphs, cutoff and alignment fallbacks. Four-rank startup agreement ran successfully.
• 9 model replies matched exact token IDs and UTF-8: three prompts, serial/drafted/concurrent, 64 tokens each.
• 24 timed burst replies matched UTF-8 across six repetitions of four 256-token requests.
• 11 approximately 8K-prompt replies passed: 5/5 burst/solo cases matched token IDs, drafting matched serial, all four recall markers were found, and cached replays matched cold runs. Solo replays reported 6144 cached tokens.
• Cancellation passed: disconnect an active generation, then compare fresh serial/drafted replies exactly.
• 90-second four-stream run: 62 requests, no worker errors.
• Fresh host checks: 14/14 focused tests passed. Full host/golden command retains the inherited macOS SO.BUSY_POLL compile blocker; corpus remains 312/319 equal, seven known differences, zero unexpected. This full command is not reported as a pass. Existing lean findings remain documented separately.

Measurements on this commit

DeepSeek-V4.1-Flash-1017 EXL3 a2f91e27…, DSpark, four GB10 hosts, patched NCCL 2.30.7, --context 1048576 --parallel 16 --max-tokens 32768, graphs enabled, served-depth policy solo. The context argument is not full-context qualification.

Workload	Aggregate tokens/s	Detail
Four-stream set-a bursts	132.63	One warmup, six measured runs; range 131.94–133.31
Set-a steady admission	148.83	30 seconds, 19 requests
Set-b sustained	169.77	90 seconds, 62 requests; TTFT p50 0.159 s / p95 0.283 s

All use greedy decoding and 256-token maximum outputs. Set-a and set-b are different workloads; their rates are not directly comparable. Complete diagonal-operation latencies were 17.81 / 27.77 / 42.89 / 87.10 µs for 8.75 / 52.5 / 140 / 420 KiB, respectively (slowest-rank means).

The hardware receipt includes machine-readable results, rank/shape results, request bodies, binary digests, runtime versions and qualification limits. See also the host receipt and configuration contract.

Acknowledgments

This work was informed by @christopherowen’s DGX Spark networking experiments and @therealarnold666’s GLM bidirectional owner-ring experiments. These projects informed the design investigation. The implementation adapts TensorFold’s existing RDMA transport and kernels to DeepSeek’s specific exchanges on a four-host physical ring; the existing transport’s provenance is recorded in ATTRIBUTION.md.

[bertholomus]

Thank you @swbiggart for this, and for the tests and the hardware receipts. The owner will decide on it.

[christopherowen]

what an honor it would be to get credited in tensorfold!

[bertholomus]

Thank you again @swbiggart, and thank you @christopherowen and @therealarnold666 for the experiments this builds on. The v0.2.0 engine is now on deepseek-v41-zig-tp4 (b65c8b9), so this PR has two small conflicts (README.md and zig/build/cuda.zig). Could you please rebase onto it? Three small asks while you are there: (1) carry the neighbor mode in the engine start-up settings check (prompt2d.zig Settings, the same way TF_DS_IDX_SPLIT is) instead of the extra all-gather, so a run with the mode unset exchanges nothing new; (2) make the ring GID index a setting (it is fixed at 3; our rings use 5); (3) keep it off by default. We will merge after the host tests and a node build pass.

[swbiggart]

Done, let me know if you need anything else. Thanks for putting this together and your consideration of my changes.

[bertholomus]

Merged, thank you @swbiggart! All three asks are in: the mode is part of the start-up settings check, the ring GID index is a setting (TF_DS_NEIGHBOR_GID_INDEX, default 5), and the feature stays off unless TF_DS_NEIGHBOR_RDMA=1, so a run without it takes the old paths. Here the host tests (your 10 new ones too), the server tests and a node build pass, also together with #31. Thanks again to @christopherowen and @therealarnold666 for the experiments it builds on!