Repository navigation
waiting policy: no sched_yield phase, a short park delay between calls, and a throttled spin when oversubscribed #9
Description
Problem
A waiting member, and the leader at a join or barrier, use backoff (src/awake.rs:60): 16,384 spin_loop() calls, then thread::yield_now() on every poll until the park delay ends. The delay is idle_spin, 200 ms by default.
This causes two problems.
- The scheduler sees every CPU as busy. A thread in the yield phase is always runnable. If two team threads end up on one CPU, nothing moves one of them away.
- The primerose probe shows this pattern in its slow runs: two team threads on one CPU with tens of thousands of involuntary context switches, and 0.2-1.4 ms from publish to share.
- team route: some processes run every repetition at the 1-thread time (16T, broccoli) faer#107 is a process-level 3x mode at 16T on broccoli that looks like this. That is not confirmed yet; the 16T probe run is pending.
- It burns CPU on a shared host. A member that is left without work keeps a CPU busy for up to 200 ms.
What other runtimes do
The comparison with libgomp, libomp and OpenBLAS is in tensor4all/faer#107 (comment 6486485300).
| Runtime | Spin phase | Then | With more threads than CPUs |
|---|---|---|---|
| libgomp | about 3 ms of PAUSE spins (300k spins with OMP_WAIT_POLICY unset) |
futex; it never yields | spin count throttled to 100 |
| libomp | yields every 256 spins | sleeps after its blocktime: 200 ms, or 0 on hybrid CPUs | yields on every spin |
| threadteam | 16k PAUSEs, then yield on every poll for up to 200 ms | condvar park | no detection |
Proposal
The numbers are provisional, taken from libgomp's defaults, and are to be checked against the broccoli measurement.
- No
sched_yieldphase. Spin with one PAUSE per iteration (YIELD on aarch64), then park. - Keep spinning inside a call, park soon between calls.
- While a scope or call is active, members spin between sections without parking, so a factorization pays no wake latency.
- Between calls, the park delay drops to about 1 ms instead of 200 ms. It stays configurable through
WaitPolicyandWaitOverride.
- Throttle the spin when oversubscribed. If the members a process has reserved outnumber the CPUs in
sched_getaffinity, spin only about 100 times before parking. - Leave threads unpinned by default, as all three runtimes do.
Acceptance
- The faer#107 probe at 16T on broccoli shows no stacked mode across 20 processes.
- An idle team costs no CPU after the park delay.
- No regression in the steady-state dispatch rows.
Related: tensor4all/faer#107, tensor4all/faer#95.
Metadata
Metadata
Assignees
Labels
No labels
Activity
placement: opt-in participant pinning, PAUSE-only reserved spin, over…
placement: opt-in participant pinning, PAUSE-only reserved spin, over…
shinaoka commentedon Oct 29, 2026
#10 merged (d3d9eca): opt-in participant pinning (
PoolBuilder::pin_participants, default off; the leader is never pinned), PAUSE-only reserved spin, and park after a short spin when a scope is oversubscribed.broccoli, 16T x20 (monitor's check):
Still open here:
threadteam 0.1.3: opt-in participant pinning (default off), PAUSE-onl…
simonb282 commentedon Oct 29, 2026
Data point for proposal 2 (the park delay), measured on threadteam 0.1.3, so with the PAUSE-only reserved spin of #10 included. tenferro-rs#2290 keeps the engine's team reserved for the engine's lifetime and notes that the 200 ms spin tradeoff was not benchmarked. In a loop of one parallel call followed by 20 ms of serial work on the leader (a serial phase shorter than idle_spin, so the members never park), the CPU time is still N times the wall time, with a bare pool and with the default policy:
threadteam::Pool, 4 threads,idle_spin200 ms (default)idle_spin0idle_spin200 msidle_spin0So the spin is cheaper per poll since #10 but still holds a core for the whole window. This is about CPU time per wall time, not the slow mode of your broccoli runs, so the two do not conflict.
Reproducer (public API only,
thread_spin team <threads> <idle_spin_ms>): https://github.com/simonb282/t4a-perf-repro at945ed04. Intel i5-1155G7, 8 logical CPUs, release profile.Question: does the roughly 1 ms park delay of proposal 2 also apply to members reserved by a
Teamfor an engine's lifetime, or only to the delay between calls? For a held team the 200 ms window never ends in a loop like this. I have not measured the wake-up cost of a shortidle_spinon a mix with many small sections, so I make no claim about the best value.shinaoka commentedon Oct 29, 2026
Thanks for the measurement and the reproducer, Simon.
To your question: yes. In threadteam 0.2.0 (the Rayon line, being prepared for release now), every team wait is bounded, including the members a
Teamkeeps reserved for an engine's lifetime (Pool::into_team). A waiting member spinsspin_pollspolls, yields a few times, then sleeps until woken, as libgomp does. So the "park after about 1 ms" of proposal 2 becomes a choice ofPoolBuilder::spin_pollsper pool rather than a separate delay.How long the default (300000 polls) lasts depends on the CPU; on one of our quiet hosts a member was still spinning after a 10 ms gap, so with your 20 ms serial phase the result is not obvious. Once 0.2.0 is published, could you rerun
thread_spinwith the default and with a smallspin_polls? That gives the CPU/wall side; we measure the wake latency side on our hosts.Where the knob lives is answered in tensor4all/tensor4all-rs#1054.