[shinaoka]

Replaces the private Rayon ThreadPool with threadteam's own std::thread workers. Design: #15 and docs/design/own-threads.md. Maintainer direction (2026-10-29): after #14's iterations on the Rayon base (e76036c, 1f2145b) did not meet (a), (b) and (c) together, move to own threads, and keep Rayon interop as an opt-in layer.

Closes #15. Supersedes #14, which stays a draft as the measurement record; its commits (the wait benchmarks a3a9320 and libgomp's waits) are the base of this branch. Refs #13, #12.

What changes

• Waits (625a593). libgomp's waits only: spin 300000 polls, yield 16 times, futex. Preemption detection, warm-up and the pressure knobs from Waits like the OpenMP runtimes: bounded poll spin, yields, then sleep; wait benchmarks (faer#107) #14 are gone.
• Own workers (0e259b5).
  • Reuse. A process-wide reserve of idle workers, created by the first explicit Pool::build (no implicit global pool). A pool with default thread settings leases workers from it and returns them on drop. A pool built per call (faer with_team) reuses running threads instead of spawning fresh ones.
  • Dedicated workers. A pool with thread_name, on_start, a stack size or a QoS of its own gets dedicated workers, which its drop joins, as before (tcontract's C API).
  • Plain tokens. fork, for_ranges and external publish a stack-allocated batch on the pool's intrusive queue; no allocation.
  • Broadcasts. They go into an ordered list.
  • Waits across pools. A worker that waits keeps serving its own pool's queue and broadcasts, so A → B → A completes.
  • Identity. A worker is found by its ThreadId; no thread-locals.
  • Slot state. One word per slot (RESERVED, LENT, BUSY, DETACH), checked with loom.
  • Features. parallel no longer pulls Rayon. rayon is a separate opt-in with Exec::external_rayon.
• Post-review fixes (d6fe3f6). The gpt-6.1-sol review of the implementation found three deadlocks, an endless wake-up and an affinity regression; all are fixed. Each has a regression test that fails or hangs with its fix undone (docs/design/own-threads.md, "Post-review resolutions").
• bench-ab.sh summary (78b59a1). Parses params that hold commas (the sections rows) and keeps the Rayon reference rows apart from threadteam's.

Decisions for the maintainer

item	before (Rayon)	now
Pool drop	joins every worker	dedicated workers: joined. Shared workers: drop waits until each one detached, then they sleep in the reserve for the next pool
Pool dropped on its own worker	detach without join	kept: the drop does not wait for or join its own worker, which acknowledges after its task. The test drop_from_a_worker_of_the_pool_does_not_hang is kept (shared and dedicated)
idle_runs_rayon	an idle member runs Rayon jobs	no effect, kept for API compatibility
idle_spin, release_idle, resident_members	Rayon-return semantics	the same knobs on the native loop (table in the design doc)
external on a team token	lends members to Rayon	lends them to the pool's queue; Rayon lending is external_rayon (feature rayon)
feature parallel	pulls Rayon	pulls nothing

These are unchanged:

• budget conservation;
• no Rayon type in the public API (check-no-public-rayon.sh);
• an allocation-free steady state (strict counting now);
• Exec is neither Clone nor Copy;
• loom for park and wake.

Verification (primerose: EPYC 7713P, 64 CPUs, shared and loaded, load average about 35)

• Local gate on d6fe3f6:
  • fmt and clippy (all features, no default features);
  • tests with default features, all features (twice) and no default features;
  • loom: 12 models;
  • deny bans, no-public-rayon, cargo doc (-D warnings) and the aarch64-apple-darwin check;
  • taskset -c 0-2: idle_clock, oversubscribed, team, own_threads and team_owner, five times each.
• New tests:
  • tests/own_threads.rs covers reuse (a spawn counter), dedicated workers with hooks, A → B → A, broadcast order, task panics, the affinity of a reused worker, an on_start pin kept on a dedicated worker, and a waiting broadcast share that does not start a later broadcast;
  • crate-internal tests cover self-drop (shared and dedicated), a pool dropped inside another pool's task, nested waits keeping the outer one wakeable, and a broadcast cursor moving past an already unlinked broadcast;
  • five new loom models: slot-state races, a batch with two claimers and a sleeping owner, a broadcast to two workers, and an idle worker against a publication, which deadlocks when the publisher's wake is removed.
• Downstream, with threadteam patched to this branch:
  • faer t4a-faer --lib: 580 passed;
  • tcontract-rs: the C-API build and the workspace tests pass;
  • tenferro tenferro-cpu: pass;
  • tlinalg-rs: the C-API build passes, and the workspace tests pass except one intermittent test, alloc_counts::the_pool_route_adds_a_constant. It failed in 2 of 4 runs on this branch (batch 2 = 99 allocations) and in 0 of 3 on 0.1.3, and passed 5 of 5 times with --test-threads=1 (0 allocations).
    • Cause: a test-isolation gap in tlinalg, not in threadteam. The test counts allocations on all threads, and gemm_path_allocations_are_a_pinned_upper_bound in the same file does not take serialised(), so its gemm allocations land in the window.
• bench-ab, a3a9320 (base, 0.1.3 waits) against d6fe3f6, OMP=1, width 16 on CPUs 48-63, 6 rounds. Medians of the round medians in µs. Loaded host, so indicative only:

scenario	metric	base	head	libgomp
acquire held_team	call	1104.7	1083.4	1143.6
acquire held_pool	call	1103.1	1081.0	-
acquire fresh_pool	call	117524.4	1089.8	-
acquire fresh_pool	total (build to drop)	129646.9	4027.2	-
gaps 0 µs	first / call	7.1 / 68.1	7.2 / 68.4	8.0 / 78.9
gaps 100 µs	first / call	7.0 / 67.8	7.0 / 67.5	8.1 / 78.9
gaps 1 ms	first / call	7.0 / 67.6	7.0 / 67.7	8.3 / 80.0
gaps 10 ms	first / call	7.1 / 68.2	43.0 / 109.5	35.7 / 113.1
sections 5 µs × 200	call	1415.9	1378.3	1588.6
sections 20 µs × 200	call	4400.9	4374.0	4612.0
sections 50 µs × 200	call	10394.6	10332.7	10609.0

The fresh_pool phases on head are build 29 µs, acquire 2.9 ms (the first call wakes reused workers), release 1.4 µs and drop 20 µs. In the 10 ms gap the head sleeps after its spin budget, like libgomp, where the base spins indefinitely: that is criterion (d).

• faer probe smoke, 16 threads, gemm 128, median ms: team_per_call 5.79 → 5.80, exec_team_per_call 5.79 → 5.81, fresh_pool 6.66 → 2.78, with_team 5.73 → 2.81. Load average was about 30.

Not yet done: the acceptance runs (criteria (a)-(d), as on #14)

broccoli (64 threads):

OMP=1 taskset -c 0-63 scripts/bench-ab.sh a3a9320 feat/own-threads 10 --scenario acquire --widths 64 --calls 40 --impl threadteam
OMP=1 taskset -c 0-63 scripts/bench-ab.sh a3a9320 feat/own-threads 10 --scenario acquire --widths 64 --calls 40 --impl threadteam --load 8:10
OMP=1 taskset -c 0-63 scripts/bench-ab.sh a3a9320 feat/own-threads 10 --scenario gaps --widths 64 --calls 40 --impl threadteam --load 8:10

(a): acquire_probe_107, every arm, on tensor4all/faer probe/107-acquire-per-call (base, threadteam 0.1.3) against probe/107-acquire-per-call-own (this branch; it reads PROBE_SPIN_POLLS only).

evo (quiet):

OMP=1 scripts/bench-ab.sh a3a9320 feat/own-threads 10 --scenario gaps --impl threadteam
OMP=1 scripts/bench-ab.sh a3a9320 feat/own-threads 5 --scenario acquire --calls 40 --impl threadteam
OMP=1 scripts/bench-ab.sh a3a9320 feat/own-threads 5 --scenario sections --impl threadteam

No publication. The monitor merges.

🤖 Generated with Claude Code

[shinaoka]

evo acceptance run for head 78b59a1 (feat/own-threads), base a3a9320. These are the three evo commands from the PR body, with OMP=1, so libgomp runs in every round.

Conditions were the same as on #14:

• quiet host, boost=0 checked before each command;
• observer (the agent and herdr) on 0-3,12-15; the measurement on taskset -c 4-11;
• threadteam ran at the default widths of the affinity set (8 and 6), libgomp at width 8;
• foreign load none in all 40 passes. The load average reached 4 to 6 during acquire and sections, which is the bench's own spinning members.

Verdict: (b) passes and (d) passes. Every quiet-host cell is within 1.7 % of the base, and the 10 ms gap matches the base with the default settings, no knob needed. The one-off pool (fresh_pool, build to drop) is 14 to 18 % faster than the base.

gaps, 10 rounds (median of the round medians, µs; first section / whole call)

width	gap	base	head 78b59a1	ratio (first / call)	libgomp
6	0	5.9 / 57.7	5.9 / 58.5	1.00 / 1.014	-
6	100 µs	5.9 / 57.7	5.9 / 58.4	1.00 / 1.012	-
6	1 ms	5.9 / 57.9	5.9 / 58.5	1.00 / 1.010	-
6	10 ms	5.9 / 57.7	5.9 / 58.4	1.00 / 1.012	-
8	0	6.1 / 59.6	6.0 / 59.5	0.98 / 0.998	6.6 / 64.3
8	100 µs	6.2 / 59.6	6.0 / 59.5	0.97 / 0.998	6.5 / 64.2
8	1 ms	6.1 / 59.5	6.1 / 59.6	1.00 / 1.002	6.7 / 64.4
8	10 ms	6.0 / 59.1	6.1 / 59.7	1.017 / 1.010	15.6 / 73.3

acquire --calls 40, 5 rounds (µs)

width	arm	base	head	ratio	libgomp
6	held_team	1038.5	1040.7	1.002	-
6	held_pool	1045.5	1043.3	0.998	-
6	fresh_pool, call	1038.3	1042.0	1.004	-
6	fresh_pool, build to drop	1227.9	1060.4	0.864	-
8	held_team	1048.1	1048.0	1.000	1069.7
8	held_pool	1054.3	1052.1	0.998	-
8	fresh_pool, call	1046.8	1047.5	1.001	-
8	fresh_pool, build to drop	1307.7	1074.7	0.822	-

These are the head's fresh_pool phases at widths 6 / 8, in µs:

phase	width 6	width 8	on #14 1f2145b (6 / 8)
build	6.7	11.9	95 / 127
acquire	4.5	3.6	14 / 20
release	0.6	0.8	1.0 / 1.4
drop	5.0	8.3	78 / 116

Reusing the reserve's workers takes the one-off overhead from 183 to 264 µs down to about 17 to 25 µs. Tail outliers of 22 to 37 ms on single calls appear on both sides, as on #14. They show only in max, and one round of the head's width-8 fresh_pool has p90 6.1 ms; no median moves.

sections, 5 rounds (per call, µs)

width	work × 200	base	head	ratio	libgomp
6	5 µs	1151.8	1162.2	1.009	-
6	20 µs	4151.1	4165.7	1.004	-
6	50 µs	10155.7	10163.5	1.001	-
8	5 µs	1186.6	1185.7	0.999	1279.2
8	20 µs	4186.6	4185.1	1.000	4278.1
8	50 µs	10186.1	10184.2	1.000	10277.7

Against the criteria

• (b) passes. On the quiet host, held_team and held_pool are 0.998 to 1.002 of the base, and gaps 0 to 1 ms are 0.97 to 1.014. Both are within 10 %. sections is within 1 % as well.
• (d) passes. The 10 ms gap is 1.00 / 1.017 on the first section and 1.012 / 1.010 on the call at widths 6 / 8, with the default 300000-poll budget. libgomp is 15.6 / 73.3 µs on the same cell.
• Against libgomp: the head is 7 to 10 % faster on the 0 to 1 ms gaps and on sections 5 µs. It is 2.6× faster on the 10 ms gap, and 2 % faster on held_team.
• (a) and (c) need the broccoli runs; this run does not cover them.

Raw rows and the waits per call lines are in shinaoka-advisor/work/runs/threadteam-16/ (gaps10, acquire5, sections5, .csv and .log).

[shinaoka]

Note on the run above: the head moved to 64684c4 while it ran. That commit changes only tests/alloc_free.rs (diagnostics for a failing allocation window). It touches neither the library nor benches/, so the measurement of 78b59a1 also covers 64684c4.

[shinaoka]

CI green at a11768f

The macOS failure. alloc_free failed on macOS with 2 allocations per window on the plain-thread paths.

• The diagnostic commit (64684c4) makes a failing window print each allocation's size, thread and backtrace, so the cause shows in the CI log alone.
• Cause. sched::enter (external on a plain token) built two std::sync::Mutexes per call. On macOS std's pthread mutex is boxed on its first lock, so each call allocated twice. Linux mutexes are futex words, so the Linux runs were clean.
• Fix (a11768f). A single-use cell instead. The one-task batch runs its task exactly once (the index is claimed under the queue lock), and the caller reads the result only after completion.

Checks. All seven pass, including cargo test (macos-latest).

Still open: the broccoli and evo acceptance runs for (a)-(d), with the commands in the description. Those runs should use this head (feat/own-threads). I'm stopping here; the monitor merges.

[shinaoka]

Acceptance runs on broccoli — head measured: 78b59a1 (feat/own-threads, the tip as of this run)

Conditions as on #14: cpufreq/boost read back 0, governor performance, taskset -c 0-63,
OMP=1 so libgomp is in every table, a 5 s load logger alongside (0.40 at the start; 45-50 during the
benches is our own 64 threads, no other user's job), scripts/bench-ab.sh a3a9320 78b59a1 10, width 64,
acquire, --calls 40, then gaps under load, then acquire_probe_107 on
probe/107-acquire-per-call (base c1d8e4c, threadteam 0.1.3) against probe/107-acquire-per-call-own
(bc0c04a, this branch), 6 processes per arm (gemm128, 50 calls) and 5 for chol2048.

(a) faer#107 arms, 64T, ms per call

arm	kernel	base (0.1.3)	head 78b59a1
held	gemm128	0.02	0.02
team_per_call	gemm128	0.02	0.02
fresh_pool	gemm128	15.38 (6/6, min 15.37)	0.03 (min 0.02, max 0.04)
with_team	gemm128	15.38 (6/6)	0.02
held	chol2048	12.93	13.92
fresh_pool	chol2048	106.17	17.96
with_team	chol2048	94.30	14.88 (one 80.7 outlier in 5)

(a) met. No process in any arm is in the slow mode: the per-call pool is 0.03 ms at gemm128 against
15.38 ms on 0.1.3 (a ~500x improvement, and better than any #14 head so far — d430347 was 0.06 ms), and
chol2048 comes down from 106/94 ms to 18/15 ms.

(b) quiet, µs per call

arm	base	libgomp	head	head/base
held_team	1 493	1 682	1 499	+0.4 %
held_pool	1 575	–	1 652	+4.9 %
fresh_pool total	131 788	–	13 508	0.10

(b) met — both held arms inside 10 %, and held_team sits at libgomp's level (1 499 against 1 682).

fresh_pool phases at the new head (µs per call): build 139 (was 1 967 with a Rayon pool),
acquire 11 528, call 1 572, release 13, drop 115 (was 1 366) — the quiet-host 60-80 ms
that survived every #14 head is gone: building a pool no longer starts a Rayon runtime, and a one-call
pool's teardown is 0.1 ms rather than 1.4 ms.

(c) under --load 8:10, µs per call

arm	base	libgomp	head	head/libgomp
held_team	1 561	1 687	1 481	0.88x
held_pool	2 088	–	1 620	0.78x base
fresh_pool call	80 970	–	1 843	–
fresh_pool total	132 692	–	16 114	0.12x base

gaps under load, first section: head 17.0 µs (1 ms gap) / 15.6 µs (10 ms) against base 16.3 / 16.9 and
libgomp 19.3 / 73.2.

(c) met — held_team is 0.88x libgomp, i.e. faster than the reference, and the gap wake is the same as
0.1.3's spin and four times better than libgomp at a 10 ms gap.

Verdict

criterion	d430347 (#14)	1f2145b (#14)	78b59a1 (#16)
(a) no slow mode in any arm	met (0.06 ms)	not met (11.01 ms)	met (0.02-0.03 ms, 6/6 clean)
(b) quiet held within 10 %	met (+2.1 / +5.2 %)	held_pool +123 %	met (+0.4 / +4.9 %)
(c) loaded within libgomp	not met (3.45x)	not met (3.33x)	met (0.88x)

All three criteria are met on this host, and the two cells that pulled against each other for five #14
rounds stop pulling: the own-thread pool makes a one-call pool cheap to build and to drop (139 / 115 µs
instead of 1 967 / 1 366 µs), which is what the fresh-pool arms needed, without any preemption throttle in
the held path.

Two notes for the record: one with_team/chol2048 process in the head set took 80.7 ms once (the other
four 13.7-14.9 ms), so that arm's tail is not zero; and held/chol2048 is 13.92 against 12.93 ms on 0.1.3
(+7.7 %), which is inside the spread but is the only cell that moved the wrong way.