Repository navigation
WidthCosts is flat above width 6: extrapolate conservatively and accept measured anchors up to 64 #6
Description
Problem
WidthCosts is measured at widths 2, 4 and 6 and is flat above the last anchor (TABLE = 6; ns_at reads the last entry for every wider width). On a 32 to 64 thread team every consumer therefore sees the cost of a 6-wide section for a 64-wide one, so width_for / width_for_by answer the full width whenever the 6-wide section pays, and a region opens all 64 participants for work that only pays at a few.
This affects every consumer that reads the table:
- tensor4all/tcontract-rs:
Exec::width_forandwidth_for_within(contraction planningplan/mod.rs, batch lanesfaer_auto); - tensor4all/tlinalg-rs through tcontract-exec;
- tensor4all/faer, which works around it locally since #97 (
utils::cost::team_section_cost: for widths above the largest measured widthw_m, at leastcost(w_m) * w / w_m).
The local workaround is a second reading of the cost in one consumer; it belongs here, once, for all of them.
Proposal
- Conservative extrapolation in
WidthCosts. Above the largest measured anchorw_m, the cost is at leastcost(w_m) * w / w_m(linear in the width, never flat). Widths up to the largest anchor keep their measured values, so no existing decision at those widths changes. - Variable anchors. Let a row carry measured anchors beyond 6 (up to 64 and more), so a row measured on a many-core host replaces the extrapolation where it has data. The table length follows the largest anchor (keep it allocation-free and
const-constructible). - A many-core row. 並列ゲートの dispatch cost が幅 6 で頭打ち、通れば常に全幅: p=32〜64 で floor 付近の region が過剰に開く faer#91 measures the team dispatch cost at widths 2 to 64 on broccoli (AMD EPYC 7702P, Zen2, 64 cores, boost off, performance governor). Add that row for the class it belongs to (Zen2 is
X86_64Avx2; today that class takes the conservative generic row) with its provenance, the same way the existing rows are documented. - Then faer drops
team_section_costand readsDispatchCostsdirectly, and tcontract-exec / tlinalg pick the change up through their lock.
Done when
WidthCosts::atis non-decreasing beyond the largest anchor and at least linear in the width there; a unit test pins it (a flat 1 us row gives 1/2/4/8 us at widths 6/12/24/48, and a region that does not pay at 6 does not pay at 64).- Rows with anchors beyond 6 are supported and tested; existing rows give identical values at widths 1 to 6.
- The pre-push checklist in
AGENTS.mdpasses (includingtests/alloc_free.rs). - A patch release (0.1.x); publishing needs the maintainer's approval.
Related: tensor4all/faer#91 (measurement), tensor4all/faer#95 (audit), tensor4all/faer#97 (local workaround), tensor4all/tcontract-rs#148.
🤖 Generated with Claude Code
Metadata
Metadata
Assignees
Labels
No labels
Activity
shinaoka commentedon Oct 29, 2026
A
X86_64Avx2(Zen2) row at widths 2-64, measured on broccoliThis is the data for item 3 of the issue (the many-core row). Method is
benches/dispatch_cost.rsitself, not a re-implementation: the Linux port that takes the pool width and the measured widths
from the environment (
TT_POOL,TT_WIDTHS), copied unchanged fromfaer-benchmarkorigin/bench/awake-team-linuxdata/probes/20261026-awake-team-linux/bench/tt/src/bin/dispatch_cost.rs,built against
threadteam 0.1.0from crates.io. I pickedteam/load10us-minus-10usas the rowbecause that is the quantity the existing anchors document ("back to back sections whose shares
carry 10 us of work",
cost.rs:277-289).team/load10us-minus-10us, µsteam/hot(empty shares) for the same widths: 1.33, 2.11, 2.53, 2.95, 3.93, 4.58, 4.62, 5.04,6.04, 10.33 µs. Raw CSV (
kind,width,ns, every row the bench emits): attached below.The two other kinds for the same run, if they are useful as a control (
plain/outside-hotandplain/in-pool, pinned to 0-63 with a 64-wide pool): 32.6/9.4 µs at width 8, 63.6/42.3 at 32,76.8/58.7 at 64. They are much larger than the team row because a plain token wakes a spinning pool,
which is not what a team section pays.
Provenance
broccoli: AMD EPYC 7702P (Zen2), 64 physical cores, SMT off, NUMA 1, L3 = 16 MiB per4-core CCX (16 domains);
cpufreq/boost = 0, governorperformance(set by the maintainer, bothread back before the run); 257 GB RAM; Linux.
taskset -c 0-63(the analogue of the 8-threadevorow, which waspinned to exactly its eight cores). Nothing else ran on the host during the run — the agent was
idle, verified with a 3 s per-CPU busy window.
parallelfeature;TT_POOL=64,TT_WIDTHS=2,4,6,8,12,16,24,32,48,64.host_default()on this host reports the generic row(team 3 µs, in pool 15, outside 40) because
HostClass::detectneeds AVX-512F for the measuredrow and Zen2 has AVX2+FMA.
Note for the row that lands
With the three existing anchors the row would be
[1.42, 1.99, 2.33] µs, and withTABLE = 6themodel then answers 2.33 µs at width 64 against 9.33 µs measured — a 4x understatement in exactly the
direction item 1 of the issue is about (it permits a 64-wide section whenever a 6-wide one pays). If
item 2 (variable anchors) lands, this row should carry the full ten widths above; the measured shape
is sublinear (≈
2.33 · (w/6)^0.45between 6 and 64) so the linear extrapolation in item 1 staysconservative against it.
Cross-reference: the same measurement, with faer's three caller routes as empty sections, is in
tensor4all/faer#91.
cost: extrapolate WidthCosts linearly above the largest anchor; varia…
shinaoka commentedon Oct 29, 2026
Stage 1 (proposals 1 and 2) is merged in #7 and released as threadteam 0.1.1 (tag v0.1.1, b2a030c): linear extrapolation above the largest anchor, and
WidthCosts::from_anchorsfor up to 12 anchors. Remaining: the broccoli Zen2 row forX86_64Avx2once tensor4all/faer#91 has the width 2-64 data (separate patch), then faer dropsteam_section_costand its mirrored flat model.cost: measured Zen2 team row for x86_64-avx2, anchors 2 to 64 (#6 sta…
cost: read the team dispatch cost from threadteam 0.1.2; drop team_se…
deps: threadteam 0.1.2 in the lock (#151)
deps: threadteam 0.1.2 in the lock (#81)