perf(aggr): scale grouping across aggregate families

[singaraiona]

What & why

Grouped queries repeated key work across aggregates, duplicated large state arrays across workers, and left dominant groups or result gathering on one worker. This change shares query-local grouping and stable row layouts across aggregate families, bounds partition scratch storage, and splits large consumers into parallel stages.

• Share native and full-value key grouping across streaming, indexed and mixed queries, including selections and composite keys.
• Emit native scalar results for every registered streaming accumulator. The new native writers share their result calculation with boxed compatibility.
• Scatter deduplicated native input fields in bounded row chunks; choose dense work from state size, row distribution and the heap budget.
• Parallelize exact median/quantile, mode, distinct, first/last and top/bottom K, including medium-sized groups that previously fell below the split threshold. Preserve nulls, symbol domains, ordering and the query-language K limit of 1–1024.

The results report includes all worker counts, cold/warm timings, per-process samples, peak memory and remaining regressions. Cold setup costs and timing variation are reported explicitly; speedups vary by aggregate and key shape.

Validation

• Required Rayforce audit: PASS on 19154617.
• All CI checks: PASS on 19154617, including Linux/macOS debug and release, TSan, fuzz-smoke, and static analysis.
• Full local make cppcheck with Cppcheck 2.13.0: PASS, no diagnostics.
• Complete ASan/UBSan suite after audit fixes: 3,832/3,832 passed at fdf365f3; restored symbol and wide count-distinct query fixtures also passed with 28 total threads.
• Aggregation TSan suite at fdf365f3: 26/26 passed, four total threads, without suppressions. Earlier five-thread attempts at the original measured revision stalled in pool shutdown before aggregation and are not counted as passes.
• Synthetic query sweep: 2,700 verified fresh-process runs, 77 cases, 1/2/4/8/16/default workers, three rounds.
• Internal top/bottom-K histogram oracle: 90/90 runs passed; includes K near the input size.
• Regression follow-up: 68 profiled runs and 130 additional fresh-process runs. Warm slowdowns in the main sweep did not reproduce in the five-process follow-up; reproducible cold-start costs remain explicit in the report.
• Full scaling sweep used release f61a4eda. Audit follow-up release fdf365f3 builds without warnings and passed 174/174 focused dense-route comparisons, including repeats of three initial slowdowns; both samples and binary hashes are retained in the report.

Four baseline distinct cases fail to execute correctly and receive no speedup claim. Their candidate results are checked across workers and against independent unit oracles.

• Sorting follow-up at 8ab7c40f: clean release builds without warnings; 130/130 ASan/UBSan tests and 41/41 parallel TSan tests passed after fixing two races exercised by the restored query fixture.

Audit fixes

• Use the equivalent _Atomic TYPE* declaration form in the five shared-directory macros. Cppcheck 2.13.0 reproduces the five pointer-to-integer diagnostics on the original code and reports none on the fixed file with CI flags. With matching build metadata, the release executable is byte-for-byte identical. Full local and hosted static analysis both passed on 19154617.
• Restore the original single-key symbol query alongside the two-key radix fixture. Normalize only the returned table for per-row key/value checks because group output order is unspecified; the query itself is unchanged.
• Add Rayfall query oracles for all nine STR/GUID/LIST key/value combinations, composite keys wider than 16 bytes, selected queries, and mixed count-distinct queries.
• Preserve zero-group results when empty-input type inference returns a domain/type error. Separate tests verify that OOM and cancellation still propagate.
• Make the sorting early-exit flag atomic and remove an unused cross-task read during in-place packed-key preparation; boundary checks remain after the dispatch barrier.
• Assert scratch-array bounds, retain generic native-width copying, and guard invalid internal rank requests. The existing dispatch barrier and ready flag cover the reported cancellation concern.

Checklist

• PR targets dev
• Commits follow Conventional Commits
• Clean release build completes without new warnings
• Tests pass; behavior, memory-budget and scheduling assertions updated

[github-actions]

Rayforce targeted audit passed

The required Rayforce audit gate passed on the latest run.

Workflow run: https://github.com/RayforceDB/rayforce/actions/runs/35107326720