cmd/sweep: add the cluster sweep orchestrator and its Typst report - #337
Open
meling wants to merge 4 commits into
Open
cmd/sweep: add the cluster sweep orchestrator and its Typst report#337meling wants to merge 4 commits into
meling wants to merge 4 commits into
Conversation
Contributor
 |
|
Overall Grade |
Security Reliability Complexity Hygiene |
Code Review Summary
| Analyzer | Status | Updated (UTC) | Details |
|---|---|---|---|
| Go |  |
Aug 12, 2026 12:38p.m. | Review ↗ |
| Shell | |
Aug 12, 2026 12:38p.m. | Review ↗ |
Important
AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.
meling
changed the title
sweep runs a benchmark matrix across a cluster and turns the results into a report. It deploys binaries to the hosts, drives each run, collects the per-node results, and can list, replay, convert, and export them; the sweep axes include the send and receive buffer dimensions, bridged to benchkit's canonical dimension names so a run is described the same way end to end. Results export as plotdata.binpb, as CSV, and as a compact transfer that can be rebuilt from a completed run without re-running it. The report is generated with Typst: figures are planned per metric and per x dimension, laid out in sections with ordered legends, and annotated with node-health, degraded-share, status, and clock-offset diagnostics, Student's t confidence intervals, named cross-repetition outliers, a failed-runs section, and optional LLM triage over the plot data. Orchestrating a real cluster is where most of the failure modes are, so the robustness work is part of this: remote paths are shell-quoted, cleanup is chunked to stay under argument-length limits, the output scanner grows its buffer for long lines, port checks are distinguished from reachability failures, launch goroutines are joined rather than abandoned, a detached run can be recovered, and -collect-no-wait salvages results from a wedged driver. The orchestrator and the report are one Go package: the driver generates a report at the end of a run, and the plot-mode planner the orchestrator needs calls into the report's figure layout. They are therefore one PR rather than the two the plan projected.
benchmarking.md covers running a benchmark, locally and across a cluster, and reading what comes back. The benchkit HTML pages document the harness surface and the troubleshooting paths for a run that fails partway. The developer guide gains the make targets these tools add.
The struct literal's field alignment was off, which gofmt -l flags. No behavior change; gofmt -l is now clean across every workspace module.
The role test was spelled out twice, once as n.server && !n.client and once as its negation, and the negated form reads as a double negative. Bind it to serverMeasuredOnly so both uses read the same and only the positive form appears.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
sweep, which runs a benchmark matrix across a cluster and turns the results into a report.It deploys binaries to the hosts, drives each run, collects the per-node results, and can list, replay, convert, and export them. The sweep axes include the send and receive buffer dimensions, bridged to benchkit's canonical dimension names so a run is described the same way end to end. Results export as
plotdata.binpb, as CSV, and as a compact transfer that can be rebuilt from a completed run without re-running it.The report is generated with Typst: figures planned per metric and per x dimension, laid out in sections with ordered legends, and annotated with node-health, degraded-share, status, and clock-offset diagnostics, Student's t confidence intervals, named cross-repetition outliers, a failed-runs section, and optional LLM triage over the plot data.
Orchestrating a real cluster is where most of the failure modes are, so the robustness work is part of this: remote paths are shell-quoted, cleanup is chunked to stay under argument-length limits, the output scanner grows its buffer for long lines, port checks are distinguished from reachability failures, launch goroutines are joined rather than abandoned, a detached run can be recovered, and
-collect-no-waitsalvages results from a wedged driver.This retires
cmd/plotand the Python plotting scripts.Why the orchestrator and the report are one PR. They were planned as two, but
benchkit/cmd/sweepis a single Go package and the halves call each other:driver.gocallsautoReport, and the plot-mode planner the orchestrator needs calls the report's figure layout,cdfRun, andtimeSeriesRunFigures. Three file partitions were tried, down to justreport.goandtypstlib.go; every one left the package uncompilable in both directions. Splitting would have meant re-authoring a report-free version of the package and then reverting it.Verification:
go test ./... -count=1,go test ./benchkit/... -count=1,gofmt -l.Top of the stack. With this merged the tree matches the reference tip exactly, apart from one deliberate
gofmtfix inreport.go.