Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ jobs:
BENCH_INPUT_CACHE_DIR=/tmp/smoke-cache OUT_DIR=/tmp/smoke/raw
EXPORT_HDF5=/tmp/smoke-export
cargo run --release --locked --bin mpo_mpo_aniso_patched
- name: smoke vertex Bethe-Salpeter contraction (downloads one 3 MB input)
run: >
BENCH_VERTEX_TAUS=1e-4 BENCH_RUNS=1 BENCH_WARMUPS=0
BENCH_INPUT_CACHE_DIR=/tmp/smoke-cache OUT_DIR=/tmp/smoke/raw
cargo run --release --locked --bin vertex_bse
- uses: julia-actions/setup-julia@v2
with:
version: "1"
Expand Down
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ Read `README.md` before changing case definitions, timing boundaries, input gene

## Maintained scope

The repository has exactly three benchmark cases: Fourier elementwise, Gaussian elementwise global versus patched, and Gaussian MPO-MPO contraction global versus patched. Do not add another case when an arm or metric in one of these cases answers the question.
The repository has exactly four benchmark cases: Fourier elementwise, Gaussian elementwise global versus patched, Gaussian MPO-MPO contraction global versus patched, and the vertex Bethe-Salpeter contraction. Do not add another case when an arm or metric in one of these cases answers the question.

Gaussian inputs use independent two-dimensional interpolative decomposition per term, deterministic balanced pairwise add-and-compress, final relative-L2/SVD tolerance `1e-6`, and fixed patch cap 128. Do not add TCI generation, whole-mixture interpolation, isotropic controls, independent `R` sweeps, or a runtime patch-cap knob without an explicit design change.

Vertex inputs are the fixed files of the `vertex-bse-inputs-v1` release. Do not regenerate or replace them; new inputs need a new release tag.

ACI residual tolerance is not an L2 tolerance. Keep its internal metric explicit and compare it with other arms only through the common achieved external error.

## Timing
Expand All @@ -18,7 +20,7 @@ Run release builds with one pinned CPU core and single-threaded Rayon and BLAS.

A machine may carry one additional scaling profile named `<machine>-<n>t`,
produced with `BENCH_THREADS=n`, which pins a contiguous core range and lets the
same three cases run on `n` Rayon and BLAS threads. Its `run.yaml` records the
same four cases run on `n` Rayon and BLAS threads. Its `run.yaml` records the
thread count and the core range. Treat it as scaling evidence only: the
single-core profile stays the reference, and the two profiles are never merged.

Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
libm = "0.2"
hdf5-metno = { version = "0.12", default-features = false }

[profile.release]
debug = 0 # CARGO_PROFILE_RELEASE_DEBUG=true restores full debug info
Expand Down
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ has the layout.
| `elementwise_fourier` | Elementwise product of two random one-dimensional Fourier QTTs | naive, zip-up, fit, ACI | measured input χ |
| `gaussian_elementwise` | Elementwise product of randomly rotated anisotropic Gaussian QTTs | global fit, patched fit, global ACI, patched ACI | compressed input χ |
| `gaussian_mpo_contraction` | MPO-MPO contraction of the same Gaussian input family | global fit, patched fit | compressed input χ |
| `vertex_bse_contraction` | Bethe-Salpeter contraction of impurity-model vertices (fixed inputs, downloaded) | zip-up, fit | input tolerance |

There is no independent `R` sweep. `R` is the smallest value that gives at least eight grid cells across the fixed minor-axis standard deviation. Gaussian count `N` and `R` are construction metadata, not analysis axes.

Expand All @@ -34,6 +35,19 @@ The terms are combined by deterministic balanced pairwise reduction. Every pair

The expensive raw input pair is cached atomically in `.cache/inputs/`. The cache key includes the schema version, pinned tensor4all-rs revision, `N`, `R`, width, aspect range, spacing, polynomial degree, interpolation tolerance, addition tolerance, and seed. Cases 2 and 3 share the same cache entry. Set `BENCH_INPUT_CACHE_REFRESH=1` to rebuild it or `BENCH_INPUT_CACHE_DIR` to move the cache.

## Vertex input

Case 4 contracts two-particle vertices of the single-impurity Anderson model (parquet approximation, magnetic channel,
β = 1000, 1024 Matsubara frequencies per axis) as a Bethe-Salpeter equation does,
Φ(ν, ν', ω) = Σ_ν₁ Γ(ν, ν₁, ω) χ0(ν₁, ω) F(ν₁, ν', ω): `hadamard` weights Γ by the bubble χ0, and `partial_contract`
joins the result with F, timed together and truncating at τ/10 per bond. The inputs are fixed files, one per input
tolerance τ (`1e-4`, `1e-6`, `1e-8`), from the
[`vertex-bse-inputs-v1`](https://github.com/tensor4all/tensor4all-benchmark/releases/tag/vertex-bse-inputs-v1)
release, whose notes describe them; they are downloaded once into `.cache/inputs/vertex_bse/`. The error is the
relative L2 error against the exact Φ at 4096 points, and a run fails above 10 τ. On one thread τ = `1e-4` takes
17 s (zip-up) and 33 s (fit) with a 1.5 GB peak, and τ = `1e-6` 9 and 14 minutes with an 18 GB peak; the default
sweep runs these two, `1e-8` is opt-in.

## Allocation counters

Every record carries the allocation calls and the requested bytes of the timed
Expand Down Expand Up @@ -95,7 +109,7 @@ Requirements are Rust, HDF5, and a BLAS/LAPACK implementation. A complete single
BENCH_CPU_CORE=0 scripts/run_all.sh linux-epyc-7713p
```

The script pins Rayon and common BLAS implementations and the tenferro thread count to one thread, runs the three maintained binaries in release mode, writes raw records under `result/<profile>/raw/`, records machine metadata in `run.yaml`, and generates `report.md`.
The script pins Rayon and common BLAS implementations and the tenferro thread count to one thread, runs the four maintained binaries in release mode, writes raw records under `result/<profile>/raw/`, records machine metadata in `run.yaml`, and generates `report.md`.

Set `BENCH_THREADS=n` for the optional scaling sweep. It uses `n` threads on the contiguous core range starting at `BENCH_CPU_CORE`, and belongs in its own profile named `<machine>-<n>t`. The single-core profile stays the source of truth.

Expand Down Expand Up @@ -128,6 +142,8 @@ Case 2 additionally accepts `BENCH_ACI_TOL`, default `1e-8`. This is an ACI resi

Case 1 accepts `BENCH_KS`, `BENCH_R`, `BENCH_TOL`, `BENCH_MAX_BOND`, `BENCH_RUNS`, `BENCH_WARMUPS`, `BENCH_SEED`, and `BENCH_ALGOS`. Its report is sorted by measured input χ even though mode count generates the instances.

Case 4 accepts `BENCH_VERTEX_TAUS` (default `1e-4,1e-6`; available `1e-4`, `1e-6`, `1e-8`), `BENCH_ALGOS` (default `zipup,fit`), `BENCH_RUNS`, `BENCH_WARMUPS`, `BENCH_INPUT_CACHE_DIR`, and `OUT_DIR`.

## Validation

```bash
Expand Down
2 changes: 1 addition & 1 deletion result/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# result/

One directory per (hardware profile, measured tensor4all-rs revision), each
holding one complete sweep of the three maintained cases:
holding one complete sweep of the maintained cases:

```text
result/INDEX.md generated table of contents over every sweep
Expand Down
22 changes: 20 additions & 2 deletions scripts/report.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Render and validate the three maintained benchmark cases."""
"""Render and validate the four maintained benchmark cases."""

import json
import sys
Expand All @@ -10,6 +10,7 @@
"elementwise_fourier": {"naive", "zipup", "fit", "aci"},
"gaussian_elementwise": {"global_fit", "patched_fit", "global_aci", "patched_aci"},
"gaussian_mpo_contraction": {"global_fit", "patched_fit"},
"vertex_bse_contraction": {"zipup", "fit"},
}


Expand Down Expand Up @@ -47,7 +48,8 @@ def main(profile: Path) -> None:
assert {record["algorithm"] for record in grouped[case]} == arms
by_instance = defaultdict(list)
for record in grouped[case]:
key = record["params"].get("k_max", record["params"].get("n_gauss"))
params = record["params"]
key = params.get("k_max", params.get("n_gauss", params.get("input_tolerance")))
by_instance[key].append(record["algorithm"])
for key, actual in by_instance.items():
assert Counter(actual) == Counter(arms), (case, key, actual)
Expand All @@ -57,6 +59,10 @@ def main(profile: Path) -> None:
assert params["input_l2_rtol"] == 1e-6
assert params["external_error_metric"] == "sampled_relative_l2"
assert record["max_error"] <= 1e-4
for record in grouped["vertex_bse_contraction"]:
params = record["params"]
assert params["external_error_metric"] == "reference_relative_l2"
assert record["max_error"] <= 10 * params["input_tolerance"]
run = metadata(profile / "run.yaml")
lines = [
"# Tensor4all benchmark results",
Expand Down Expand Up @@ -96,6 +102,18 @@ def main(profile: Path) -> None:
lines.append(
f"| {record['input_max_bond_dim']} | {raw_chi} | {params['n_gauss']} | {params['r']} | {record['algorithm']} | {record['wall_time_median_secs']:.6f} | {record['max_error']:.3e} | {input_patches} | {input_patch_chi} | {record.get('n_patches', 1)} | {record.get('n_params', '')} | {alloc_calls(record)} | {alloc_mib(record)} | {speedup:.3f} |"
)
lines += [
"",
"## Case 4: vertex Bethe-Salpeter contraction",
"",
"| input tolerance | input χ (Γ / F) | arm | time (s) | relative L2 vs reference | best attainable | output χ | parameters | allocations | allocated (MiB) |",
"|---:|---:|---|---:|---:|---:|---:|---:|---:|---:|",
]
for record in sorted(grouped["vertex_bse_contraction"], key=lambda item: (-item["params"]["input_tolerance"], item["algorithm"])):
params = record["params"]
lines.append(
f"| {params['input_tolerance']:.0e} | {params['gamma_chi']} / {params['f_chi']} | {record['algorithm']} | {record['wall_time_median_secs']:.6f} | {record['max_error']:.3e} | {params['best_attainable_error']:.3e} | {record['output_max_bond_dim']} | {record.get('n_params', '')} | {alloc_calls(record)} | {alloc_mib(record)} |"
)
(profile / "report.md").write_text("\n".join(lines) + "\n")


Expand Down
1 change: 1 addition & 0 deletions scripts/run_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,5 +74,6 @@ EOF
run cargo run --release --locked --bin elementwise_fourier
run cargo run --release --locked --bin elementwise_gauss2d_patched
run cargo run --release --locked --bin mpo_mpo_aniso_patched
run cargo run --release --locked --bin vertex_bse
python3 scripts/report.py "$out"
python3 scripts/gen_index.py
Loading
Loading