Repository navigation
Add reproducible scientific and performance reports - #121
Merged
Merged
Conversation
mveril
force-pushed
the
feature/scientific-validation-reports
branch
2 times, most recently
from
October 12, 2026 11:08
6db5bd8 to
a2c4391
Compare
mveril
force-pushed
the
feature/scientific-validation-reports
branch
from
October 12, 2026 16:13
a2c4391 to
79e94ec
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Regression coverage
Added focused cases for reference convergence failures and unknown status, missing/unsupported schema versions, malformed output sections, non-finite energies and spin values, recorded execution errors, combined HF/MP2/spin status, and failed-case retention in Markdown and JSON. Criterion tests cover report parsing and stale result rejection.
Verification on commit
0f75639abc77802b46e1db088413e46deb5bb7beuv run --locked pytest tools/reference tools/benchmarks— 55 passed.uv run --locked ruff check tools/reference tools/benchmarks— passed.cargo fmt --all -- --check— passed.cargo clippy --workspace --all-targets --all-features -- -D warnings— passed.uv run --locked python tools/reference/report.py --output-dir target/scientific-report— generated a report with 16/16 cases passing; report metadata records this commit and a clean working tree.RUSTIQ_BENCH_FILTER=h2-sto3g cargo bench --features bench-support --bench eri_timingsRUSTIQ_MP2_SIZES=12 RUSTIQ_MP2_MEMORY='32 MiB' cargo bench --features bench-support --bench mp2_timingsuv run --locked python tools/benchmarks/report.py --criterion-dir target/criterion --output-dir target/performance-reporteri/h2-sto3g/sto-3g) and MP2 dimension 12 from Criterion 0.8.2. The means were approximately 0.985 ms for ERI, 2.581 ms for dense MP2, and 1.075 ms for blocked MP2. The configured MP2 workspace budget was 32 MiB. Both report metadata sections record this commit and a clean working tree; peak RSS was not measured.Remaining limitations
The scientific suite covers selected geometries, basis sets, methods, and tolerances; agreement does not establish general chemical accuracy. Only small representative benchmarks were run locally; larger benchmark cases remain machine-dependent and were not measured here. The MP2 performance inputs are synthetic and do not represent complete molecular calculations. The workflow is opt-in via
workflow_dispatch; its dispatched benchmark job was not run in this review.Merge readiness
Pending: CI run 629 for this commit is still in progress. The requested local checks and report generation passed.