Skip to content

Prepare effective Hamiltonian halves in FiniteEnvironments - #546

Open
lkdvos wants to merge 15 commits into
mainfrom
ld-envs
Open

lkdvos wants to merge 15 commits into
mainfrom
ld-envs

Conversation

@lkdvos

@lkdvos lkdvos commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

FiniteEnvironments now caches the prepared left and right halves of the AC and AC2 effective Hamiltonians, so finite algorithms stop re-preparing the same operator halves at every local update.

Changes

  • Cached halves. lefthalf/righthalf(Val(N), envs, site, ψ) sit on top of leftenv/rightenv. Each half is prepared lazily and rebuilt when the environment it came from is replaced. AC_hamiltonian/AC2_hamiltonian with prepare = true assemble the operator from the cached halves through hamiltonian_from_halves whenever the environments provide them.
  • Generic MPO derivatives are split into prepare_operator_left/prepare_operator_right halves, joined by assemble_operator_halves.
  • Jordan MPO derivatives.
    • When the operator is prepared, each half folds its onsite, not-started and finished terms into its own largest term when it is built, so prepare_operator!! only prepares the continuing term. Unprepared derivatives are applied only once, so they skip the folding.
    • Fresh and cached construction share jordan_hamiltonian_left/jordan_hamiltonian_right.
    • Bra and ket no longer have to be equal: when they differ, the boundary environments are kept as explicit tensors.
  • Galerkin error in DMRG.
    • calc_galerkin takes prepare = false. With prepare = true, it builds the derivative from the cached halves, which the effective Hamiltonian built afterwards at the same site reuses. Single-site DMRG without bond expansion uses this, so the error costs no separate projection.
    • With OptimalExpand, the single-site derivative is the two-site projection the expansion forms anyway, contracted with the isometry on the other site of the bond. changebond! for OptimalExpand is split into AC2_projection and expand_from_projection!, and DMRG feeds one projection to both the error and the expansion.
    • Other expansions compute the error unprepared, as before, since the expansion would leave the prepared halves stale.
  • Tests. test/hamiltonian/prepared_halves.jl compares cached against freshly built effective Hamiltonians at every window, for several operator types including bra ≠ ket, and checks DMRG with and without expansion.

Performance

Measured on Rusty: exclusive genoa nodes (AMD EPYC 9474F), single-threaded, Julia 1.13.2, main at 76a58d7 with identical dependency versions.

  • One node per configuration, with the variants alternating on it.
  • Times are medians over 2–4 runs; same-node repeats agree to about 1%.
  • Peak memory is the max RSS of each process's first run.
  • Energies agree with main to 3e-14.

Models:

  • nearest-neighbour: transverse-field Ising, L = 24.
  • chemistry: a spinless-fermion surrogate with all-pairs hopping and all four-orbital interactions, L = 18.
  • large sparse: Ising plus 150 random four-site strings, L = 24.

DMRG and DMRG2 without expansion

model χ kind main this PR outside eigensolver allocated peak memory
nearest-neighbour 512 DMRG 40.1 s 37.6 s (0.94×) 12.0 → 11.2 s 9.9 → 8.1 GiB 2.2 → 2.3 GiB
DMRG2 74.7 s 73.4 s (0.98×) 16.2 → 15.2 s 15.1 → 14.4 GiB 2.5 → 2.6 GiB
chemistry 128 DMRG 240.7 s 235.5 s (0.98×) 17.6 → 21.6 s 33.3 → 20.3 GiB 3.6 → 5.1 GiB
DMRG2 253.1 s 248.8 s (0.98×) 22.3 → 18.1 s 21.5 → 17.8 GiB 3.6 → 5.7 GiB
chemistry 256 DMRG 605.5 s 600.8 s (0.99×) 32.6 → 39.3 s 51.4 → 34.2 GiB 7.1 → 10.4 GiB
DMRG2 466.5 s 462.6 s (0.99×) 35.9 → 32.2 s 34.0 → 31.1 GiB 6.7 → 9.9 GiB
large sparse 256 DMRG 761.4 s 747.6 s (0.98×) 59.1 → 64.4 s 91.7 → 53.5 GiB 4.7 → 7.3 GiB
DMRG2 997.4 s 985.5 s (0.99×) 67.0 → 55.4 s 58.3 → 50.1 GiB 4.5 → 8.1 GiB

This table was measured at de162e56. The later commits only touch the Galerkin error and OptimalExpand, and a separate run of single-site DMRG at the final commit matched it to within 0.5% for every model.

In single-site DMRG, preparing the halves moves time from the eigensolver to the update, where the halves are now built, and the eigensolver gains more than that.

Single-site DMRG with OptimalExpand

model χ main this PR outside eigensolver allocated peak memory
chemistry 128 173.1 s 166.8 s (0.96×) 23.9 → 18.8 s 42.6 → 40.9 GiB 4.8 → 5.6 GiB
chemistry 256 870.0 s 846.6 s (0.97×) 80.2 → 62.4 s 94.9 → 88.3 GiB 9.0 → 12.2 GiB
large sparse 256 761.8 s 741.9 s (0.97×) 79.3 → 62.9 s 92.0 → 92.4 GiB 4.7 → 8.0 GiB

This uses alg_expand = OptimalExpand(; trunc = truncrank(χ ÷ 8)) and trunc = truncrank(χ). Most of the gain comes from reusing the expansion's projection for the Galerkin error, which removes about a fifth of the time outside the eigensolver. Computing the error from the cached halves instead was slower, because the expansion leaves half of what it prepared stale.

Trade-offs

  • Peak memory. Keeping the halves costs up to about 3.6 GiB of peak memory for MPOs with many terms (+35–80% over main here). It is negligible for nearest-neighbour models.
  • Unprepared derivatives (AC_projection, AC2_projection, calc_galerkin over a whole chain) allocate as on main with BlockTensorKit ≥ 0.3.22, and their run time is within the benchmark's resolution.
    • Each half builder slices the continuing channels out of its environment, so an environment is sliced twice as often as on main. Dense BlockTensorMap slices used to allocate a full set of blocks only to discard them; Slice dense block tensors without allocating blocks BlockTensorKit.jl#88 (released in 0.3.22) removed that.
    • Measured with BlockTensorKit 0.3.22 for both main and this PR, over 5–10 runs per configuration:
model χ kernel time vs main allocated main → this PR
chemistry 128 AC_projection 0.83× 2.48 → 2.48 GiB
AC2_projection 0.95× 5.59 → 5.60 GiB
calc_galerkin 0.84× 2.48 → 2.48 GiB
chemistry 256 AC_projection 1.06× 5.87 → 5.87 GiB
AC2_projection 1.05× 12.44 → 12.45 GiB
calc_galerkin 1.06× 5.87 → 5.88 GiB
large sparse 256 AC_projection 1.05× 5.13 → 5.13 GiB
AC2_projection 1.04× 11.57 → 11.57 GiB
calc_galerkin 1.06× 5.14 → 5.14 GiB

The per-run times of both overlap at χ = 256.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.06780% with 21 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
.../algorithms/derivatives/hamiltonian_derivatives.jl 94.11% 16 Missing ⚠️
src/algorithms/derivatives/mpo_derivatives.jl 88.88% 4 Missing ⚠️
src/environments/finite_envs.jl 95.65% 1 Missing ⚠️
Files with missing lines Coverage Δ
src/algorithms/changebonds/optimalexpand.jl 98.90% <100.00%> (+0.02%) ⬆️
src/algorithms/groundstate/dmrg.jl 93.10% <100.00%> (+0.65%) ⬆️
src/algorithms/toolbox.jl 97.91% <100.00%> (+0.02%) ⬆️
src/environments/finite_envs.jl 97.50% <95.65%> (+2.76%) ⬆️
src/algorithms/derivatives/mpo_derivatives.jl 80.46% <88.88%> (+0.99%) ⬆️
.../algorithms/derivatives/hamiltonian_derivatives.jl 93.65% <94.11%> (-4.61%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lkdvos and others added 14 commits October 27, 2026 08:43
Keep full-MPO environment transfers and cache independent effective-Hamiltonian sides and fixed Jordan data for the duration of the solve. Advance environments at each local window and prepare only the two-site terms needed by plain DMRG2.

Consolidate cache ownership, constrained preparation types, shared operator storage, expansion handling, and regression coverage.
Add nearest-neighbor, sparse, and chemistry-surrogate workloads, scaling measurements, isolated operator-construction accounting, profiles, and numerical agreement checks.
Collect full-MPO transfer comparisons, sweep and construction timing, CPU and allocation profiles, GC accounting, and pair-only preparation results with numerical agreement checks.
Add isolated snapshot orchestration, shared serialized inputs, dependency and compilation audits, complete-call and four-sweep timings, GC-aware stage profiles, state validation, and reproducible raw results.
…fy the cache

The cached AC2 preparation contracted the full GL*A (and A*GR) intermediate with
B (and C). Block contractions permute every stored channel block of that large
intermediate, making AB and CA 3-4x slower than the original constructor, which
contracts the small operator-only products A1*B2 and C1*A2 first. Cache those
products per pair and contract them with GL/GR, and build the restricted
continuing terms directly. One cached preparation now costs at most one
original construction, so the cache does half the work per sweep.

Also simplify the cache: records keep only the prepared sides (unprepared
queries rebuild from GL/GR), DMRG with expansion caches one-site contributions
only, and the local_only refresh and shared-intermediate plumbing are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated outputs (CSVs, figures, profiles, results write-ups, metadata) now live
in the ignored benchmark/results directory. Remove the one-off investigation
scripts and the remainder/profile leftovers in dmrg_cache.jl, keeping the shared
models, the ordinary-versus-cached sweep benchmark, and the main comparison
harness with its frozen manifest and self-contained setup instructions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…riven absorbs

- Replace the one_site/two_site flags with nsites; records hold a single
  contribution field, consistently typed as Union{Missing, P}.
- Drop JordanOperatorData and the per-site JordanSiteData: the cache keeps only
  the operator-only pair products (built for DMRG2 only) and side builders take
  the JordanMPOTensor directly, like the original constructors.
- Replace the per-algorithm absorb schedule with center_window!, which absorbs
  the neighbours of a window only if they changed since the records were built.
  The cache tracks its own dependencies, since read-only callbacks may refresh
  the ordinary manager's. Absorb counts and results match the previous schedule.
- leftenv/rightenv use something(), restoring type stability with a clear error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…action tests

- Build the fresh Jordan AC/AC2 constructors from the same left/right halves the
  sweep cache prepares, removing the duplicated term construction.
- Move the sweep cache into src/environments/dmrg_sweep_cache.jl.
- Rebuild the cache with a warning when a finalizer modifies the state or
  environments, instead of asserting.
- Test cached effective Hamiltonians against fresh environments at every local
  window and ground-state energies against exact diagonalization; check operator
  reuse structurally instead of with an instrumented backend.
- Stop tracking the benchmark scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Generic MPO derivatives prepare their left and right halves through
  `left_half`/`right_half`, dispatched on the derivative type, as the Jordan
  derivatives do.
- Rename the sweep cache to `DMRGEnvironments` (src/environments/dmrg_envs.jl).
  It keeps only the prepared halves and reads `GL`/`GR` from its parent
  `FiniteEnvironments`; site transfers are `transfer_leftenv!`/`transfer_rightenv!`
  as for the infinite environments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each prepared half is rebuilt when queried if the environment it was prepared
from has been replaced; the environments themselves come from the parent
`FiniteEnvironments` and its dependency tracking. This removes the explicit
window updates, the separate dependency vectors and the finalizer check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FiniteEnvironments now stores the prepared left and right halves of the
AC and AC2 effective Hamiltonians, with an interface `lefthalf`/`righthalf`
on top of `leftenv`/`rightenv`. Halves are prepared lazily per kind and
rebuilt when the environment they were prepared from is replaced, so every
algorithm on finite environments reuses them and DMRGEnvironments, together
with its wrapping in the DMRG driver, is removed. The storage types are
derived from the environment and operator types.

The Jordan MPO derivatives no longer require the bra and ket to be equal:
the not-started left and finished right environments are kept as explicit
boundary tensors when they are not identities.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calc_galerkin takes `prepare = false`; with `prepare = true` and environments that cache prepared
halves, the single-site derivative is assembled from those halves, which the effective
Hamiltonian at that site reuses when they are still current (after bond expansion, the half on
the unchanged side). Single-site DMRG passes `prepare = true` and otherwise matches main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each half of a JordanMPO derivative now folds its onsite, not-started and finished
terms into its own largest term when it is built, so prepare_operator!! only prepares
the continuing term, and the fresh and cached constructions share the same builders.
The half helpers dispatch on the window size Val{N} and are named after their callers
(prepare_operator_left/right, jordan_hamiltonian_left/right), the four
_*_hamiltonian_from_halves methods become hamiltonian_from_halves, and the Galerkin
error in DMRG is computed inline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With OptimalExpand, single-site DMRG takes the Galerkin error from the projected two-site update
the expansion forms anyway, contracted with the isometry on the other site of the bond, instead of
building the single-site derivative separately. The expansion itself is split off as
expand_from_projection!.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Folding the onsite, not-started, finished and operator-only terms into the largest term of
each half allocates a temporary of that term's size per fold, and only pays off for an
operator that is applied repeatedly. Unprepared derivatives (`AC_projection`,
`AC2_projection`, `calc_galerkin`) now keep these terms separate, as before the halves were
introduced, while the cached halves and `prepare = true` still fold them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant