Skip to content

evd: the real Schur deflation-window sort always ends (#144) - #151

Closed
shinaoka wants to merge 1 commit into
mainfrom
fix/144-aed-sort-termination
Closed

shinaoka wants to merge 1 commit into
mainfrom
fix/144-aed-sort-termination

Conversation

@shinaoka

Copy link
Copy Markdown
Member

Closes #144.

Problem

aggressive_early_deflation sorts the deflation window's eigenvalues with a bubble sort ported from LAPACK dlaqr3. The port had two faults:

  • It sorted the deflated part ns..jw. dlaqr3's initial I = NS + 1 only sets the first pass's end, and the port read it as the start of the sort.
  • It set each next window from the position of the last swap.

schur_swap reports success even when schur_swap_22 rejects an ill-conditioned 2 x 2 swap, so the pass recorded the same position every time and compared the same pair forever. Mat::eigen of the f32 matrix J_n + 1e-6 G, scaled to 2^-100, did not return at n = 128 and 257.

Fix

The sort follows dlaqr3 line by line:

  • It sorts the undeflated part infqr..ns.
  • Each pass ends at kend, one before where the previous pass stopped, so the sort ends after at most ns passes whatever the swaps do.
  • It swaps with schur_swap_checked (the dlaexc ports for 1 x 2 and 2 x 1), and a rejected swap leaves the pair in place, as dlaqr3 does after DTREXC fails.

After a successful swap, the next position i + n2 equals DTREXC's adjusted ILST for all four block-size pairs.

Tests

  • New tests::aed_sort_terminates_when_a_swap_fails: runs the reproducer at n = 128 and 257 through hessenberg_schur_real, each under a 120 s timeout. On origin/main it times out at n = 128; here it finishes in about 2 s. Both factors must be finite, and:
    • at 2^-100, the residual must be within LAPACK's smlnum = safmin * n / eps;
    • the same matrix at 2^-80 must have residual within 10 n eps |H|;
    • orthogonality must be within 10 n eps.
  • Separate observation, not changed here: at 2^-90 and below, the residual relative to |H| is large on origin/main too, and also with the sort disabled. The input there lies below smlnum, and ?HSEQR likewise does not scale its input. Mat::eigen scales its input since LAPACK validation: the nonsymmetric EVD fixtures exceed the GET22-style residual threshold (4 dtypes) #112.
  • cargo test --release -p t4a-faer --lib: 553 passed, 0 failed.
  • Ignored LAPACK validation under linalg::evd: 51 of 52 pass. The failure is the known c32 entry in scripts/lapack-known-failures.json (LAPACK validation: the nonsymmetric EVD fixtures exceed the GET22-style residual threshold (4 dtypes) #112); it goes through the complex Schur, which this PR does not touch.
  • Timing: none. This is a correctness fix; the sort now runs over the undeflated window and terminates.

Records

  • PATCHSET.md: new row (KEEP).
  • Upstream report: upstream-reports/faer-029-real-schur-aed-sort-hang. It reproduces on faer 0.24.4 (neither size returns within 60 s), and upstream main 0ecd846 has the same code.
  • Review: an independent review (gpt-6.1-sol) found no correctness defect in the rewrite. It flagged a duplicate report number and an overstated accuracy claim; both are fixed.

🤖 Generated with Claude Code

aggressive_early_deflation sorted the window with a bubble sort ported
from LAPACK dlaqr3, but over the deflated part (dlaqr3's initial
I = NS + 1 only sets the first pass's end) and with each next window
set from the last swap's position. schur_swap reports success even when
schur_swap_22 rejects an ill-conditioned swap, so the same pair was
compared forever: Mat::eigen of the f32 matrix J_n + 1e-6 G scaled to
2^-100 did not return at n = 128 and 257.

The sort now follows dlaqr3: it sorts the undeflated part infqr..ns,
ends each pass at kend, one before where the previous pass stopped, and
swaps with schur_swap_checked (the dlaexc ports for 1 x 2 and 2 x 1),
leaving a rejected pair in place. Upstream faer 0.24.4 has the same
code; the upstream report is faer-029.

Closes #144

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shinaoka

Copy link
Copy Markdown
Member Author

Combined into #154 together with the other two nonsymmetric EVD fixes (#148, #151, #152), as the maintainer directed on 2026-10-29. The commits are carried over with their authorship; the upstream reports are renumbered faer-029 (#112), faer-030 (#144) and faer-031 (#145).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

real Schur: the deflation-window sort never terminates when a 2 x 2 swap fails

1 participant