Skip to content

fix(lpse2d): interpolated field save on a coarser x-space grid; saved ex/ey without the stray 4/(nx ny) - #388

Merged
physicistphil merged 3 commits into
ergodicio:mainfrom
physicistphil:fix/lpse2d-interpolated-field-save
Oct 21, 2026
Merged

physicistphil merged 3 commits into
ergodicio:mainfrom
physicistphil:fix/lpse2d-interpolated-field-save

Conversation

@physicistphil

@physicistphil physicistphil commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Defect

Two defects in the saved EPW fields (fields.xr / k-fields.xr). Neither affects the solver, the in-solver series or the scalar metrics.

A. Interpolated save (save.fields.x.dx / save.fields.y.dy set; get_save_quantities → save_func, make_field_xarrays). Measured on ce5d3fe: 6.4 × 1.6 µm TPD box, 0.05 µm grid, 0.08 µm save spacing, 20 fs, compared against the full-grid save of the same run sampled at the save points.

  1. No coarsening. The save grid kept the simulation's nx × ny points, spread between xmin + dx_save/2 and xmax - dx_save/2; dx_save only set the margins.
  2. EPW saved wrongly. The epw state is the k-space potential phi_k, but it went through the x-space interpolator, and make_field_xarrays then read it as k-space on kx = fftfreq(nx, d=dx_save). Saved phi, ex, ey differed from the full-grid save by 68 %, 67 %, 73 % of their maximum, and the k axis was scaled by dx/dx_save (×0.625).
  3. NaN edges. With dx_save below the grid spacing, the outermost save points fell outside the outermost cell centres, where interpax returns NaN.
  4. E0, E1 and the IAW fields were interpolated correctly (to ~3e-6), but onto the wrong grid.

B. Stray 4 / (nx ny) on saved ex/ey (every save, full-grid and interpolated). make_field_xarrays wrote ex = ifft2(-i kx phi_k) / nx / ny * 4 (ey likewise), while the saved phi is ifft2(phi_k) unscaled and the solver's field is E = ifft2(-i k phi_k) (SpectralEPWSolver.phi_k_to_e_fields; the IAW ponderomotive term, TPD pump depletion and the e_sq / epw_energy / epw_dissipation series use the same unscaled E). fields.xr ex/ey were therefore smaller than the solver's field, and than −∇ of the saved phi, by 4 / (nx ny): 1/1024 on 128 × 32, 1/32 on 128 × 1, ~1/14000 on 512 × 108. k-fields.xr was already unscaled, so the x- and k-space datasets disagreed by the same factor. The factor dates from 02e112f (PR #20), where the first EPW solver stored phi with fft_norm = nx ny / 4; f77784f (PR #41) and c641495 (PR #142) changed the state convention and kept it.

Scope

152 of 177 envelope-2d configs (adept configs/, tests/, and the srs-campaign repo) set save.fields.x/y, so their saved EPW fields (phi, ex, ey, and their k-space versions) are affected by A. None of them sets a save spacing finer than the grid, so none is refused by the new check. B affects fields.xr ex/ey of every lpse2d run.

Fix

  • Save grid: n_save = round(L / dx_save) cell-centred points per axis, spacing L / n_save. A save grid with more points than the simulation grid raises a ValueError at setup. Giving only one of save.fields.x / save.fields.y raises (y alone used to be silently ignored). Quasi-1D (ny = 1) keeps a single row.
  • Save function: phi_k is transformed to x-space with the solver's convention (phi = ifft2(phi_k), E = ifft2(-i k phi_k)) on the full grid, and phi, ex, ey are interpolated onto the save grid. E0, E1 and the IAW fields are interpolated as before. E comes from the full grid, not from a spectral derivative on the save grid, which would scale the interpolation error by k and assign the wrong k to content above the save grid's Nyquist wavenumber.
  • make_field_xarrays: x-space ex/ey are ifft2(-i k phi_k), i.e. −∇ of the saved phi, in both saves — the 4 / (nx ny) factor is dropped (commit 9eb7ff5, cherry-picked from fix/lpse2d-saved-epw-field-scale). k-space fields of the interpolated save are the DFT on the save grid, on its own k axes, scaled by (nx·ny)/(nx_save·ny_save) so a resolved mode keeps its full-grid coefficient.
  • Full-grid saves: at 19cbe2e (A only), fields.xr / k-fields.xr were bitwise identical to ce5d3fe (x64 and float32, 2D and quasi-1D). 9eb7ff5 changes only the ex/ey lines, so full-grid saves differ from ce5d3fe in fields.xr ex/ey only, by the factor nx ny / 4.

Behavior change

  • Interpolated saves now have n_save points per axis instead of nx × ny.
  • The raw diffrax fields save holds phi, ex, ey (x-space, on the save grid) instead of epw. Its only reader, make_field_xarrays, is updated, and the dataset variable names are unchanged.
  • A save spacing finer than the grid now raises instead of producing NaN edge rows.
  • fields.xr ex and ey grow by nx ny / 4 relative to every earlier save (all configs, full-grid and interpolated). To compare with an earlier run, divide its ex/ey by 4 / (nx ny). phi, k-fields.xr of full-grid saves, series.xr and the logged metrics are unaffected.
  • docs/source/solvers/lpse2d/config.md and overview.md are updated.

Tests

Tolerances are fixed in the docstrings.

tests/test_lpse2d/test_field_save.py (A; all 6 fail on ce5d3fe):

  • test_interpolated_save_matches_full_grid_save[2d, 1d]: TPD+SRS+IAW box, 6 fs, save spacing 1.6 × grid. Every x-space field equals the full-grid field linearly interpolated to the save points, within 2000 eps of its maximum (round-off budget derived in the file; measured ≤ 2.3 eps). Also checks shapes 80×20 / 80×1, save and k axes to 1e-13, and that every value is finite.
  • test_interpolated_save_of_band_limited_state[2d, 1d]: single-mode state checked against analytic x-space fields (ex = -i kx phi, ey = -i ky phi, no factor since 9eb7ff5) and the mode's k-space coefficient, within the bilinear bound √2 (q h)²/8 plus round-off.
  • test_interpolated_save_spacing_bounds[x, y]: half the grid spacing raises; equal spacing reproduces the grid axis bitwise with finite output; x or y alone raises.

tests/test_lpse2d/test_saved_epw_fields.py (B; all 4 fail on ce5d3fe; 128 × 32 and quasi-1D 128 × 1, full-grid save, round-off bound 200 eps fixed before the first run):

  • test_saved_epw_fields_single_mode[2d, 1d]: fields.xr and k-fields.xr phi/ex/ey equal the analytic single mode.
  • test_saved_epw_fields_are_the_solver_field[2d, 1d]: saved ex/ey equal −∇(saved phi), SpectralEPWSolver.phi_k_to_e_fields(phi_k) and ifft2 of the k-fields.xr ex/ey, and dx dy Σ|E|² of the saved fields equals the e_sq series. On ce5d3fe the relative L2 error is 0.99902 = 1 − 4/4096 (2D) and 0.96875 = 1 − 4/128 (quasi-1D).

lpse2d suite (uv run --locked pytest tests/test_lpse2d, as CI) at 9eb7ff5 on macOS: 80 passed / 1 skipped / 3 deselected.

Independent of the LPSE parity stack (#372 / #378 / #387), which also edits helpers.py; whichever merges second will need a conflict resolution in get_save_quantities / make_field_xarrays.

🤖 Generated with Claude Code

…space

With save.fields.x.dx / save.fields.y.dy set, the interpolated field save
(get_save_quantities -> save_func, make_field_xarrays) was wrong in four
ways (measured on ce5d3fe: 6.4 x 1.6 um TPD box, 0.05 um grid, 0.08 um
save spacing, 20 fs, compared against the full-grid save of the same run
sampled at the save points):

1. The save grid was not coarser. It kept the simulation's nx x ny
   points, spread between xmin + dx_save/2 and xmax - dx_save/2, so
   dx_save only set the margins.
2. The saved epw state is the k-space potential phi_k (FFT-ordered), but
   it went through the same x-space interpolator as the real-space
   fields, and make_field_xarrays then read the result as k-space on
   kx = fftfreq(nx, d=dx_save). The saved phi, ex and ey differed from
   the full-grid save by 68 %, 67 % and 73 % of their maximum, and the
   saved k axis was scaled by dx / dx_save (x0.625).
3. With dx_save below the grid spacing, the outermost save points fell
   outside the outermost cell centres, where interpax returns NaN.
4. E0, E1 and the IAW fields were interpolated correctly, but onto the
   wrong (non-coarsened) grid.

Fix:
- The save grid has n_save = round(L / dx_save) cell-centred points per
  axis, with spacing L / n_save (_interpolated_save_axis). A save grid
  with more points than the simulation grid raises a ValueError at
  setup. Giving only one of save.fields.x / save.fields.y raises
  NotImplementedError (x alone already did; y alone was silently
  ignored).
- The save function (_interpolated_field_save) transforms phi_k to
  x-space with the solver's convention (phi = ifft2(phi_k),
  E = ifft2(-i k phi_k)) on the full grid and saves phi, ex and ey on the
  save grid. E0, E1 and the real fields are interpolated as before. ex
  and ey come from the full grid rather than from a spectral derivative
  on the save grid, because that derivative would scale the
  interpolation error by k and assign the wrong k to content above the
  save grid's Nyquist wavenumber.
- make_field_xarrays, interpolated branch: the x-space ex and ey carry
  the full-grid save's 4 / (nx * ny) factor with the simulation nx, ny.
  The k-space fields (phi, ex, ey, IAW) are the DFT on the save grid, on
  its own k axes, scaled by (nx * ny) / (nx_save * ny_save) so that a
  resolved mode keeps its full-grid coefficient.
- The full-grid save path is unchanged. fields.xr and k-fields.xr of a
  full-grid run are bitwise identical to ce5d3fe (x64 and float32, 2D and
  quasi-1D, TPD+SRS+IAW box).

Behavior change: interpolated saves now have n_save points per axis
instead of nx x ny, and the raw diffrax "fields" save holds phi, ex and
ey (x-space, on the save grid) instead of epw.

Tests (tests/test_lpse2d/test_field_save.py; all 6 fail on ce5d3fe):
- test_interpolated_save_matches_full_grid_save[2d, 1d]: noise-seeded
  TPD+SRS+IAW box with a static pump, 6 fs, save spacing 1.6 x the grid
  spacing. Every x-space field equals the full-grid field linearly
  interpolated to the save points, within 2000 eps of its maximum
  (round-off budget derived in the file; measured <= 2.3 eps). Shapes are
  80 x 20 (2D) / 80 x 1 (quasi-1D), save and k axes match the save grid
  to 1e-13, and all values are finite.
- test_interpolated_save_of_band_limited_state[2d, 1d]: a single-mode
  state checked against analytic values: the x-space phi, ex, ey, E0, E1
  and IAW fields and the mode's k-space coefficient, each within the
  bilinear-interpolation bound sqrt(2) (q h)^2 / 8 plus round-off.
- test_interpolated_save_spacing_bounds[x, y]: half the grid spacing
  raises. Equal spacing reproduces the grid axis bitwise and gives finite
  output equal to linear interpolation within 2000 eps. Giving x or y
  alone raises.

Docs: docs/source/solvers/lpse2d/config.md describes the save grid, the
spacing bound, the interpolation and the normalizations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@ergodic-reviewer ergodic-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No material findings in the complete diff at 19cbe2e6fb4676f2f31ddf7d9afb9ba910720d47.

Postprocessing coverage: the interpolated phi/ex/ey outputs feed the updated make_field_xarrays, the existing field/spectral plots, and persisted fields.xr / k-fields.xr artifacts through the ergoExo lifecycle. Default-series diagnostics remain separate. The added tests cover full-grid interpolation parity, analytic mode coefficients, quasi-1D saves, and spacing validation.

Validation: static review only; I did not execute tests or inspect generated plots/artifacts. The test-lpse2d CI check is still in progress.

make_field_xarrays wrote the x-space EPW fields of fields.xr as
ex = ifft2(-i kx phi_k) / nx / ny * 4 (ey likewise), while the saved phi
is ifft2(phi_k) unscaled and the solver's field is E = ifft2(-i k phi_k)
(SpectralEPWSolver.phi_k_to_e_fields; the IAW ponderomotive term, the TPD
pump depletion and the e_sq / epw_energy / epw_dissipation series use the
same unscaled E). fields.xr ex/ey were therefore smaller than the field
the solver evolves, and than -grad of the saved phi, by 4 / (nx ny):
1/1024 on a 128 x 32 grid, 1/32 on 128 x 1, ~1/14000 on 512 x 108.
k-fields.xr (phi_k, -i kx phi_k, -i ky phi_k) was already unscaled, so
the x- and k-space datasets disagreed by the same factor.

History: the factor dates from 02e112f (PR ergodicio#20), where the first EPW
solver stored phi in k-space with fft_norm = nx ny / 4 (its get_eh_x
multiplied by nx ny / 4; the save used the reciprocal 4 / (nx ny)).
f77784f ("new units and solver based on matlab code", squashed into
df1510a / PR ergodicio#41) moved the state to the plain numpy FFT convention but
kept the factor on ex/ey, and c641495 (PR ergodicio#142) moved the state to the
k-space phi_k and kept it again. The MATLAB prototype has no such factor;
in the current code it has no physical meaning.

Fix: drop the factor. The x-space ex/ey are ifft2(-i k phi_k), i.e.
-grad(phi) of the saved phi. Both branches of make_field_xarrays lose
it: the full-grid one, and the interpolated (save.fields.x / y) one of
the previous commit, which had applied the same factor with the
simulation nx, ny to its x-space ex/ey so the two saves agreed.
k-fields.xr, phi, the light fields and the IAW fields are unchanged.
Plots in plot_fields / plot_kt use autoscaled colour limits, no
normalization assumed the scaled values. docs/source/solvers/lpse2d/
overview.md now states what the saved EPW entries are.

Tests (tests/test_lpse2d/test_saved_epw_fields.py, 4 tests, all fail on
ce5d3fe; 128 x 32 and quasi-1D 128 x 1 boxes, full-grid save, a single
grid mode plus a random band-limited phi_k, round-off bound 200 eps fixed
before the first run):
- test_saved_epw_fields_single_mode[2d, 1d]: fields.xr phi/ex/ey and
  k-fields.xr phi/ex/ey equal the analytic single mode. On ce5d3fe ex is
  off by 5.9e-3 of a 5.9e-3 field.
- test_saved_epw_fields_are_the_solver_field[2d, 1d]: saved ex/ey equal
  -grad(saved phi) (spectral), SpectralEPWSolver.phi_k_to_e_fields(phi_k)
  and ifft2 of the k-fields.xr ex/ey, and dx dy sum |E|^2 of the saved
  fields equals the e_sq series. On ce5d3fe the relative L2 error is
  0.99902 = 1 - 4/4096 (2D) and 0.96875 = 1 - 4/128 (quasi-1D).
- test_field_save.py::test_interpolated_save_of_band_limited_state now
  expects ex = -i kx phi, ey = -i ky phi on the save grid (factor
  removed from its expected values and docstring; tolerances unchanged).
  test_interpolated_save_matches_full_grid_save needs no change: it
  compares the interpolated save with the full-grid save.

Cherry-picked from fix/lpse2d-saved-epw-field-scale (6b6d278, off
ce5d3fe); the conflict in make_field_xarrays is resolved as above.

Behavior change: fields.xr ex and ey grow by nx ny / 4 relative to every
earlier save (all configs, full-grid and interpolated). phi, k-fields.xr,
series.xr and the logged metrics are unaffected (none read fields.xr
ex/ey).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@physicistphil physicistphil changed the title fix(lpse2d): put the interpolated field save on a coarser grid, in x-space fix(lpse2d): interpolated field save on a coarser x-space grid; saved ex/ey without the stray 4/(nx ny) Oct 20, 2026
@physicistphil
physicistphil marked this pull request as draft October 20, 2026 23:07
@physicistphil
physicistphil marked this pull request as ready for review October 20, 2026 23:07

@ergodic-reviewer ergodic-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking implementation defects found in the complete diff at 9eb7ff59414746b2d6a1cede8fb224bc9320d9b0. One non-blocking documentation correction is noted inline: the overview's Fourier identities need to distinguish full-grid from interpolated saves.

Postprocessing coverage: the saved fields feed make_field_xarrays, the existing spatial/spectral plots, and persisted fields.xr / k-fields.xr artifacts through ergoExo; default-series diagnostics remain separate. Added tests cover interpolation parity, analytic modes, quasi-1D grids, spacing validation, persisted-field round trips, and agreement with the solver fields and e_sq.

Validation: static review only; I did not execute tests or inspect generated plots/artifacts. GitHub's test-lpse2d check reports success.

Comment thread docs/source/solvers/lpse2d/overview.md Outdated
…aves

The overview stated that k-fields.xr holds phi_k, -i kx phi_k, -i ky phi_k
and that fields.xr is their ifft2, with ex, ey = -grad(phi). That holds
for a full-grid save only. An interpolated save (save.fields.x / y)
stores the full-grid fields interpolated to the save points, and its
k-fields.xr is their DFT on the save grid scaled by
(nx ny) / (nx_save ny_save); neither ifft2(k-fields.xr) nor a derivative
of the coarse phi reproduces its fields.xr. The paragraph now says so and
links to the save.fields reference in config.md.

Review thread r4331320857 (PR ergodicio#388). Docs only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@ergodic-reviewer ergodic-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No material findings in the complete diff at 984f0f63e85eacce7099d11d4cb74804cc1e2085. The overview correction addresses the earlier feedback by distinguishing full-grid Fourier identities from interpolated-save conventions.

Postprocessing coverage: the changed saved fields feed make_field_xarrays, the existing spatial/spectral plots, and persisted fields.xr / k-fields.xr artifacts through ergoExo. Default-series diagnostics remain separate. Added tests cover interpolation parity, analytic modes, quasi-1D grids, spacing validation, persisted-field round trips, and agreement with the solver fields and e_sq.

Validation: static review only; I did not execute tests or inspect generated plots/artifacts. The current head's test-lpse2d CI check is still in progress.

@physicistphil
physicistphil merged commit b5d6cde into ergodicio:main Oct 21, 2026
15 checks passed
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 22, 2026
…into lpse2d/lpse-parity

Brings the interpolated field-save fix and the saved-EPW ex/ey scale fix
(and the main commits under 984f0f6). helpers.py conflicts resolved by
keeping both sides' new helpers; the interpolated save is ergodicio#388's
_interpolated_field_save, wrapped with this branch's combined-mode
Raman-light projection (raman_light_part) and the epw_ledger skip, as the
full-grid save does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 22, 2026
…finer than the test grid

After the PR ergodicio#388 merge the interpolated field save refuses a save
spacing finer than the simulation grid (it used to write non-finite edge
rows). Five tests build a 0.1 um grid from tpd.yaml, whose save.fields
x / y spacing is 80 nm / 128 nm, and failed in setup with that refusal:
test_combined_split_step_runs_with_pump_depletion_iaw_noise_and_reports_transverse_light,
test_poynting_of_a_plane_wave_is_group_velocity_times_intensity,
test_thomson_probe_picks_the_mode_in_its_window,
test_ledger_reported_in_epw_energy_units,
test_ledger_off_leaves_state_and_series_unchanged.
They test diagnostics on full-grid states (the old interpolated save kept
nx x ny points, so those happened to fit), so their configs now drop
save.fields.x / y; no assertion changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 22, 2026
…lpse-parity-plan2

Brings PR ergodicio#388 (interpolated field save on a coarser grid, saved EPW ex/ey
without the stray 4/(nx ny)) via lpse2d/lpse-parity. helpers.py conflicts
resolved toward plan2's additions on ergodicio#388's code:
- make_field_xarrays: ergodicio#388's unscaled ex / ey with plan2's three-component
  light fields (e0z / e1z).
- interpolated save: ergodicio#388's _interpolated_field_save, with plan2's skipped
  keys (fd IAW super-sampled fields, iaw_density_old, epw_ledger) and the
  QLE vdf / gamma_L saved verbatim.
- Needed by the coarser save grid: QLE's gamma_L enters the k-fields at
  the save grid's k modes (the simulation-grid modes of the same integer
  frequency, _grid_index_of_save_modes); with ergodicio#388's coarse k axes the
  full-grid gamma_L no longer matched them and make_field_xarrays raised
  (256 vs 160 modes). Test: test_qle_rate_is_saved_on_an_interpolated_save_grid
  (exact; fails without the index map).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 22, 2026
…rum probes on this branch

- test_field_save.py (from PR ergodicio#388): its box ran TPD and SRS together on
  the separate EPW solver, which this branch refuses (as LPSE); SRS is now
  off (TPD + IAW; the combined solver would save E1 as its transverse
  part, which the analytic longitudinal E1 would not survive). The saved
  field set includes the out-of-plane e0_z / e1_z this branch's state
  carries (plan 2 F.1); tolerances unchanged.
- test_light_spectrum.py combined probes: full-grid field save (tpd.yaml's
  0.08 um save spacing is finer than their grid, refused since PR ergodicio#388).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 22, 2026
…eat/lpse2d-hpe-spatial-fv

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 22, 2026
…ons) into feat/lpse2d-hpe-spatial-fv

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
physicistphil added a commit to physicistphil/adept that referenced this pull request Oct 23, 2026
PR ergodicio#388's interpolated save accepts a save grid of one x point
(n_save = max(1, round(L / dx_save)), e.g. save.fields.x.dx the box
length). With save.fields.poynting the flux maps take d/dx on the save
grid, and np.gradient raised in make_field_xarrays after the solve,
before the series, checkpoint and artifacts were written (PR ergodicio#372
review). get_save_quantities now refuses that combination at setup;
two or more x points are accepted. Docs: the save.fields.poynting text.

Test: test_poynting_refuses_a_single_point_x_save_grid (dx = box length
refused, half the box accepted with 2 points). Before: did not raise.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant