Repository navigation
Fix issues #179, #181-#185, #188-#190 - #204
Merged
asalmgren merged 1 commit intoSep 27, 2026
Merged
Conversation
…-Fluids#188-AMReX-Fluids#190 AMReX-Fluids#179 NS_derive: the EB one-sided vorticity stencils in dermgvort read velocity two cells away without checking those cells are uncovered -- isConnected only certifies the immediate neighbor. Certify the intermediate cell before taking the quadratic form, drop to a two-point difference when only one cell is available, and leave the derivative at zero when both sides are disconnected (2D x/y and 3D x/y/z). AMReX-Fluids#181 ViscBench: the loop over global grid indices dereferenced MultiFabs by global index and FillVar(FArrayBox*,...) fills on one rank only, so the tool was broken under MPI. Use the distributed FillVar(MultiFab&,...) overload, drive the exact-solution loop from MFIter, and print through amrex::Print() so the collective norm calls stay on every rank. AMReX-Fluids#182 getForce: the getForceVerbose min/max loops indexed State and force by absolute state component, reading past the end of the velocity-only State FAB and the 0-indexed force FAB that scalar callers pass. Index each FAB by its own layout; the printed label keeps the absolute component. AMReX-Fluids#183 main: the regrid-on-restart test lacked the max_step < 0 guard that the time loop below applies, so with only stop_time set it regridded and wrote a spurious plotfile/checkpoint before stepping anyway. AMReX-Fluids#184 HIT fast force: the coarse hi bound was one node short whenever ihi mod ff_factor was in [1, ff_factor-2], so the interpolation read past ff_force. Use ihi/ff_factor + 1, which is exactly the node the kernel touches, and give ff_force an Elixir so its device memory is not freed while the asynchronous kernels are still reading it. AMReX-Fluids#185 TurbulentForcing: nothing constrained the mode indices writing into the flat stack array tmp, so large nmodes or a high-aspect domain corrupted the stack; assert the mode counts fit array_size. The GPU copy out of tmp was asynchronous with no sync before the stack frame died; use the synchronous htod_memcpy. Also fixes the verbose block, which printed nxmodes in place of nymodes and nzmodes. AMReX-Fluids#188 init_ConvectedVortex: cases meanFlowDir = +/-2 assigned vel_x = v_vort and vel_y = u_vort, giving a non-solenoidal, essentially irrotational field instead of a vortex. Un-swap them so only the mean flow term moves to the y component. Make "no mean flow" an explicit case 0 and abort on the host for out-of-range directions, which a bare default silently accepted. AMReX-Fluids#189 MacProj: the known-edgestate mac_sync_compute overload passed a default-constructed MultiFab as the state, which ComputeAofs indexes unconditionally; pass an alias of Sync instead. mac_sync_compute built the divu constraint at prev_time only, omitting the 0.5*dt*dsdt half-time correction that velocity_advection and scalar_advection apply, so the sync reconstructed edge states with a different divu than the advance used. test_umac_periodic did host-side FArrayBox work on device-resident data; give it pinned host copies. AMReX-Fluids#190 ScalMinMax: both limiters seeded the running maximum with numeric_limits<Real>::min(), the smallest positive normal rather than lowest(), so the upper clip bound was wrong for any all-negative stencil. With lowest() binding, skip covered cells so they keep the COVERED_VAL invariant. do_denminmax called ConservativeScalMinMax with Density as both the scalar and the normalizing density, clamping rho/rho == 1 against rhoo/rhoo == 1 -- a provable no-op; call ConvectiveScalMinMax on the raw density instead. The do_denminmax fix makes ns.do_denminmax=1 change results for the first time, so the eb_run2d and eb_run3d hotspot benchmarks need regolding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #179. Fixes #181. Fixes #182. Fixes #183. Fixes #184. Fixes #185. Fixes #188. Fixes #189. Fixes #190.
(#180 is not included -- it was already fixed on
developmentby c44cbcf.)What's in here
Source/NS_derive.cppdermgvortread velocity two cells away without checking those cells are uncovered --isConnectedonly certifies the immediate neighbor. Now the intermediate cell is certified before taking the quadratic form, with a two-point difference when only one cell is available and zero when both sides are disconnected (2D x/y, 3D x/y/z).Tutorials/TaylorGreen/benchmarks/ViscBench.cppFillVar(FArrayBox*,...)fills on one rank only, so the tool could not work under MPI. Switched to the distributedFillVar(MultiFab&,...)overload, drove the exact-solution loop fromMFIter, and moved the prints toamrex::Print()so the collectivenorm*calls stay on every rank.Source/NS_getForce.cppgetForceVerbosemin/max loops indexedStateandforceby absolute state component, reading past the end of the velocity-onlyStateFAB and the 0-indexedforceFAB that scalar callers pass. Each FAB is now indexed by its own layout; the printed label keeps the absolute component.Source/main.cppmax_step < 0guard the time loop applies, so with onlystop_timeset it regridded and wrote a spurious plotfile/checkpoint before stepping anyway.Tutorials/HIT/NS_getForce.cppihi mod ff_factorwas in[1, ff_factor-2], so the interpolation read pastff_force. Nowihi/ff_factor + 1, exactly the node the kernel touches (the threeihi == ff_ihi*ff_factoradjustments become dead and are removed).ff_forcealso gets anElixirso its device memory is not freed while the asynchronous kernels still read it.Tutorials/HIT/TurbulentForcing_def.Htmp, so largenmodesor a high-aspect domain corrupted the stack; the mode counts are now asserted to fitarray_size. The GPU copy out oftmpwas asynchronous with no sync before the stack frame died -- switched to the synchronoushtod_memcpy. Also fixes the verbose block, which printednxmodesin place ofnymodesandnzmodes.Source/prob/prob_init.cppmeanFlowDir = +/-2assignedvel_x = v_vortandvel_y = u_vort, giving a non-solenoidal, essentially irrotational field instead of a vortex; un-swapped so only the mean flow term moves to the y component. "No mean flow" is now an explicitcase 0, with a host-sideAbortfor out-of-range directions that the baredefaultsilently accepted.Source/MacProj.cppmac_sync_computeoverload passed a default-constructedMultiFabas the state, whichComputeAofsindexes unconditionally -- now an alias ofSync.mac_sync_computebuilt the divu constraint atprev_timeonly, omitting the0.5*dt*dsdthalf-time correction thatvelocity_advectionandscalar_advectionapply, so the sync reconstructed edge states with a different divu than the advance used.test_umac_periodicdid host-sideFArrayBoxwork on device-resident data; it now gets pinned host copies.Source/NavierStokesBase.cppnumeric_limits<Real>::min(), the smallest positive normal rather thanlowest(), so the upper clip bound was wrong for any all-negative stencil. Withlowest()binding, covered cells are skipped so they keep theCOVERED_VALinvariant.do_denminmaxcalledConservativeScalMinMaxwithDensityas both the scalar and the normalizing density, clampingrho/rho == 1againstrhoo/rhoo == 1-- a provable no-op; it now callsConvectiveScalMinMaxon the raw density.Heads-up: benchmarks need regolding
The
do_denminmaxfix makesns.do_denminmax=1change results for the first time (it was previously a bit-exact no-op).Exec/eb_run2d/regtest.2d.hotspotandExec/eb_run3d/regtest.3d.hotspotenable the flag, so their benchmarks will need to be regolded. Oneb_run2dhotspot at step 4, tracer max goes from 1.0012879887 to 1.0000060234 and density max from 1.0011269247 to 1.0011264227 -- i.e. the limiter now does its job.Build and test
Built clean:
run2d,run3d,eb_run2d,eb_run3d,Tutorials/HIT(bothUSE_FAST_FORCE=TRUEandFALSE), andViscBenchwithUSE_MPI=TRUE.All shipped regtests were run in 2D and 3D, EB and non-EB.
eb_run2d/regtest.2d.shock_past_cylinderaborts with "MLMG failed" -- this fails identically on unmodifieddevelopment, so it is pre-existing and unrelated.Targeted verification against baseline binaries built from
development:meanFlowDir=2now givesmax|u| = 0.0832,max|v| = 1.0832, the exact mirror ofmeanFlowDir=1.meanFlowDir=4aborts with the new message.max_step < 0guard, fires when only stop_time is set #183 -- baseline restart with onlystop_timeset writes 3 plotfiles and regrids level 0 twice; with the fix, 2 plotfiles and one regrid.turb.nmodes=40now trips the assertion instead of writing ~264 KB past the end of the stack array.turb.ff_factor=6(a remainder that was previously out of bounds), and tracks the exact-forcing build.mag_vortis sane in plotfiles (max 2619 in 2D, 2199 in 3D), no1e40values.ns.getForceVerbose=1, scalar calls printState comp 0/1from the velocity-only FAB instead of reading past it.mac_proj.check_umac_periodicity=1passes on a periodic 3D case under MPI.Plotfile comparisons vs. baseline at step 4:
poiseuille,traceradvect_bds,eb_double_shear_layerandeb_flow_past_cylinderare bit-identical.ns_hotspot/ns_hotspot_rzdiffer at 1e-12 / 1e-9 relative, from thedsdtterm in #189 (tiny here becausetemp_cond_coef = 1e-8).eb_hotspotdiffers at ~0.2%, the intended #190 change described above.Nothing here was run on GPU -- the GPU-specific parts of #184, #185 and #189 are compile-checked and reasoned, not executed on device.
🤖 Generated with Claude Code