Repository navigation
Add submesoscale eddy parametrization - #438
Conversation
|
The parametrization unit tests are passing. However, after turning it on in the baroclinic channel polaris test with the default parameters, I see NaNs after about 3 hours of simulation time, so there is still some debugging to be done. Moreover, user and developer documentation needs to be added. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a new submesoscale eddy parameterization module (Fox-Kemper
et al. 2011) into Omega’s ocean model, wires it into initialization/finalization
and tendency/auxiliary-state workflows, and adds a dedicated unit test plus
broad test-harness updates to support new analytic-field helper signatures.
Changes:
- Add
SubmesoEddiesmodule (config-driven) and integrate it into ocean module
init/finalize and auxiliary-state computation. - Introduce
TransportAuxVars::NormalTransportVelocityand use it in thickness
and tracer tendency computations as the transport velocity (optionally
including submesoscale-induced velocity). - Update unit test utilities and many tests to accommodate updated analytic
field setters; addSubmesoEddiesTest.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| components/omega/test/timeStepping/TimeStepperTest.cpp | Initialize/destroy SubmesoEddies in the time stepper test harness. |
| components/omega/test/ocn/TendencyTermsTest.cpp | Update analytic-field lambdas to match updated setScalar/setVectorEdge signatures. |
| components/omega/test/ocn/TendenciesTest.cpp | Initialize/destroy SubmesoEddies; update analytic-field lambdas and setScalar/setVectorEdge call signatures. |
| components/omega/test/ocn/SubmesoEddiesTest.cpp | New unit test covering mixed-layer depth, buoyancy gradient, and eddy velocity computations. |
| components/omega/test/ocn/OceanTestCommon.h | Extend setScalar/setVectorEdge helpers to pass element indices and optional Z coordinates. |
| components/omega/test/ocn/HorzOperatorsTest.cpp | Update analytic-field lambdas to match updated helper signatures. |
| components/omega/test/ocn/AuxiliaryVarsTest.cpp | Update analytic-field lambdas to match updated helper signatures. |
| components/omega/test/ocn/AuxiliaryStateTest.cpp | Initialize/destroy SubmesoEddies; update analytic-field lambdas and helper call signatures. |
| components/omega/test/CMakeLists.txt | Register new SUBMESOEDDIES_TEST. |
| components/omega/src/timeStepping/ForwardBackwardStepper.cpp | Pass tracer array into thickness tendency computation. |
| components/omega/src/ocn/Tendencies.h | Extend thickness tendency API to accept a tracer array (for aux-state computations). |
| components/omega/src/ocn/Tendencies.cpp | Route thickness/tracer tendencies through new aux-state compute routines and use NormalTransportVelocity. |
| components/omega/src/ocn/SubmesoEddies.h | New SubmesoEddies public interface. |
| components/omega/src/ocn/SubmesoEddies.cpp | New implementation and IO field registration for SubmesoEddies. |
| components/omega/src/ocn/OceanInit.cpp | Initialize SubmesoEddies during ocean module init. |
| components/omega/src/ocn/OceanFinal.cpp | Destroy SubmesoEddies during ocean finalize. |
| components/omega/src/ocn/auxiliaryVars/TransportAuxVars.h | New aux-var container for NormalTransportVelocity. |
| components/omega/src/ocn/auxiliaryVars/TransportAuxVars.cpp | Register/unregister IO field for NormalTransportVelocity. |
| components/omega/src/ocn/AuxiliaryState.h | Add TransportAuxVars member and new aux compute entry points. |
| components/omega/src/ocn/AuxiliaryState.cpp | Compute Brunt–Väisälä frequency, compute transport velocity (optionally + submeso), and use it for vertical pseudo-velocity. |
| components/omega/configs/Default.yml | Add Omega.Submeso configuration group and defaults. |
| KOKKOS_LAMBDA(int IEdge, const TeamMember &Team) { | ||
| const int KMin = MinLayerEdgeBot(IEdge); | ||
| const int KMax = MaxLayerEdgeTop(IEdge); | ||
| const int KRange = vertRangeChunked(KMin, KMax); | ||
|
|
||
| parallelForInner( | ||
| Team, KRange, INNER_LAMBDA(int KChunk) { | ||
| LocPseudoThicknessAux.computeVarsOnEdge( | ||
| IEdge, KChunk, PseudoThick, NormalVelEdge); | ||
| }); |
| /// Destroy instance of SubmesoEddies | ||
| void SubmesoEddies::destroyInstance() { | ||
| delete Instance; | ||
| Instance = nullptr; | ||
| } |
1d3f283 to
9825eda
Compare
TestingCTest unit tests
Polaris
|
|
The baroclinic channel test is now running stably with the submesoscale eddy parametrization turned on. This PR is ready for review. However, I haven't checked that the parametrization has the intended effect, and I'm not sure that I'm the best person to do that. @vanroekel could you help with this kind of testing ? |
|
@mwarusz I ran a couple tests with baroclinic channel and it's too hard to tell what the effects must be. We need to add a mixed layer. Let me look and see how straight forward that might be. |
|
update - I have a mixed layer variant of baroclinic channel I have a test run in the queue on frontier. Hopefully will have results by Monday |
| PseudoThickML, GradBuoyML, BVFreqML); | ||
|
|
||
| GradBuoyML /= PseudoThickML; | ||
| BVFreqML /= PseudoThickML; |
There was a problem hiding this comment.
@mwarusz for small ML this could be be problematic, it's not clear above in the MLD code if there is a floor. maybe it's not an issue though
There was a problem hiding this comment.
I agree that we should probably add a tiny value here. MPAS-O has it.
| Team, Range{MinLyrEdgeBot, IndexMLEdge}, | ||
| INNER_LAMBDA(int K, Real &AccumThick, Real &AccumGradBuoy, | ||
| Real &AccumBVFreq) { | ||
| const Real PseudoThickKm1 = |
There was a problem hiding this comment.
If your function starts at MinLyrEdgeBot can this lead to values lower than min layer edge?
There was a problem hiding this comment.
No, this variable is set to
((K - 1) >= MinLyrEdgeBot ? MeanPseudoThickEdge(IEdge, K - 1) : 0)
which is 0 for K = MinLyrEdgeBot. In the end an average of PseudoThickKm1 and PseudoThickK is used. This means that for K = MinLyrEdgeBot you get
PseudoThickAvg = 0.5 * (0 + MeanPseudoThickEdge(IEdge, MinLyrEdgeBot));
This is a kinda hacky treatment of the first half layer. I will add a comment to explain this.
|
@mwarusz in testing I'm getting NaN for EddyVelocity quickly. CoPilot added some guards on the ML average and zero initializing and it fixed the NaN issue. I can point you at what the changes made were if you'd like |
Please do. |
|
here it is - https://github.com/vanroekel/E3SM/tree/temp-submeso-branch There is some extra stuff that is adding more diagnostics as well. |
|
Hmm, Copilot added this check but afterwards it is also explicitly checking for NaNs and skipping calculations if it finds any. It would good to see if the first check is enough. |
3660296 to
a7bd36d
Compare
|
|
||
| // Compute mixed layer index and depth based on the density difference | ||
| // criterion | ||
| void computeDenMixLayerDepth(const Array2DReal &SpecVol); |
There was a problem hiding this comment.
@mwarusz, you were going to move this to an aux variable, correct?
There was a problem hiding this comment.
Yes. I haven't worked on this PR in a while. I was planning to get back to it after the split-explicit is merged, since the transport velocity refactoring that I did there affects this PR in non-trivial ways.
a7bd36d to
0bf23ad
Compare
|
To test this branch after rebasing on
Here are some plots comparing temperature, mixed layer depth, and eddy velocity kinetic energy at the final time. @vanroekel Do you think these look reasonable ? Omega
MPAS-O
|
| - `Tau`: MLI timescale parameter (s). | ||
| - `Ce`: nondimensional efficiency coefficient. | ||
| - `LfMin`: minimum frontal width limiter (m). | ||
| - `DsMax`: maximum grid-length limiter used in the closure (m). |
There was a problem hiding this comment.
somewhere, either here or in the SubmesoEddy class we should include reasonable ranges of these parameters.
|
|
||
| parallelForOuter( | ||
| LaunchConfig({Mesh->NEdgesAll}, | ||
| TeamScratch<Real>(3 * NVertLayers + 3 * NVertLayersP1)), |
There was a problem hiding this comment.
Could you explain the 3 * here? I don't understand where they come from
There was a problem hiding this comment.
This is the number of scratch arrays used in this loop. Three scratch arrays for layer data and three for interface data. I added a comment and changed the code slightly to hopefully make it more clear.
| const int K = MaxLyrEdgeTop + 1; | ||
| const int JCell0 = CellsOnEdge(IEdge, 0); | ||
| const int JCell1 = CellsOnEdge(IEdge, 1); | ||
| BVFSqEdge(K) = 0.5_Real * (BruntVaisalaFreqSq(JCell1, K) + |
There was a problem hiding this comment.
how is this part different than just above on L183?
There was a problem hiding this comment.
It's not. In addition to BVF, the loop above interpolates fields that are located at layer mid-points (SpecVol), so it only loops over layers. This code handles the last interface interpolation for BVF. I rephrased the comment above to make it more clear.
There was a problem hiding this comment.
I see. thanks. Maybe say 'ocean bottom' instead of 'bottom interface' in the comment or 'bottom of the column'? But also on second read I clearly see the 'MaxLayerEdge' definition in K that I missed earlier
Co-authored-by: Luke Van Roekel <lvanroekel@lanl.gov>
Co-authored-by: Luke Van Roekel <lvanroekel@lanl.gov>
4648e59 to
b97fb83
Compare
|
I've looked over the code and it looks good to me, but Copilot found the following two issues:
@mwarusz, if you don't think these are an issue, I am happy to approve. |
|
I pushed a fix for the first issue. I think the second one doesn't need a fix. I believe we never call the |
katsmith133
left a comment
There was a problem hiding this comment.
I pushed a fix for the first issue. I think the second one doesn't need a fix. I believe we never call the
setXfunctions with generic or overloaded functors, and I don't think there is any use case for this.
Great, thanks @mwarusz! I am happy to approve this now. Approving based on visual inspection and testing done by @mwarusz.
Testing on frontierTested the merge of #438 (
frontier, craygnu, mpichPolaris
|
|
While testing this branch with a new Polaris mixed-layer-eddies task, I found that In a nearly neutral mixed layer the density-threshold depth can jump by whole layers between the two. At 8 km on day 1, the output depth was the one the eddy velocity was computed with on only about a third of the edges in the frontal zone (about 80% by day 7). Within the scheme the two are consistent, so this only affects anyone diagnosing it from output. Writing the edge mixed-layer depth the scheme used, or a note in the user guide, would help. I only looked at the split-explicit stepper. Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly. |
Testing on pm-cpu and pm-gpuTested the merge of #438 (
pm-cpu, gnu, mpichPolaris
|
The new develop head brings in Omega's submesoscale eddy parameterization (E3SM-Project/Omega#438), which the mixed_layer_eddies tasks validate, along with frazil, penetrating shortwave, a KPP sea-ice buoyancy-flux fix and a division-by-zero guard. Steven Brus's testing of E3SM-Project#438 merged into this develop found omega_pr bit-for-bit with the previous submodule. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The new develop head brings in Omega's submesoscale eddy parameterization (E3SM-Project/Omega#438), which the mixed_layer_eddies tasks validate, along with frazil, penetrating shortwave, a KPP sea-ice buoyancy-flux fix and a division-by-zero guard. Steven Brus's testing of E3SM-Project#438 merged into this develop found omega_pr bit-for-bit with the previous submodule. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>






This PR adds the submesoscale eddy parametrization in the form presented in Fox-Kemper et al. 2011. The parametrization class provides three main methods that
respectively. A new auxiliary variable named
NormalTransportVelocityhas been added. This velocity variable is used to advect pseudo-thickness and tracers, and can optionally include the submesocale induced velocity.Checklist
Testingwith the following:have been run on and indicate that are all passing.
has passed, using the Polaris
e3sm_submodules/Omegabaseline-pfor both the baseline (Polarise3sm_submodules/Omega) and the PR build