Repository navigation
Add monthly averages - #553
Conversation
There was a problem hiding this comment.
Read the diff and built the branch merged with #481 on Chrysalis (intel, openmpi); a one-year QU240 run with MonthlyAverages on is queued behind today's maintenance and I will follow up with what it wrote. Two things need changing before the output can be used as monthly means: the fields are not marked as time means, so their stream gets no time_bnds, and the active-layer loops skip the bottom interface of fields on NVertLayersP1. Both are inline. Copying Units and StdName into the time mean fixes #545 for TimeMean; the spatial operators still drop them, so that issue stays open. I could not check restart behavior yet.
Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.
| auto OutputField = | ||
| Field::create(OutputNames[0], | ||
| "Time average of " + InputNames[0], // Description | ||
| "", // Units | ||
| "", // Standard name | ||
| -std::numeric_limits<Real>::max(), // Min valid value | ||
| std::numeric_limits<Real>::max(), // Max valid value | ||
| NDims, // Rank | ||
| DimNames // Dimension names | ||
| "Time average of " + InputDescr, // Description | ||
| InputUnits, // Units | ||
| InputStdName, // Standard name | ||
| -std::numeric_limits<Real>::max(), // Min valid value | ||
| std::numeric_limits<Real>::max(), // Max valid value | ||
| NDims, // Rank | ||
| DimNames // Dimension names | ||
| ); |
There was a problem hiding this comment.
The output field is created without cell_methods = "time: mean", so nothing marks it as a time mean and its stream gets no time_bnds (#549). #481 adds the stamp right here, OutputField->addMetadata("cell_methods", std::string("time: mean")), and has IOStream write time_bnds for any stream whose fields carry it. If #481 merges first, a rebase brings both; otherwise this PR needs them, since a monthly mean without its bounds is not usable as one.
There was a problem hiding this comment.
After the latest updates, there's a conflict in AnalysisOperatorTest.cpp. Let's merge #481 in first, and then I'll do a rebase to fix everything.
TestingI ran one simulated year at QU240 with The run completed and wrote the twelve Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly. |
The design expected the monthly means to arrive in a stream of their own rather than in History, where the mock-up had them, and E3SM-Project/Omega#553 gives them one: the MonthlyAverages analysis group. Its stream is found the way the global statistics' is. Omega names a monthly mean for the instant it was finalized, the start of the following month (E3SM-Project/Omega#554), so the file named 0001-02 holds January of year 1. expand_template takes a month offset and every time-mean stream with a month in its template is expanded with an offset of one; the year and month a SimFile carries are still the ones it covers. Each field of a time mean is named for the field and the period, Temperature_TimeMean1Month; time_mean_suffix says what to strip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@brian-oneill, I am testing this branch in combination with a new mixing diagnostic (DVD) written by Cage Mitchell. The variables he added are: and I added those to the Monthly Average. I get the error so it looks like underscores in the variable name cause this problem, since we are normally using CamelCase. Do you agree? You are using the underscore to parse the |
|
@mark-petersen Yes, the analysis parser uses underscores as a delimiter, breaking the chains into the names of either fields or operators. Since we are using PascalCase for variable names, I would suggest the new variables use names like |
|
Thanks! That was a simple change and it all works now. |
mark-petersen
left a comment
mark-petersen
left a comment
There was a problem hiding this comment.
I tested on perlmutter-cpu with gnu using the SOMA test case and this works great! I added numerous variables, ran for a year, and visualized the last month. Analysis was with Cage Mitchell. Thanks!
|
@brian-oneill, now that #481 is merged, this is ready for a rebase. Thanks for all this hard work!! |
182aad1 to
54456ec
Compare
|
I will test this soon and hopefully get it merged. I'm doing some Polaris updating first. |
Testing: chrysalis, oneapi-ifx, openmpi (Polaris
|
Testing: lint and docsOmega CI's
Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly. |
Testing: pm-gpu, gnugpu, mpichCTests: 50 of 50 passed. Tested the merge of #553 ( Polaris
|
Testing: pm-cpu, gnu, mpichCTests: 50 of 50 passed. Tested the merge of #553 ( Polaris
|
| InputField->getMetadata("Description", InputDescr); | ||
| InputField->getMetadata("Units", InputUnits); | ||
| InputField->getMetadata("StdName", InputStdName); | ||
| InputField->getMetadata("ValidMin", InputValidMin); |
There was a problem hiding this comment.
With KineticEnergyCell in MonthlyAverages: Fields, Omega aborts at init with std::bad_any_cast here. That field registers ValidMin as the int literal 0 (src/ocn/auxiliaryVars/KineticAuxVars.cpp:44), so reading it as double throws. Changing that literal to 0.0 looks like enough, but I haven't tested it. A clearer error here for a mismatched type might be worth adding too.
Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.
There was a problem hiding this comment.
@brian-oneill, I'm hoping you won't mind me pushing the fix if it works.
The int literal 0 was stored as an int, so TimeMeanOp's read of ValidMin as a Real threw std::bad_any_cast when KineticEnergyCell was averaged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Testing: frontier, craygnu-mphipcc, mpichCTests: 50 of 50 passed. Tested the merge of #553 ( Polaris
|
Testing: frontier, craygnu, mpichCTests: 50 of 50 passed. Tested the merge of #553 ( Polaris
|
Analysis TestingI ran the one-year QU240 It passed and wrote all 12 Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly. |
The design expected the monthly means to arrive in a stream of their own rather than in History, where the mock-up had them, and E3SM-Project/Omega#553 gives them one: the MonthlyAverages analysis group. Its stream is found the way the global statistics' is. Omega names a monthly mean for the instant it was finalized, the start of the following month (E3SM-Project/Omega#554), so the file named 0001-02 holds January of year 1. expand_template takes a month offset and every time-mean stream with a month in its template is expanded with an offset of one; the year and month a SimFile carries are still the ones it covers. Each field of a time mean is named for the field and the period, Temperature_TimeMean1Month; time_mean_suffix says what to strip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Testing: aurora, oneapi-ifx, mpichCTests: 50 of 50 passed. Tested the merge of #553 ( Polaris
|
xylar
left a comment
xylar
left a comment
There was a problem hiding this comment.
I've tested the heck out of this branch both in the Polaris analysis run using E3SM-Project/polaris#787 and with the omega_pr suite and CTests. Everything looks good to me.
Thanks so much @brian-oneill!
The design expected the monthly means to arrive in a stream of their own rather than in History, where the mock-up had them, and E3SM-Project/Omega#553 gives them one: the MonthlyAverages analysis group. Its stream is found the way the global statistics' is. Omega names a monthly mean for the instant it was finalized, the start of the following month (E3SM-Project/Omega#554), so the file named 0001-02 holds January of year 1. expand_template takes a month offset and every time-mean stream with a month in its template is expanded with an offset of one; the year and month a SimFile carries are still the ones it covers. Each field of a time mean is named for the field and the period, Temperature_TimeMean1Month; time_mean_suffix says what to strip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add MonthlyAverages to Omega's analysis framework that computes calendar-month time averages of requested fields and refactor the TimeMeanOp operator to use hierarchical parallelism bounded by the VertCoord active-layer arrays so only active vertical layers are averaged and inactive layers retain their fill values. Register the new MonthlyAverages analysis group, wire it into the analysis configuration and default config, and extend the operator unit test to verify active layers equal the analytic time mean. This adds a commonly requested diagnostic output and makes time averaging correct and efficient for column-structured mesh fields.
Summary
Add a
MonthlyAveragesanalysis group to Omega and refactorTimeMeanOpto average only active vertical layers.Changes
MonthlyAverages analysis group
MonthlyAveragesanalysis group (header, implementation, registration inGroups.handAnalysis.cpp).configs/Default.ymlso the group can be enabled and configured.doc/userGuide/Analysis.md.TimeMeanOp refactor
initialize()override that detects the horizontal index space (cells, edges, or vertices) from the input field's dimension names and stores the appropriateVertCoordMinLayer/MaxLayerarrays for rank-2/3 mesh fields.compute()as a three-phase state machine delegating toaccumInit,accumAdd, andfinalizehelpers.[MinLayer, MaxLayer]using hierarchicalparallelForOuter/parallelForInner, preservingFillValueRealin inactive layers; fall back to a flat parallel loop for scalars, rank-1, and non-mesh outputs.TimeMeancontinues to work for non-mesh outputs such as spatial-reduction scalars and lat/depth arrays.Checklist
Documentation:
Linting
Building
Testing
aurora, oneapi-ifx, mpich
chrysalis, oneapi-ifx, openmpi
frontier, craygnu, mpich
frontier, craygnu-mphipcc, mpich
pm-cpu, gnu, mpich
pm-gpu, gnugpu, mpich
Provide relevant details in a comment to the PR titled
Testingwith the following:have been run on and indicate that are all passing.
has passed, using the Polaris
e3sm_submodules/Omegabaseline-pfor both the baseline (Polarise3sm_submodules/Omega) and the PR buildPerformance related PRs: Please include a relevant PACE experiment link documenting performance before and after.
New tests:
Stealth Features