Skip to content

Enforce velocity field masking for NormalTransportVelocity - #638

Open
mwarusz wants to merge 1 commit into
E3SM-Project:developfrom
mwarusz:omega/transport-vel-masking
Open

Enforce velocity field masking for NormalTransportVelocity#638
mwarusz wants to merge 1 commit into
E3SM-Project:developfrom
mwarusz:omega/transport-vel-masking

Conversation

@mwarusz

@mwarusz mwarusz commented Oct 26, 2026 •
edited by andrewdnolan
Loading

Copy link
Copy Markdown
Member

We enforce the following 3-zone masking for NormalVelocity

  • fill value outside of [MinLayerEdgeTop, MaxLayerEdgeBot]
  • 0 outside of [MinLayerEdgeBot, MaxLayerEdgeTop] and inside [MinLayerEdgeTop, MaxLayerEdgeBot]
  • prognostic values inside [MinLayerEdgeBot, MaxLayerEdgeTop]

The same should be true for NormalTransportVelocity, but I did not compute it in a way that preserves this masking. This PR fixes it.

Checklist

  • Linting

  • Building

    • CMake build does not produce any new warnings from changes in this PR

  • Testing

    aurora, intel, mpich

    • CTests Pass
    • Polaris omega_pr Pass

    aurora, intelgpu, mpich

    • CTests Pass
    • Polaris omega_pr Pass

    chrysalis, intel, openmpi

    • CTests Pass
    • Polaris omega_pr Pass
    • CIME test cases:
      • ERS_Ld5.T62_oQU240.GOMEGA-IAF
      • SMS_Ld3.ne4pg2_oQU240.WCYCL1850NS-OMEGA

    frontier, craygnu, mpich

    • CTests Pass
    • Polaris omega_pr Pass

    frontier, craygnu-mphipcc, mpich

    • CTests Pass
    • Polaris omega_pr Pass
    • CIME test cases:
      • ERS_Ld5.T62_oQU240.GOMEGA-IAF
      • ERS_Ld3.ne4pg2_oQU240.WCYCL1850NS-OMEGA

    pm-cpu, gnu, mpich

    • CTests Pass
    • Polaris omega_pr Pass

    pm-gpu, gnugpu, mpich

    • CTests Pass
    • Polaris omega_pr Pass

  • Provide relevant details in a comment to the PR titled Testing with the following:

    • Which machines CTest unit tests
      have been run on and indicate that are all passing.
    • The Polaris omega_pr test suite
      has passed, using the Polaris e3sm_submodules/Omega baseline
    • Document machine(s), compiler(s), and the build path(s) used for -p for both the baseline (Polaris e3sm_submodules/Omega) and the PR build
    • Indicate "All tests passed" or document failing tests
    • Document testing used to verify the changes including any tests that are added/modified/impacted.

Fixes #588
Closes #628

@mwarusz mwarusz mentioned this pull request Oct 26, 2026
Closed
17 of 18 tasks
@mwarusz

mwarusz commented Oct 26, 2026

Copy link
Copy Markdown
Member Author

Testing on pm-cpu

Polaris omega_pr suite

  • Baseline workdir: /pscratch/sd/m/mwarusz/omega-pr-testing/transport-vel/../math/baseline-gnu
  • Baseline build: /pscratch/sd/m/mwarusz/omega-pr-testing/math/baseline-gnu/ocean_build
  • PR build: /pscratch/sd/m/mwarusz/omega-pr-testing/transport-vel/build-gnu
  • PR workdir: /pscratch/sd/m/mwarusz/omega-pr-testing/transport-vel/pr-gnu
  • Baseline Omega: cdb935bb5a8d995b0878616e001a2701a2627519 (eamxx-v0.1.0-5440-gcdb935bb5a)
  • PR Omega: 878c46d266278b42e9a4bc820d38afbc089718a3 (eamxx-v0.1.0-5526-g878c46d266-dirty) (at setup, not build)
  • Machine: pm-cpu
  • Compiler: gnu
  • Build type: Release
  • Log: /pscratch/sd/m/mwarusz/omega-pr-testing/transport-vel/pr-gnu/polaris_omega_pr.o60293667
  • Result:
    • Failures (1 of 32):
      • ocean/spherical/realistic_global/QU.240km/restart
    • Diffs (7 of 32):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.318e-10  l2=1.112e-11  linf=1.315e-12
      - normalVelocity: l1=3.045e-12  l2=3.099e-13  linf=6.261e-14
      - salinity:       l1=3.411e-13  l2=5.684e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - normalVelocity: l1=1.783e-08  l2=4.951e-11  linf=1.170e-12
      - temperature:    l1=1.401e-10  l2=1.282e-12  linf=3.055e-13
      - salinity:       l1=1.261e-10  l2=1.051e-12  linf=4.974e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=1.776e-13  l2=3.855e-14  linf=1.599e-14
      - normalVelocity: l1=1.388e-17  l2=1.388e-17  linf=1.388e-17
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Baseline Omega:

cdb935bb5a Merge pull request #600 from xylar/omega/fix-pm-mpi-flags
b90c3dae8d Merge pull request #601 from andrewdnolan/omega/cxx20-for-coupled
9641c6cf9e Merge pull request #546 from xylar/omega/fix-cf-units
55c3de8c15 Merge pull request #566: e3sm/master into omega/develop
9aba4a8032 Merge pull request #597 from xylar/omega/fix-real-literal-nvcc

PR Omega:

878c46d266 Enforce velocity field masking for NormalTransportVelocity
044014eb4d Work
135755f734 Merge pull request #438 from mwarusz/omega/submesoscale-eddies
a341e65245 Merge pull request #562 from andrewdnolan/omega/guard-division-by-zero
46aa54cd68 Merge pull request #603 from vanroekel/vanroekel/omega/kpp-bug-fix

Notes

The diffs are identical to the diffs on pm-cpu in #628 (comment)

@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

Thanks @mwarusz, I'll take a look but I agree this seems like a better fix than #628.

@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

I'm going to test things out on Chrysalis to make sure this fixes #588 for me. Assuming it does, I'll run the testing on all supported machines.

@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

Verification on Chrysalis

This fixes #588. On Chrysalis with Intel, a one-year QU240 run and the 30 one-day overflow runs used for #628 (SE-RK2 and RK4) are bit-for-bit identical to #628, since the horizontal FCT provisional thickness was the only code reading the transport velocity on inactive edge layers.

Bottom layer of cell 2982 in one-year QU240 runs with develop and this PR

Daily global minimum temperature in one-year QU240 runs with develop and this PR

HorzTracerFluxLimiterMonotonicityCheckEnable finds no out-of-bounds values in 40 days of QU240 or 9 hours of overflow, where develop has 520,389 and 23,040 in the first two steps.

Runs: /lcrc/group/e3sm/ac.xylar/polaris_1.2/chrysalis/test_20261026/omega638-transport-vel

Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.

@xylar xylar mentioned this pull request Oct 26, 2026
Draft
18 tasks done
@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

Testing on frontier

Tested the merge of #638 (84d073c073) into develop (695f548841) as 5f70437af9, against Polaris' Omega submodule (23b13afac5).

Row CTests omega_pr New build warnings
frontier, craygnu, mpich 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
frontier, craygnu-mphipcc, mpich 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
frontier, craygnu, mpich

Polaris omega_pr suite

  • Baseline workdir: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/baselines/frontier_craygnu_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/baselines/frontier_craygnu_polaris-c3fc893_omega-23b13af/build
  • PR build: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu/build
  • PR workdir: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: frontier
  • Partition: batch
  • Compiler: craygnu
  • Build type: Release
  • Log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu/omega_pr/polaris_omega_pr.o5766410
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.370e-09  l2=4.857e-11  linf=4.036e-12
      - normalVelocity: l1=7.783e-10  l2=6.628e-12  linf=2.816e-13
      - salinity:       l1=4.029e-12  l2=1.791e-13  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.317e-10  l2=1.112e-11  linf=1.315e-12
      - salinity:       l1=1.421e-14  l2=1.421e-14  linf=1.421e-14
      - normalVelocity: l1=1.011e-15  l2=2.031e-17  linf=1.870e-18
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.370e-09  l2=4.857e-11  linf=4.036e-12
      - normalVelocity: l1=7.783e-10  l2=6.628e-12  linf=2.816e-13
      - salinity:       l1=4.029e-12  l2=1.791e-13  linf=1.421e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - normalVelocity: l1=6.598e-05  l2=3.312e-05  linf=2.845e-05
      - temperature:    l1=1.343e-11  l2=3.034e-13  linf=1.066e-13
      - salinity:       l1=1.208e-11  l2=3.008e-13  linf=1.421e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=1.918e-13  l2=3.517e-14  linf=1.243e-14
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: frontier
  • Compiler: craygnu
  • Build type: Release
  • Build: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu/build/ctests.log

Build warnings

No new warnings (3 in the PR build, 3 in the baseline build).

  • Baseline build log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/baselines/frontier_craygnu_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu/build/build_omega.log
frontier, craygnu-mphipcc, mpich

Polaris omega_pr suite

  • Baseline workdir: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/baselines/frontier_craygnu-mphipcc_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/baselines/frontier_craygnu-mphipcc_polaris-c3fc893_omega-23b13af/build
  • PR build: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu-mphipcc/build
  • PR workdir: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu-mphipcc/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: frontier
  • Partition: batch
  • Compiler: craygnu-mphipcc
  • Build type: Release
  • Log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu-mphipcc/omega_pr/polaris_omega_pr.o5766473
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.365e-09  l2=4.856e-11  linf=4.041e-12
      - normalVelocity: l1=7.471e-10  l2=5.463e-12  linf=2.634e-13
      - salinity:       l1=1.002e-11  l2=2.797e-13  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.318e-10  l2=1.112e-11  linf=1.316e-12
      - normalVelocity: l1=5.214e-13  l2=9.153e-14  linf=3.103e-14
      - salinity:       l1=1.421e-14  l2=1.421e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.365e-09  l2=4.856e-11  linf=4.041e-12
      - normalVelocity: l1=7.471e-10  l2=5.463e-12  linf=2.634e-13
      - salinity:       l1=1.002e-11  l2=2.797e-13  linf=1.421e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - temperature:    l1=1.688e-13  l2=3.530e-14  linf=1.599e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=2.043e-13  l2=4.016e-14  linf=1.599e-14
      - normalVelocity: l1=1.665e-16  l2=1.241e-16  linf=1.110e-16
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: frontier
  • Compiler: craygnu-mphipcc
  • Build type: Release
  • Build: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu-mphipcc/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu-mphipcc/build/ctests.log

Build warnings

No new warnings (6 in the PR build, 6 in the baseline build).

  • Baseline build log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/baselines/frontier_craygnu-mphipcc_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /lustre/orion/cli115/scratch/xylar/polaris_1.1/frontier/omega_pr_test/pr638-84d073c/frontier_craygnu-mphipcc/build/build_omega.log

Notes

Both rows differ from the baseline in the same 7 tasks. These come from this PR: its AuxiliaryState.cpp change is the only Omega source change in the test merge. #622, also merged since the Polaris pin, changes only config_machines.xml and docs, and both builds link the same METIS. They are the 7 tasks mwarusz's pm-cpu Testing comment above lists, and the merry_go_round and seamount norms match it to every printed digit. That fits this PR changing the transport velocity the horizontal FCT reads on inactive edge layers.

On craygnu, overflow/.../smoke_test_horiz_adv_order_3 has a larger normalVelocity difference (linf 2.8e-5) than on craygnu-mphipcc or pm-cpu (about 1e-12). I did not look into it further. Testing #628 on Frontier found that this nonlinear FCT test can amplify round-off on craygnu.

Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.

@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

Testing on chrysalis

Tested the merge of #638 (84d073c073) into develop (695f548841) as 5f70437af9, against Polaris' Omega submodule (23b13afac5).

Row CTests omega_pr New build warnings
chrysalis, intel, openmpi 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
chrysalis, intel, openmpi

Polaris omega_pr suite

  • Baseline workdir: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/baselines/chrysalis_intel_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/baselines/chrysalis_intel_polaris-c3fc893_omega-23b13af/build
  • PR build: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/pr638-84d073c/chrysalis_intel/build
  • PR workdir: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/pr638-84d073c/chrysalis_intel/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: chrysalis
  • Partition: debug
  • Compiler: intel
  • Build type: Release
  • Log: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/pr638-84d073c/chrysalis_intel/omega_pr/polaris_omega_pr.o1315991
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.318e-10  l2=1.112e-11  linf=1.315e-12
      - normalVelocity: l1=3.045e-12  l2=3.099e-13  linf=6.261e-14
      - salinity:       l1=3.411e-13  l2=5.684e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - normalVelocity: l1=1.783e-08  l2=4.951e-11  linf=1.170e-12
      - temperature:    l1=1.401e-10  l2=1.282e-12  linf=3.055e-13
      - salinity:       l1=1.261e-10  l2=1.051e-12  linf=4.974e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=1.776e-13  l2=3.855e-14  linf=1.599e-14
      - normalVelocity: l1=1.388e-17  l2=1.388e-17  linf=1.388e-17
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: chrysalis
  • Compiler: intel
  • Build type: Release
  • Build: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/pr638-84d073c/chrysalis_intel/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/pr638-84d073c/chrysalis_intel/build/ctests.log

Build warnings

No new warnings (37 in the PR build, 37 in the baseline build).

  • Baseline build log: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/baselines/chrysalis_intel_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /lcrc/group/e3sm/ac.xylar/polaris_1.1/chrysalis/omega_pr_test/pr638-84d073c/chrysalis_intel/build/build_omega.log

Notes

The 7 baseline differences match the Chrysalis results for #628 (#628 (comment)) task for task and in every norm. That run tested against develop at the Polaris pin. The only merge into develop since then, #622, changes just cime_config/machines/config_machines.xml and a docs page.

Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.

@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

Testing on pm-cpu and pm-gpu

Tested the merge of #638 (84d073c073) into develop (695f548841) as 5f70437af9, against Polaris' Omega submodule (23b13afac5).

Row CTests omega_pr New build warnings
pm-cpu, gnu, mpich 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
pm-gpu, gnugpu, mpich 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
pm-cpu, gnu, mpich

Polaris omega_pr suite

  • Baseline workdir: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/baselines/pm-cpu_gnu_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/baselines/pm-cpu_gnu_polaris-c3fc893_omega-23b13af/build
  • PR build: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-cpu_gnu/build
  • PR workdir: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-cpu_gnu/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: pm-cpu
  • Compiler: gnu
  • Build type: Release
  • Log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-cpu_gnu/omega_pr/polaris_omega_pr.o60310498
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.318e-10  l2=1.112e-11  linf=1.315e-12
      - normalVelocity: l1=3.045e-12  l2=3.099e-13  linf=6.261e-14
      - salinity:       l1=3.411e-13  l2=5.684e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - normalVelocity: l1=1.783e-08  l2=4.951e-11  linf=1.170e-12
      - temperature:    l1=1.401e-10  l2=1.282e-12  linf=3.055e-13
      - salinity:       l1=1.261e-10  l2=1.051e-12  linf=4.974e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=1.776e-13  l2=3.855e-14  linf=1.599e-14
      - normalVelocity: l1=1.388e-17  l2=1.388e-17  linf=1.388e-17
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: pm-cpu
  • Compiler: gnu
  • Build type: Release
  • Build: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-cpu_gnu/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-cpu_gnu/build/ctests.log

Build warnings

No new warnings (4 in the PR build, 4 in the baseline build).

  • Baseline build log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/baselines/pm-cpu_gnu_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-cpu_gnu/build/build_omega.log
pm-gpu, gnugpu, mpich

Polaris omega_pr suite

  • Baseline workdir: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/baselines/pm-gpu_gnugpu_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/baselines/pm-gpu_gnugpu_polaris-c3fc893_omega-23b13af/build
  • PR build: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-gpu_gnugpu/build
  • PR workdir: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-gpu_gnugpu/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: pm-gpu
  • Compiler: gnugpu
  • Build type: Release
  • Log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-gpu_gnugpu/omega_pr/polaris_omega_pr.o60310514
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.370e-09  l2=4.855e-11  linf=4.034e-12
      - normalVelocity: l1=9.325e-10  l2=6.948e-12  linf=4.131e-13
      - salinity:       l1=1.065e-11  l2=2.882e-13  linf=2.132e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.317e-10  l2=1.112e-11  linf=1.315e-12
      - normalVelocity: l1=9.819e-13  l2=1.556e-13  linf=6.206e-14
      - salinity:       l1=4.263e-14  l2=2.010e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.370e-09  l2=4.855e-11  linf=4.034e-12
      - normalVelocity: l1=9.325e-10  l2=6.948e-12  linf=4.131e-13
      - salinity:       l1=1.065e-11  l2=2.882e-13  linf=2.132e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - normalVelocity: l1=1.979e-04  l2=5.737e-05  linf=2.845e-05
      - temperature:    l1=1.199e-10  l2=9.344e-13  linf=1.172e-13
      - salinity:       l1=1.327e-10  l2=1.062e-12  linf=2.842e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=1.332e-13  l2=2.593e-14  linf=8.882e-15
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: pm-gpu
  • Compiler: gnugpu
  • Build type: Release
  • Build: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-gpu_gnugpu/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-gpu_gnugpu/build/ctests.log

Build warnings

No new warnings (852 in the PR build, 852 in the baseline build).

  • Baseline build log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/baselines/pm-gpu_gnugpu_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /pscratch/sd/x/xylar/polaris_1.1/omega_pr_test/pr638-84d073c/pm-gpu_gnugpu/build/build_omega.log

Notes

On both rows, the 7 baseline differences match the results for #628 (#628 (comment)) task for task and in every norm. That run tested against develop at the Polaris pin. The only merge into develop since then, #622, changes just cime_config/machines/config_machines.xml and a docs page.

Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.

@xylar

xylar commented Oct 26, 2026

Copy link
Copy Markdown

Testing on aurora

Tested the merge of #638 (84d073c073) into develop (695f548841) as 5f70437af9, against Polaris' Omega submodule (23b13afac5).

Row CTests omega_pr New build warnings
aurora, intel, mpich 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
aurora, intelgpu, mpich 61 of 61 passed 26 of 33 passed, 7 with baseline differences 0
aurora, intel, mpich

Polaris omega_pr suite

  • Baseline workdir: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/baselines/aurora_intel_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/baselines/aurora_intel_polaris-c3fc893_omega-23b13af/build
  • PR build: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intel/build
  • PR workdir: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intel/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: aurora
  • Compiler: intel
  • Build type: Release
  • Log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intel/omega_pr/polaris_omega_pr.o8966920
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.318e-10  l2=1.112e-11  linf=1.315e-12
      - normalVelocity: l1=3.045e-12  l2=3.099e-13  linf=6.261e-14
      - salinity:       l1=3.411e-13  l2=5.684e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.367e-09  l2=4.855e-11  linf=4.036e-12
      - normalVelocity: l1=6.686e-10  l2=6.201e-12  linf=3.463e-13
      - salinity:       l1=6.693e-12  l2=2.445e-13  linf=1.421e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - normalVelocity: l1=1.783e-08  l2=4.951e-11  linf=1.170e-12
      - temperature:    l1=1.401e-10  l2=1.282e-12  linf=3.055e-13
      - salinity:       l1=1.261e-10  l2=1.051e-12  linf=4.974e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=1.776e-13  l2=3.855e-14  linf=1.599e-14
      - normalVelocity: l1=1.388e-17  l2=1.388e-17  linf=1.388e-17
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: aurora
  • Compiler: intel
  • Build type: Release
  • Build: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intel/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intel/build/ctests.log

Build warnings

No new warnings (14 in the PR build, 14 in the baseline build).

  • Baseline build log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/baselines/aurora_intel_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intel/build/build_omega.log
aurora, intelgpu, mpich

Polaris omega_pr suite

  • Baseline workdir: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/baselines/aurora_intelgpu_polaris-c3fc893_omega-23b13af/omega_pr
  • Baseline build: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/baselines/aurora_intelgpu_polaris-c3fc893_omega-23b13af/build
  • PR build: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intelgpu/build
  • PR workdir: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intelgpu/omega_pr
  • Polaris: c3fc893b7ab4363dfaee1398c741eae6c0b8abce (1.1.0-342-gc3fc893b7-dirty)
  • Baseline Omega: 23b13afac5f4fbf9ba18d6cec416d347e4a615f7 (Omega-v0.1.0-alpha.1-6283-g23b13afac5)
  • PR Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Machine: aurora
  • Compiler: intelgpu
  • Build type: Release
  • Log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intelgpu/omega_pr/polaris_omega_pr.o8966925
  • Result:
    • Diffs (7 of 33):
    - ocean/planar/baroclinic_channel/split_explicit/10km/decomp
      - temperature:    l1=1.365e-09  l2=4.856e-11  linf=4.038e-12
      - normalVelocity: l1=7.578e-10  l2=5.495e-12  linf=2.627e-13
      - salinity:       l1=9.784e-12  l2=2.760e-13  linf=2.132e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/restart
      - temperature:    l1=2.317e-10  l2=1.112e-11  linf=1.315e-12
      - normalVelocity: l1=5.181e-13  l2=9.152e-14  linf=3.103e-14
      - salinity:       l1=3.553e-14  l2=2.132e-14  linf=1.421e-14
    - ocean/planar/baroclinic_channel/split_explicit/10km/threads
      - temperature:    l1=1.365e-09  l2=4.856e-11  linf=4.038e-12
      - normalVelocity: l1=7.578e-10  l2=5.495e-12  linf=2.627e-13
      - salinity:       l1=9.784e-12  l2=2.760e-13  linf=2.132e-14
    - ocean/planar/merry_go_round/default
      - tracer1:        l1=4.235e+00  l2=1.500e-01  linf=1.475e-02
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_3
      - temperature:    l1=1.545e-13  l2=3.061e-14  linf=1.243e-14
    - ocean/planar/overflow/linear/zstar/smoke_test_horiz_adv_order_4
      - temperature:    l1=2.007e-13  l2=3.485e-14  linf=1.066e-14
    - ocean/planar/seamount/linear/zstar/short
      - temperature:    l1=7.333e-03  l2=1.236e-03  linf=4.265e-04
      - normalVelocity: l1=1.180e-03  l2=5.410e-05  linf=8.686e-06
      - layerThickness: l1=2.205e-04  l2=2.050e-05  linf=5.688e-06
Recent commits

Polaris:

c3fc893b7 Merge pull request #730 from xylar/add-omega-analysis-climatology-maps
2f9888f2a Merge pull request #901 from xylar/update-e3sm-project-submodule
fc3cf13cf Merge pull request #899 from xylar/restore-aurora-oneapi-2026
680e3a2c7 Merge pull request #893 from xylar/add-pr-test-parallel
56a6d0eb9 Merge pull request #891 from xylar/fix-dependency-pickle-growth

Baseline Omega:

23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp
ede8e22014 Merge pull request #625 from grnydawn/ykim/omega/standalone-cpu-arch-flags
eeef0a29dc Merge pull request #611 from brian-oneill/omega/analysis-restarts-and-samples

PR Omega:

5f70437af9 Test merge of E3SM-Project/Omega#638 (84d073c073) into develop (695f548841)
695f548841 Merge pull request #622 from sbrus89/omega/metis-root-update
23b13afac5 Merge pull request #506 from hyungyukang/omega/compute-mesh-scaling
330b297447 Merge pull request #606 from andrewdnolan/omega/optional-cout-redirect
f13879a93a Merge pull request #612 from alicebarthel/omega/add-salinity-clamp

CTest unit tests:

  • Machine: aurora
  • Compiler: intelgpu
  • Build type: Release
  • Build: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intelgpu/build
  • Omega: 5f70437af9eb464139cdeca27cca5f0846b7704c (Omega-v0.1.0-alpha.1-6289-g5f70437af9)
  • Result: All tests passed
  • Log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intelgpu/build/ctests.log

Build warnings

No new warnings (226 in the PR build, 263 in the baseline build).

  • Baseline build log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/baselines/aurora_intelgpu_polaris-c3fc893_omega-23b13af/build/build_omega.log
  • PR build log: /lus/flare/projects/E3SM_Dec/xylar/polaris_1.1/aurora/omega_pr_test/pr638-84d073c/aurora_intelgpu/build/build_omega.log

Notes

On both rows, the 7 baseline differences match the results for #628 (#628 (comment)) task for task and in every norm. That run tested against develop at the Polaris pin. The only merge into develop since then, #622, changes just cime_config/machines/config_machines.xml and a docs page.

Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.

xylar
xylar approved these changes Oct 26, 2026
View reviewed changes

@xylar xylar left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mwarusz, this looks great! Approved based on an inspection of the code changes and my testing.

@xylar xylar added bug Something isn't working Omega non-BFB PR makes roundoff changes to answers. labels Oct 26, 2026
@sbrus89 sbrus89 added this to the 1st E3SM Merge milestone Oct 26, 2026
@andrewdnolan

andrewdnolan commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

Testing on chrysalis with intel compiler

Test were run with a local merge of this branch into develop at f368d66830.

  • ERS_Ld5.T62_oQU240.GOMEGA-IAF.chrysalis_intel.20261026_145521: PASS
  • SMS_Ld3.ne4pg2_oQU240.WCYCL1850NS-OMEGA.chrysalis_intel.20261026_145522: PASS

@andrewdnolan

andrewdnolan commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

Testing on frontier with craygnu-mphipcc compiler

Test were run with a local merge of this branch into develop at f368d66830.

  • ERS_Ld5.T62_oQU240.GOMEGA-IAF.frontier_craygnu-mphipcc.20261026_161019: PASS
  • ERS_Ld3.ne4pg2_oQU240.WCYCL1850NS-OMEGA.frontier_craygnu-mphipcc.20261026_161050: PASS

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xylar xylar xylar approved these changes
@andrewdnolan andrewdnolan andrewdnolan approved these changes

Assignees

No one assigned

Labels

bug Something isn't working non-BFB PR makes roundoff changes to answers. Omega

Projects

None yet

Milestone

1st E3SM Merge

Development

Successfully merging this pull request may close these issues.

Bottom-cell temperatures fall far below the initial minimum in realistic global runs

4 participants

@mwarusz @xylar @andrewdnolan @sbrus89