Repository navigation
Subgrid orography for OpenIFS, computed the way OpenIFS expects it - #63
Closed
JanStreffing wants to merge 7 commits into
Closed
JanStreffing wants to merge 7 commits into
JanStreffing wants to merge 7 commits into
Conversation
paleo_subgrid_oro.py cannot serve OpenIFS. It shells out to calnoro, an ECHAM tool implementing a different definition of the four gravity-wave-drag fields. Fed ECMWF's own TCO1279 orography, calnoro returns a global mean sdor of 43.3 against ECMWF's 60.4, with a point-by-point RMS difference of 74.8 against a field mean of 60.4. Moving its input from ECHAM5 T511 (26 km, 2007) to TCO1279 (9 km) closed about 9% of the gap, so the problem is the formulation and not the data. paleo_subgrid_oro.py is left alone for ECHAM. This computes the Lott and Miller gradient correlation tensor directly from a fine orography onto the reduced Gaussian grid, streaming the DEM in latitude bands. A global ETOPO1 pass takes 16 seconds. Not yet validated. Against climate.v020/95_4 over ice-free land the box-mean orography matches (corr 0.967), which confirms the grid mapping, and: sdor ratio 0.80 corr 0.848 slor ratio 1.69 corr 0.888 isor ratio 1.04 corr 0.381 The patterns are right and the calibration is not. Nothing here writes an ICMGG until the validation tool says it reproduces the ECMWF fields.
The slope came out 69% high against climate.v020/95_4. Two hypotheses, one dead and one right. Removing the resolved plane through each box before taking the gradient statistics, which is what Baines and Palmer do, changes slor by 0.8%. Inside a 100 km box the fine-scale gradients dominate the resolved slope entirely. Kept as an option, default on, but it is not the answer. The gradient length scale is. On ETOPO1 the slor ratio against ECMWF runs stencil 1 (1.9 km) ratio 1.693 corr 0.888 stencil 2 (3.7 km) ratio 1.272 corr 0.896 stencil 4 (7.4 km) ratio 0.880 corr 0.901 stencil 8 (15 km) ratio 0.589 corr 0.887 crossing one near 5 km while the correlation barely moves. ECMWF evaluate the slope over about 5 km rather than at the raw spacing of their orography database, which is what a filtering step in their preprocessing does. Stencil 3 gives ratio 1.009, corr 0.898, slope 0.961. State against climate.v020/95_4 over ice-free land: sdor ratio 0.795 corr 0.848 slor ratio 1.009 corr 0.898 isor ratio 0.976 corr 0.361 sdor is 20% low because ETOPO1 is 1.8 km and ECMWF build from about 1 km, so the 1-2 km variance is missing. isor matches in distribution (p05 0.52 against 0.46, p50 0.76 against 0.74, p95 0.93 against 0.92) but not cell by cell, and that does not improve at any stencil. Its correlation rises with roughness, 0.31 in flat terrain to 0.49 in rough, which is what an ill-conditioned eigenvalue ratio does when the tensor is near degenerate.
The IFS orographic drag documentation and Elvidge et al. (ECMWF Tech Memo 893) give the procedure the shipped fields come from, and it has two steps this was missing. The 30 arc-second source is averaged to 2'30", about 5 km, first, because scales below that belong to TOFD and the effective roughness scheme. That is the same 5 km the stencil sweep found empirically, arrived at independently. The statistics then come from a band-passed residual carrying only 5 km to grid length, not from the orography relative to its box mean. Leaving the near-grid-scale ramp in the gradient tensor barely moves the standard deviation or the slope, but it ruins the anisotropy, which is a ratio of the tensor's eigenvalues and so is destroyed by a term common to both. Both forms of the band-pass are implemented. Subtracting the grid-scale orography interpolated back onto the fine grid gives isor 0.45; smoothing the prefiltered source, which is what the IFS does, gives 0.51, because it has no box-edge structure of its own for the gradient to find. The zonal smoothing is applied in latitude bands with a per-band sigma, since a single sigma in grid points would smooth over far too short a distance near the poles, which is exactly where the ice sheets are. Against climate.v020/95_4 over ice-free land, was -> now: sdor 0.795/0.848 -> 0.770/0.847 slor 1.009/0.898 -> 0.915/0.907 isor 0.976/0.361 -> 0.935/0.521 isor is the whole point of the change and it is still the weak one. sdor stays about 20% low at every setting tried, which is not the missing sub-5 km variance, since ECMWF exclude those scales too.
The cutoff is the grid length, so it scales. Checked over a fourfold resolution range against climate.v020: held fixed at 150 km the slor ratio drifts 0.910 at TCO79 to 0.955 at TCO319; scaled as 1.5 times the equatorial spacing it stays inside 0.911 to 0.927. How the three fields travel, TCO79 to TCO319: sdor ratio 0.758-0.796 corr 0.833-0.861 transfers slor ratio 0.911-0.927 corr 0.901-0.910 transfers isor ratio 0.920-0.937 corr 0.516 -> 0.394 does not The anisotropy degrading with resolution is not understood. It is not sample starvation: at TCO319, taking the prefilter from 3' to 1' raises the fine points per box from 31 to 277 and the correlation goes 0.394, 0.410, 0.375. That same test shows the 5 km prefilter is a physical choice and not a free parameter. Refining it breaks the slope, 0.93 to 1.20 to 1.73, because the scales it removes belong to TOFD and the effective roughness scheme.
A regular lat/lon cell shrinks as cos(lat): the 3 arcmin prefilter that gives a 5.5 km cell at the equator gives 5.5 km by 1 km at 80S. The gradient tensor was then sampled five times more finely in longitude than in latitude, which biases the anisotropy toward zonal structure in a pattern organised radially about the pole. It was plainly visible as radial spokes in isor over Antarctica, which is exactly where an ice sheet coupling needs the field. Three changes. The zonal prefilter now uses a per-row width, 1 at the equator and 1800 at the polar row, so the cell is about 5 km in both directions everywhere. The zonal gradient difference spans the same physical distance as the meridional one rather than the same number of cells. And box means are area weighted by cos(lat), so a box mean is an area mean and not a point mean. Over ice-free land, before -> after: sdor 0.770/0.847 -> 0.767/0.847 slor 0.915/0.907 -> 0.844/0.913 isor 0.935/0.521 -> 0.967/0.576 Over Antarctica with PISM's surface, where the bug lived: sdor 0.75/0.743 -> 0.75/0.744 slor 1.65/0.809 -> 0.78/0.850 isor 0.58/0.148 -> 1.07/0.255 The slope was 65% too steep there and is now 22% low, and the anisotropy amplitude was 42% low and is now 7% high. isor's correlation over Antarctica stays below ECMWF's own 0.459 lag-1 coherence there, but that is this test rather than the design: it replaces the ice sheet with PISM's 8 km surface, which is coarser than the 5 km the statistics want, so the fine texture ECMWF see is missing. Production adds the ice thickness change to the DEM instead and keeps that texture.
All 28782 sea cells of climate.v020/95_4 are exactly zero in all four fields. This computed them everywhere, and the scattered dots that left in the ocean were plainly visible in the anisotropy on a south polar map. They are not noise, they are cells holding a scrap of coast or an island. isor is a ratio of the gradient tensor's eigenvalues, so it is scale free: a cell with one metre of structure carries the same 0 to 1 range as the Himalaya. On ETOPO1 that is 4446 sea cells with isor above 0.3, of which 2148 have sdor below one metre and whose median land fraction is exactly zero. sdor and slor are contaminated in the same cells, 3295 and 21506 of them, but at metres and 1e-7 respectively, so they disappear into the dark end of a colour scale and only the normalised field shows the problem. Land statistics are unchanged, since the mask only touches sea.
The two ways to produce these fields differ by algorithm, not by whether the run is paleo: paleo_subgrid_oro.py calnoro, for ECHAM subgrid_oro_lott_miller.py Lott and Miller, for OpenIFS Both are subgrid orography, so the old name said nothing about which one to reach for.
Owner
Author
|
Superseded by #64, which is the same work on a clean branch. One commit here staged unrelated untracked files that were sitting in the working tree: Rather than clean it up in place and leave the blobs in history, the branch has been deleted so the objects are unreferenced. #64 carries the same six reviewed commits plus a rename, and touches exactly two files. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. Adds
ocp_tool/subgrid_orography.pyandtools/validate_subgrid_orography.py. Nothing is wired into a pipeline yet and nothing writes an ICMGG.Why not
paleo_subgrid_oro.pyThat path shells out to
calnoro, an ECHAM tool implementing a different definition of the four gravity-wave-drag fields. Fed ECMWF's own TCO1279 orography it returns a global meansdorof 43.3 against ECMWF's 60.4, with a point-by-point RMS difference of 74.8 against a field mean of 60.4. Moving its input from ECHAM5 T511 (26 km, 2007) to TCO1279 (9 km) closed about 9% of the gap, so the problem is the formulation, not the data.paleo_subgrid_oro.pyis untouched and stays as it is for ECHAM.What this does
Lott and Miller (1997) statistics computed directly onto the reduced Gaussian grid, following the procedure in the IFS orographic drag documentation and Elvidge et al. (ECMWF Tech Memo 893):
A global ETOPO1 pass takes about 15 seconds.
Validation
tools/validate_subgrid_orography.pycompares against what ECMWF ship beside the model,climate.v020/<res>/{stdgwd,isogwd,slogwd}. Against95_4over ice-free land, with ETOPO1:The last column matters. ECMWF's
isorbarely agrees with itself one grid box away, because anisotropy is a ratio of the tensor's eigenvalues and is genuinely noisy. All three fields here track ECMWF more closely than ECMWF tracks its own neighbour, so 0.576 is at the field's noise floor rather than short of a target.Box-mean orography against
95_4/orogis the control, correlation 0.967, which is what says the grid mapping is right.Things worth a reviewer's attention
slorratio drifts 0.910 at TCO79 to 0.955 at TCO319; scaled as 1.5 times the equatorial spacing it stays inside 0.911 to 0.927.sdorunder a metre.Open
sdorsits about 20% low at every resolution and setting tried. Stable enough that a single factor would be defensible, but not understood.isordegrades with target resolution, 0.52 at TCO79 and TCO95 down to 0.39 at TCO319, and it is not sample starvation: nine times the fine points per box moves it 0.394, 0.410, 0.375.