Convert checkpoint files from legacy domains to layered CF

[Ed Hone (EdHone)]

PR Summary

Sci/Tech Reviewer: iboutle
Code Reviewer:

The checkpointing I/O system used by all the LFRic apps uses 'legacy' checkpointing domains which are extremely non-performance at scale. The aim is convert to using the same UGRID format for output as the diagnostic files, which would involve changing the way that fields on the W2 function space are set up for I/O in XIOS. However, progress has been difficult for implementing this goal, as the UGRID implementation inside XIOS has some requirements and bugs which we cannot circumvent at the required timescales.
To get a performant checkpointing system working as soon as possible, this PR switches checkpoint files to use a layered CF I/O implementation, which will circumvent the bugs in UGRID whilst maintaining the improved performance gains. This will not be the long-term solution, as the output files will lack some required metadata but in the mean time this will allow us to start testing at scale with checkpoint-restart.

Note: The new checkpointing format still has issues for biperiodic meshes, such as those used by the SCM lfric_atm configurations. Workarounds are provided in the code to use the old-style checkpointing for these runs. The bug that necessitates this does not seem to be present in XIOS 3

REVIEWER GUIDE - please run the lfric_atm_extra group to update the KGOs

• linked Enable the use of layered CF output for checkpoint-restart lfric_core#468

• is blocked-by Coupled checkpoint restart #774

Code Quality Checklist

• I have performed a self-review of my own code
• My code follows the project's style guidelines
• Comments have been included that aid understanding and enhance the readability of the code
• My changes generate no new warnings
• All automated checks in the CI pipeline have completed successfully

Testing

• I have tested this change locally, using the LFRic Apps rose-stem suite
• If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
• I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
• Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_apps - lfric_apps-layered-cf-cp/run21

Suite Information

Item	Value
Suite Name	lfric_apps-layered-cf-cp/run21
Suite User	edward.hone
Workflow Start	2026-09-30T12:03:49
Groups Run	developer

Dependency	Reference	Main Like
casim	MetOffice/casim@2026.07.1	True
jules	MetOffice/jules@c22aeea	True
lfric_apps	EdHone/lfric_apps@layered-cf-cp	False
lfric_core	EdHone/lfric_core@ugrid-ckp-flag	True
moci	MetOffice/moci@2026.07.1	True
SimSys_Scripts	MetOffice/SimSys_Scripts@77a5166	True
socrates	MetOffice/socrates@2026.07.1	True
socrates-spectral	MetOffice/socrates-spectral@2026.07.1	True
ukca	MetOffice/ukca@c372b36	True

Task Information

❌ failed tasks - 4

Task	State
check_jedi_lfric_tests_forecast_nwp_gal9_da-C12_azspice_gnu_fast-debug-64bit	failed
check_jedi_lfric_tests_forecast_nwp_gal9_da-C12_azspice_gnu_full-debug-64bit	failed
check_jedi_lfric_tests_forecast_nwp_gal9_da-C12_ex1a_cce_fast-debug-64bit	failed
kgo_groups_checker	failed

✅ succeeded tasks - 1211

⌛ waiting tasks - 2

Task	State
housekeep_azspice	waiting
housekeep_ex1a	waiting

Security Considerations

• I have reviewed my changes for potential security issues
• Sensitive data is properly handled (if applicable)
• Authentication and authorisation are properly implemented (if applicable)

Performance Impact

• Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

• Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

• Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

• If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the HPC Optimisation Team

Sci/Tech Review

• I understand this area of code and the changes being added
• The proposed changes correspond to the pull request description
• Documentation is sufficient (do documentation papers need updating)
• Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

• All dependencies have been resolved
• Related Issues have been properly linked and addressed
• CLA compliance has been confirmed
• Code quality standards have been met
• Tests are adequate and have passed
• Documentation is complete and accurate
• Security considerations have been addressed
• Performance impact is acceptable

[iboutle]

Again, just adding a note that this is blocked by #774

[DanStoneMO]

Ran JEDI with this and it works fine. No linked PR will be needed for it

[iboutle]

Just one question from me

[iboutle]

Why have some (but not all) grids included here had a _cf version added? None of the _cf versions are directly referenced in the xml files - is something behind the scenes causing these to be picked up and used though? If so, should every grid_def be duplicated? Or if not, do any of them need duplicating?

[Ed Hone (EdHone)]

It's a mixture of our current I/O implementation and expediency. The _cf domains are created during XIOS initialisation automatically as part of the lfric_core part of this PR. However, some of the checkpoint fields for lfric_atm are on grids that reference the UGRID versions of the domains, and these grids need a corresponding _cf version, which references the CF version of the domain instead of the UGRID one. It would probably be possible to automatically generate these grids as well (in lfric_core's I/O XIOS grids are dynamically generated according to which fields I/O is requested for), but in the interest of keeping this change as simple as possible I decided to just create definitions in the XML by hand. The CF grids that are present are the only ones needed to get all checkpointing to work.

[iboutle]

Thanks, that makes sense. So essentially any domain or grid referenced in the lfric_dictionary file (i.e. fields which are potentially checkpointed) will need to have a _cf version added?

Would you be able to document this somewhere - I would suggest adding a comment to the lfric_dictionary file itself, saying that any domain or grid referenced in here needs to have 2 versions (with and without the _cf) defined in the grid_def files