Migration to C++20 and C++23

[kdrienCG]

Hi everyone,

I've recently done a research work for a possible migration to C++ version 23.
This research was first started after the discussion around the cpptrace PR (#4042), and its alternative solution (std::stacktrace) coming with C++23, and transitioned into a report of the current state and possibilities to migrate to C++20 and C++23.

This document gather information on the current state of GEOS, the readiness of the systems around it against C++20 and C++23, an inventory of the interesting features coming in both versions with their system requirements, the problems (possible or already encountered) when migrating and some actions to take or discuss.

A goal of this document is also to (re)start the discussion around these possible migrations (here or in the previous discussion for C++20), but also to have a "common" place to have deeper infos on the readiness of each system and partners.

There are some part of the document that are made to be written in (e.g. the "Action" cells in CI/host-configs/ configs to mark whether they are still relevant, should be sunsetted, etc., or the minimum compiler/libs to support)

Current state

GEOS is currently on C++17. Some parts of the code already prepare for C++20.

Codebase characteristics

Characteristic
Dual formatting paths	Format.hpp both maintain {fmt} (when __cplusplus < 202002L), and std::format + concept
Custom span	geos::Span (~20 call sites) (std::span comes with C++20)
SFINAE	~60 files with enable_if
if constexpr	Already in ~70 files (C++17)
No C++20 language adoption yet	No <=>, ranges, coroutines, modules in first-party code (except Format.hpp)
Exception error model	GEOS_THROW / GeosExceptions.hpp - no std::expected

Documentation drift

Some of the documentation is not up to date with the reality.
The migration could be the right time to update it.

• CodeStyle.rst still claims that the CI uses Ubuntu 18.04, CentOS 7, CUDA 10.1, Xcode 11.2 while it is not the case.
• ContinuousIntegration.rst talks about using Ubuntu and CentOS, while it is Ubuntu and Rocky Linux

Discussions already started

GEOS#3223 - EPIC "Move forward to C++20":

• Some questions where asked/answered in this discussion whether each system had at least CUDA 12 and compilers for C++20. Most of the system do, only Pangea 3 at that time had incompatible drivers with CUDA 12 (but Pangea 3 support have been dropped).

thirdPartyLibs#351 - "move to c++20" (open):

• fmt -> 12.1.0 for C++20 compatibility.
• RAJA/Umpire/Camp bumps and BLT 0.7.2 for CUDA TPL builds.
• Hardcode TPL builds to C++20

Last task: "confirm geos compiles once it moves to c++20"

Features we might gain from C++20

1. Using the standard <format> instead of {fmt}
   Standard string formatting (rules between std::format and fmt can vary for decimals).

2. Using std::span instead of geos::Span
   Standard non-owning view of contiguous 1D buffer.

3. Concepts / requires
   Named type constraints on templates (cleaner & simpler than some complex enable_if/traits).

4. Spaceship operator <=>
   Automatically generated comparisons operators with a single <=> operator.

5. constinit / consteval
   Guaranteed compile time init or function evaluation (constexpr is not guaranteed to be).

6. Ranges
   Composable lazy views/algorithms over sequences.

Features readiness (C++20)

Feature	libstdc++	libc++	GCC 12	GCC 13	Clang 14	Clang 19–22	CCE 15/20	CUDA 12.x	HIP ROCm 6.x
(Language) -std=c++20	-	-	Yes	Yes	Yes	Yes	Yes	Yes	Yes
std::format	≥13	≥17	No	Yes	*	*	*	Host only + *	Host only + *
std::span	≥10	≥7	Yes	Yes	*	*	*	Host only + *	Host only + *
Concepts / requires	-	-	Yes	Yes	Yes	Yes	Yes	Host, device unverified	Host, device unverified
<=>	-	-	Yes	Yes	Yes	Yes	Yes	Host + device	Host + device
constinit	-	-	Yes	Yes	Yes	Yes	Yes	Host + device	Host + device
consteval	-	-	Yes	Yes	Partial	Yes	CCE15: Partial CCE20: Yes	Host + device	Host + device

* : Yes only if the linked STL meets the version requirements (libstdc++ or libc++ columns)

Features that might not be used directly (C++20)

1. Coroutines
   Suspendable functions (co_wait, co_yield, etc.) for async.

2. Modules (import)
   Replace header inclusion with compiled module interfaces, can reduce the compilation time.
   Difficult to implement in GEOS (and pre C++20 code bases) due to large refactors required and possible lack of support with compilers, debuggers, static analyzers, build systems, etc.

Minimum compilers and standard library implementations versions (C++20)

Starting a migration to C++20 should require us to agree on a minimum compiler.

GCC & libstdc++: TBD
Clang: TBD
libc++ (MacOS / -stdlib=libc++): TBD
CCE: TBD
CUDA: TBD
HIP / ROCm: TBD

Features we might gain from C++23

1. std::stacktrace (potentially replacing the work on cpptrace)
   Capture a portable backtrace. Standard for GEOS#4042

2. std::expected (and monadic and_then, etc.)
   Return either the value or a typed error without throwing. Monadic "version" can chain success/error pipelines (and_then, or_else, etc.) on expected
   It could be used in this code:

     try
     { self->group->cleanup( time, cycleNumber, 0, 0, domain ); }
     catch( std::out_of_range const & e )
     { std::cout<<"Target not found. Impossible output."<<std::endl; }
   

3. std::move_only_function
   Type-erased callable that can hold move-only lambdas (unlike std::function)

Features that might not be used directly (C++23)

1. std::print/println (against GEOS_LOG*)
   Directly prints a formatted string to stdout/stderr.

2. std::mdspan (against LvArray)
   Multi dimensional span, non owning-view of a flat block of memory as an N-D array.

Features readiness (C++23)

Feature	libstdc++	libc++	GCC 12	GCC 13	Clang 19–22	CCE 15/20	CUDA/HIP
-std=c++23	-	-	Yes	Yes	Yes	Unverified	Host, device unverified
std::stacktrace	≥12	-	Yes (if libstdc++ built with backtrace)	Yes (same)	*	*	Host only + *
std::expected	≥12	≥16	Yes	Yes	*	*	Host only + *
std::expected monadic (with and_then, etc.)	≥13	≥17	No	Yes	*	*	Host only + *
std::move_only_function	≥12	-	Yes	Yes	*	*	Host only + *
Features that might not be used directly
std::print/println	≥14	≥17/18	No	No	*	*	Host only + *
std::mdspan	≥16	≥17/18	No	No	*	*	Host uses std::, device uses cuda::std::/libhipcxx (not ISO)

* : Yes only if the linked STL meets the version requirements (libstdc++ or libc++ columns)

Minimum compilers and standard library implementations versions (C++23)

Starting a migration to C++23 should require us to agree on a minimum compiler.

GCC & libstdc++: TBD
Clang: TBD
libc++ (MacOS / -stdlib=libc++): TBD
CCE: TBD
CUDA: TBD
HIP / ROCm: TBD

Configurations and their support for C++20 and C++23

This section covers the different configurations available (some of them may be outdated or not used anymore) and the support of their compilers for versions 20 and 23 of the C++ language.

A configuration labeled Ready still requires to have the TPL rebuilt with the correct language version, and to flip any variable in GEOS. Compilers themselves support the standard.

Partial, unless described further, is for configurations using a compiler that does not fully support the language standard.

CI configurations

Config	Toolchain or Image	CPU/GPU	Status	C++20	C++23	Action
Ubuntu 24.04 GCC 13	geosx/ubuntu24.04-gcc13	CPU	Active	Ready	Ready	TBD
Ubuntu 24.04 Clang 20	geosx/ubuntu24.04-clang20	CPU	Active	Ready	Ready	TBD
Rocky 8 GCC 12	geosx/rockylinux8-gcc12	CPU	Active	Ready	Partial (language Ready)	TBD
Rocky 8 GCC 13	geosx/rockylinux8-gcc13	CPU	Active	Ready	Ready	TBD
Rocky 8 Clang 19	geosx/rockylinux8-clang19	CPU	Active	Ready	Ready	TBD
Rocky 9 Clang 22	geosx/rockylinux9-clang22	CPU	Active	Ready	Ready	TBD
Integrated tests	Ubuntu 24.04 GCC 12	CPU	Active	Ready	Partial (language Ready)	TBD
Ubuntu GCC 13 + CUDA 12.9.1	-gcc13-cuda12.9.1	CUDA	Active	Ready	Host Ready, device Partial	TBD
Ubuntu Clang 19 + CUDA 12.9.1	-clang19-cuda12.9.1	CUDA	Active	Ready	Host Ready, device Partial	TBD
Rocky8 GCC 13 + CUDA 12.9.1	-gcc13-cuda12.9.1	CUDA	Active	Ready	Host Ready, device Partial	TBD
Rocky8 Clang 19 + CUDA 12.9.1	commented	CUDA	-	-	-	TBD
Ubuntu GCC 12 CPU	commented	CPU	-	-	-	TBD
Ubuntu Clang 19 CPU	commented	CPU	-	-	-	TBD
macOS 14	commented (macos-14-xlarge)	CPU	-	-	-	TBD

• gcc 12 can support C++23 but the library features will be incomplete compared to gcc 13+.
• Rocky Linux + Clang + CUDA compile error ([EPIC] Move forward to C++20 #3223). Fixed?

host-configs/ configurations

These configurations are extracted from the host-configs/, some may be outdated, any corrections are welcome.

LLNL

File	Machine	Compiler	Backend	Pins	Status	C++20	C++23	Action
dane-...-gcc@12.1.1.cmake	Dane	GCC 12.1.1	CPU	BLT_CXX_STD c++17	Active	Ready	Partial	TBD
dane-...-gcc@13.3.1.cmake	Dane	GCC 13.3.1	CPU	c++17	Active	Ready	Ready	TBD
dane-...-llvm@14.0.6.cmake	Dane	Clang 14.0.6	CPU	c++17	Active	Partial	No	TBD
dane-...-llvm@19.1.3.cmake	Dane	Clang 19.1.3	CPU	c++17	Active	Ready	Ready	TBD
matrix-...-gcc@12.1.1-cuda@12.6.0.cmake	Matrix	GCC 12.1.1	CUDA 12.6 arch 90	c++17 + CUDA_STANDARD 17	Active	Ready	Partial	TBD
matrix-...-gcc@13.3.1-cuda@12.9.1.cmake	Matrix	GCC 13.3.1	CUDA 12.9 arch 90	c++17 + CUDA_STANDARD 17	Active	Ready	Ready	TBD
matrix-...-llvm@14.0.6-cuda@12.6.0.cmake	Matrix	Clang 14 + CUDA 12.6	CUDA	c++17 + CUDA_STANDARD 17	Active	Partial	No	TBD
matrix-...-llvm@19.1.3-cuda@12.9.1.cmake	Matrix	Clang 19 + CUDA 12.9	CUDA	c++17 + CUDA_STANDARD 17	Active	Ready	Ready	TBD
tuolumne-...-cce@20.0.0-rocm@6.4.3.cmake	Tuolumne	CCE 20	HIP gfx942 ROCm 6.4.3	c++17 + HIP_STANDARD 17	Active	Ready	Unverified	TBD
tuolumne-...-llvm-amdgpu@6.4.3-rocm@6.4.3.cmake	Tuolumne	amdclang++ 6.4.3	HIP gfx942	c++17 + HIP_STANDARD 17	Active	Ready	Unverified	TBD

ORNL

File	Machine	Compiler	Backend	Status	C++20	C++23	Action
frontier-base.cmake + frontier-cce@15.0.0.cmake	Frontier	CCE 15 + ROCm	HIP (HYPRE device HIP)	Active	Ready (#3223)	Unverified	TBD
ascent-gcc@8.1.1.cmake	Ascent	GCC 8.1.1 + CUDA/ESSL	CUDA (Power)	Legacy	Blocked (GCC 8)	No	TBD

TOTAL (Maple)

File	Compiler	Backend	Status	C++20	C++23	Action
maple-gcc12.2.0-...cmake	GCC 12.2	CPU	Active	Ready	Partial	TBD
maple-gcc12.2.0-cuda12.4.131-...cmake	GCC 12.2	CUDA 12.4	Active	Ready	Partial	TBD
maple-llvm17.0.4-...cmake	LLVM 17	CPU	Active	Ready	Ready	TBD
maple-llvm17.0.4-cuda12.4.131-...cmake	LLVM 17	CUDA 12.4	Active	Ready	Ready	TBD

Stanford

File	Compiler	Backend	Status	C++20	C++23	Action
sherlock-gcc10.cmake	GCC 10	CPU	Legacy	Partial	No	TBD
sherlock-gcc10-cuda12-sm70.cmake	GCC 10	CUDA 12 sm_70	Legacy	Partial	No	TBD

Apple

File	Compiler	Backend	Status	C++20	C++23	Action
macOS*.cmake	AppleClang	CPU (CUDA off)	host-config, commented in CI	Unverified	Unverified	TBD

LBL / NERSC

File	Machine	Compiler	Status	C++20	Action
cori-gcc@8.1.0.cmake	Cori	GCC 8.1	Legacy	Blocked	TBD
cori-intel.cmake	Cori	Intel + -std=c++14	Legacy	Blocked	TBD
corigpu-gcc.cmake	CoriGPU	GCC	Legacy		TBD
anag-gcc.cmake	ANAG	system g++	Unverified		TBD

Configurations

Machines	Status	C++20	C++23	Action
LLNL	Confirmed ready for CUDA12/modern compilers (#3223)	Ready	Unverified	TBD
ORNL Frontier / AMD HIP		Ready	Unverified	TBD
TotalEnergies Maple / Pangea3		Maple Ready, Pangea3 Unverified	Unverified	TBD
TotalEnergies Pangea4		Unverified	Ready	TBD
Stanford Sherlock	Legacy GCC 10 configs	Partial	No	TBD

Most of the information comes from #3223.

Dependencies/TPL

Some actions have already been done for C++20.

RAJA / CHAI / Umpire / Camp versions are currently bumped for C++20 in TPL #351
fmt is being updated to fmt 12.1.0 (TPL #351), or replaceable with std::format

Other TPL should be rebuild with C++20 or C++23 flags. I mostly didn't have any issues with TPL rebuilt on those versions (cf. next section Local attempt).

Local attempt (C++17 -> C++23)

Blocking points

I tried to migrate a local clone of GEOS from C++17 to C++23 on Pangea4, and used the gcc13 toolset for this.
Here are the problems/errors I encountered:

1. Migrating Format.hpp from using {fmt} to using the standard library <format>:
   In C++20, std::format (and even {fmt}) requires a compile-time format string std::format( "my msg: {}" <- this ). In TableData.cpp, columnFmt/rowFmt are built at runtime. A potential solution could be to use std::vformat()
   Some classes will complain about missing a formatter when using <format>. This is the case for Path.hpp

2. Implicit capture of this via [=] is deprecated in C++20.
   Multiple files in GEOS have this implicit capture, I have updated them in the branch
   One LvArray unitTests have this implicit capture (src/coreComponents/LvArray/unitTests/testTensorOpsEigen.cpp), it is not part of the branch

3. Failing testCompMultiphaseFlow and testCompMultiphaseFlowHybrid
Both tests fail a single time with LvArray CRSMatrixView.

PR #4088 fixes the issue with testCompMultiphaseFlow and testCompMultiphaseFlowHybrid

Passing procedure

The local attempt does compile and run on C++23 with a GCC 13 toolchain.

To do so, I rebuilt the TPL with the correct C++ 23 flags, with a host-config specifying the path to the GCC 13 compiler.

I however kept the {fmt} library at version 11, and did not update it to 12.1.0 (cf. Blocking points).

Then in GEOS, the Format.hpp switches (via #ifdef macros) to the std::format on C++ versions ≥20. This implementation causes errors so I manually forced the use of {fmt} for all versions, after a certain. Those first errors and their possible solutions are detailed in the "Blocking points" section.

I replaced all the implicit capture of this in GEOS lambdas. One is present in src/coreComponents/LvArray/unitTests/testTensorOpsEigen.cpp and has also been modified to be able to compile.

GEOS correctly compiles and launch, though further testing whether GEOS still produces the same results as before should be done.

Risks and blockers

Some systems could have incompatible drivers with CUDA and other "dependencies" (similar to Pangea3 discussed in #3223)

Needed contributions

There are still TBD (to be done) mentions in this document, they are open to anyone's contribution in order to finish this study.

Questions

Do some configs in the CI or in host-configs/ need to be sunset?

Are there any drivers incompatibilities?

There doesn't seem to have a CI config for HIP

Is macOS and its configs still used?

[kdrienCG]

I published a draft PR dedicated to a C++23 migration.