GROMACS 2020-2024 patches: fix the pressure, virtual sites, MTS, HREX, COMMITTOR, rerun, energy minimization, expanded ensemble and more by invemichele-peptone · Pull Request #1505 · plumed/plumed2

invemichele-peptone · 2026-10-15T07:24:32Z

Description

#1504 fixed checkpointing in the GROMACS 2025/2026 patches. Going through the other GROMACS patches, I found many bugs in those for GROMACS 2020.7 to 2024.3, some of them there since the first patches. This PR fixes them for all the GROMACS versions in v2.9. The GROMACS 2022.5, 2023.5 and 2024.3 patches are identical in v2.9, v2.10 and master, so merging this into v2.10 fixes them there too. The GROMACS 2025/2026 patches and the PLUMED 2.10 kernel get their fixes in #1506 (for v2.10), and #1507 ports both PRs to master.

Each commit fixes one bug in all the affected patches, with a CHANGES entry.

Wrong results without any warning

Pressure (all versions, since the GROMACS 2019 patch). PLUMED runs inside do_force(), so its forces were in the force buffer when GROMACS computed the virial, and md.cpp then also added PLUMED's own virial: the virial of PLUMED's forces was counted twice, unless the input needed the energy. This affects the pressure, and so NPT simulations, with any bias that applies forces. On the steps when GROMACS computes the virial, PLUMED now runs after do_force(), as it already did when it needs the energy (only adding PLUMED's virial would not do: GROMACS's single sum is wrong for atoms in different periodic images).
Energy minimization (all versions). The local atom indices were passed to PLUMED only at initialization, but the minimizer repartitions the atoms: PLUMED then used the wrong atoms. Since GROMACS 2022 this happened also on a single rank, where domain decomposition is the default, i.e. in every minimization with PLUMED.
Virtual sites (all versions, since PLUMED 2.0). When the input needs the energy, PLUMED runs after GROMACS has spread the forces on the virtual sites and cleared them: the forces PLUMED applied on virtual sites (e.g. TIP4P M sites) were lost, in MD and minimization. They are now spread, with their virial; in MD only on the steps when PLUMED uses virtual sites (from createFullList, the same on all ranks), since the extra spread costs about 45 us per step (2% of a GPU step, 68k-atom TIP4P box).
MTS (2021-2024). With the energy, PLUMED's forces and the rescaling for a bias on ENERGY went to the normal force buffer, which the update does not use on the slow steps; on the fast steps the energy lacked the slow terms. The forces now go to the combined buffer, and mdrun stops with an error if the energy is needed on a fast step.
HREX energies (2024; 2022-2024 with more than one rank per replica). The 2024 patch computed the energy of the swapped coordinates with the workload of the previous step (often without the energy), so exchanges were almost never accepted. With domain decomposition, the patches also skipped the updates GROMACS does after repartitioning, so the swapped energy could be wrong by thousands of kT.
HREX double update (all versions). The do_force() call on the swapped coordinates also ran PLUMED, for the previous step: e.g. METAD deposited a second hill and PRINT printed a second line on the step before an exchange, when their stride divided it.
GPU force buffer operations (all versions). PLUMED did not count as CPU force work: with GPU buffer ops on a single PP rank and nothing else on the CPU, its forces were lost on the steps without the virial.
Expanded ensemble (2022-2024). The PLUMED bias replaced GROMACS's lambda weights for all lmc-stats but Wang-Landau, instead of only for lmc-stats = no as in the 2020/2021 patches; without -plumed, Wang-Landau with lmc-forced-nstart did the forced pass unlike GROMACS. In the 2024 patch, PLUMED's lambda also lagged one move behind.
-nex (all versions). Multiple exchanges ignored the PLUMED bias (and could hang with several ranks per replica): mdrun now stops with an error.

Hangs, crashes and requests that were ignored

COMMITTOR and other stop requests (all versions). Only the simulation that asked stopped, so other walkers hung; the 2022-2024 patches also wrote no final checkpoint. The request is now a stop condition of the GROMACS stop handler, as in the 2025/2026 patches, and reruns no longer ignore it.
mdrun -rerun (2020, 2024). The 2020 patch did not patch the rerun, and the 2024 patch lost most of it (in the update from 2024.2 to 2024.3): it stopped with "isEnergyNeeded should be only used after plumed initialization".
md-vv (2021-2024). The condition that disables the modular simulator with PLUMED was misplaced, so md-vv with PLUMED stopped with an error instead of using the legacy simulator, as with the 2020 patch.

Errors instead of silent misbehaviour

Shell (polarizable) particles: PLUMED was silently not applied (the shell relaxation never set it up).
Several thread-MPI ranks: they all shared the same PLUMED object.
-hrex with PLUMED exchange patterns (RANDOM_EXCHANGES): the existing check never triggered.

Target release

I would like my code to appear in release v2.9

Type of contribution

changes to code or doc authored by PLUMED developers, or additions of code in the core or within the default modules
changes to a module not authored by you
new module contribution or edit of a module authored by you

Copyright

I agree to transfer the copyright of the code I have written to the PLUMED developers or to the author of the code I am modifying.
the module I added or modified contains a COPYRIGHT file with the correct license information. Code should be released under an open source license. I also used the command cd src && ./header.sh mymodulename in order to make sure the headers of the module are correct.

Tests

I added a new regtest or modified an existing regtest to validate my changes.
I verified that all regtests are passed successfully on GitHub Actions.

The CI does not build GROMACS, so I wrote regression tests for PLUMED with GROMACS, and proposed them to the PLUMED testcenter in plumed-testcenter/plumed-testcenter#95, where they would run every week on all the GROMACS versions that PLUMED can patch. They compare PLUMED with references that do not depend on the versions (e.g. a RESTRAINT against the GROMACS pull code, the same run with and without ENERGY, or on 1, 2 and 3 ranks), so the same 22 tests run on any combination of GROMACS, patch and kernel. With GROMACS 2020.7, 2021.7, 2022.5, 2023.5 and 2024.3 patched with this branch, all of them pass, also with the kernels of v2.10 and master (with their fixes); with v2.9, they fail exactly on the bugs above. The table below shows the numbers of the main cases. I also tested by hand: GROMACS 2020.7, 2021.7, 2022.5, 2023.5 and 2024.3, built with MPI and patched (runtime mode) with this branch and with v2.9, PLUMED v2.9 kernel; one GROMACS 2024.3 CUDA build for the GPU test. Systems: a TIP4P water box (3636 atoms), alanine dipeptide in vacuum, and a SW water dimer for shells.

test	v2.9	this PR
water, restraint between two oxygens: virial ZZ at step 0 (-55120 without PLUMED; GROMACS 2026 patch: -51653)	-48179 (restraint counted twice)	-51653, also on 2 ranks
minimization, restraint between two oxygens, 1 rank	wrong atoms from the 4th step (2022-2024)	same distances on 1 and 2 ranks, in all versions
minimization, restraint between two M sites, 2 ranks	NaN at step 18	same as on 1 rank
MD, restraint between two M sites: step-0 forces with ENERGY in the input, against without	off by 6.9e3 kJ/mol/nm	identical
MTS (2021-2024), ENERGY printed every 2 steps: restraint trajectory against without ENERGY	off by 0.045 nm in 10 steps	same (1e-5 nm)
MTS, ENERGY needed every step	runs with the wrong energy	error
`-hrex`, two identical replicas, no bias, 1 rank each: energy difference	24-38 kT (2024.3), 1e-8 kT (others)	1e-8 kT in all
the same with 2 ranks per replica (water, PME)	up to 55 kT (2022.5), 1.2e3 kT (2023.5), 2e28 kT then crash (2024.3)	0
`-hrex`, METAD PACE=3, exchanges every 10 steps	second hill at step 9, duplicate COLVAR lines	none
restraint, nonbonded and PME on GPU, `GMX_USE_GPU_BUFFER_OPS=1` (2024.3): distance after 50 steps (0.283 nm without buffer ops; 1.039 nm without restraint)	0.954 nm	0.283 nm
expanded ensemble (2023.5), Wang-Landau with forced nstart, no `-plumed`	forced pass (0,0,0,1,1,1,2,2,2,3,...)	normal moves
expanded ensemble, metropolis-transition, PLUMED only printing	weights stay 0	updated by GROMACS
expanded ensemble (2024.3): PLUMED's lambda against GROMACS's state	4 of 21 differ	all equal
COMMITTOR, 1 and 2 ranks	2020-2021: stop 1 step after; 2022-2024: at the commit step, no final checkpoint	stop within 10 steps, with a final checkpoint
COMMITTOR, 2 walkers	hang	both stop at step 201
rerun with `-plumed`	error (2020, 2024)	works on 1 and 2 ranks, same CVs in all versions
rerun with COMMITTOR committing at the first frame	all 6 frames	stops at frame 1
md-vv with PLUMED	modular simulator error (2021-2024)	runs
`-nex 4` with PLUMED	runs, bias ignored	error
shell particles	PLUMED silently not applied	error

Also checked:

Unchanged results: a bias on ENERGY on 1 and 2 ranks and OPES with two MPI walkers give bitwise identical COLVAR, STATE and energies before and after (with -dlb no). METAD gives identical hills on 1 rank, with a different pressure (the fixed virial); on 2 ranks the hills are identical for about 1400 steps and then diverge by round-off, as PLUMED's forces are now added after the other forces on the virial steps.
PLUMED 2.10 kernel: with GROMACS 2022.5-2024.3, the same tests pass with the kernel of GROMACS 2025/2026 patches and kernel: fix the virial with an ENERGY bias, a replica exchange deadlock, COMMITTOR, virtual sites, MTS and shells #1506.
Patch files: patching a clean GROMACS tree gives exactly the files of this branch.

A side effect to know: with multiple time stepping, on the steps when PLUMED now runs after do_force() (the virial steps, and the energy steps as before), PLUMED applies its forces to the combined buffer that the update uses, so the forces written to the trajectory on those steps do not include them; the dynamics are unaffected (tested against the pull code).

Not changed, but worth knowing: in energy minimization PLUMED's bias energy is not part of the energy GROMACS minimizes, and conjugate gradients passes step -1; the PLUMED lambda weights are set after GROMACS computed the weighted energies of the step, so they act one expanded step later; -update gpu, when forced, is refused by the 2023/2024 patches and only gives a warning with 2021/2022.

Merging into v2.10 and master

GROMACS 2022.5, 2023.5 and 2024.3: the patches are identical in v2.9, v2.10 and master, and merge cleanly.
GROMACS 2020.7 and 2021.7 are not in v2.10: merging gives modify/delete conflicts, and adds new rerun.cpp and sim_util.cpp pairs (with their .preplumed) under patches/gromacs-2020.7.diff/, all to be removed.

🤖 Generated with Claude Code

…path When PLUMED needs the potential energy, it runs after do_force(), after GROMACS has spread the forces on the virtual sites and cleared them. The forces PLUMED applied on virtual sites were then lost, in MD and in energy minimization. They are now spread with their virial, as GROMACS does for the forces with a direct virial. In MD, only on the steps when the atoms that PLUMED uses include virtual sites: that list is the same on all ranks, so that all take the same decision, and inputs that only bias the energy, e.g. OPES multithermal, do not pay for the extra spread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ime stepping With multiple time stepping, the update uses the combined force buffer on the slow steps, but PLUMED applied its forces, and the rescaling for a bias on the energy, to the normal buffer when it needed the potential energy: they were lost. They now go to the combined buffer on the slow steps. On the fast steps, the slow forces and their energy are not computed, so the energy passed to PLUMED was wrong: mdrun now stops with an error if PLUMED needs the energy on such a step, as with the GROMACS 2025 and 2026 patches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ks to When PLUMED asked to stop (e.g. COMMITTOR), the patches ended the loop of the simulation that asked only. With multiple simulations, which share state when PLUMED is used, the others then hung. The GROMACS 2022-2024 patches also ended it right after the force calculation of that step, so no checkpoint was written at the last step. PLUMED's request is now a stop condition of the GROMACS stop handler, as in the GROMACS 2025 and 2026 patches: GROMACS communicates it to all the ranks and the simulations at the next step at which it communicates signals (every nstglobalcomm steps, and at least every 200 steps between multiple simulations), and mdrun stops there, writing a checkpoint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The GROMACS 2020 patch did not patch the rerun, and the GROMACS 2024 one lost most of it (when updated from 2024.2 to 2024.3): PLUMED was never initialized, and do_force() stopped with "cmd("isEnergyNeeded") should be only used after plumed initialization". The rerun is now patched as in the GROMACS 2021, 2022 and 2023 patches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…mization step In energy minimization, the patches passed the local atom indices to PLUMED only at initialization, but GROMACS repartitions the atoms during the minimization (at every evaluation with nstlist > 0). PLUMED then read the positions of, and applied its forces to, the wrong atoms. Since GROMACS 2022 this happened also on a single rank, where domain decomposition is used by default: in a TIP4P water test with a restraint between two oxygens, the restrained distance jumped between unrelated values from the fourth step, and with a restraint between two M sites the minimization stopped with a NaN energy on 2 ranks. The indices are now passed before each evaluation, as the MD patch does after each repartitioning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

With -hrex, the patches call do_force() on the swapped coordinates to get their energy, without setting PLUMED up for that call. Since the call to PLUMED moved inside do_force(), PLUMED then calculated and updated again with the data of the previous step: e.g. METAD deposited a second hill and PRINT printed a second line on the step before an exchange, whenever their stride divided it. PLUMED is now switched off for that call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…rticles With shell (polarizable) particles, GROMACS relaxes the shells with relax_shell_flexcon(), which the patches never set PLUMED up for: PLUMED was silently not applied (no output, no bias) in MD and in reruns. Making it work would need the positions and forces of the shell iterations to be passed to PLUMED inside do_force(), so mdrun now stops with an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Since the GROMACS 2019 patch, PLUMED runs inside do_force() and adds its forces to the force buffer before GROMACS computes the virial from it; md.cpp then also adds the virial computed by PLUMED. The virial of PLUMED's forces was then counted twice, and the pressure was wrong, unless the PLUMED input needed the potential energy (then PLUMED runs after do_force()). Just not adding PLUMED's virial would not be right either: GROMACS's single sum over the force buffer is wrong for forces between atoms that are not in the same periodic image, e.g. with domain decomposition. On the steps when GROMACS computes the virial, PLUMED now runs after do_force(), as when it needs the energy, so that also its forces on virtual sites and the MTS buffers are handled as there. Energy minimization and rerun always compute the virial, so PLUMED always runs after do_force() there. TIP4P water, restraint between two oxygens, virial ZZ at step 0: -55119.9 without PLUMED, -48179.4 with it (the restraint counted twice), now -51653.4, as when ENERGY is in the input and as with the GROMACS 2026 patch, also on 2 ranks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Since GROMACS 2024, do_force() takes its workload (energy, pair search...) from runScheduleWork_ instead of from flags, and the patch did not set it up for the do_force() call on the swapped coordinates of -hrex: it used the workload of the previous step, often without the energy, and without a pair search after the repartitioning. With two replicas with the same Hamiltonian and no PLUMED bias, the energy difference was 24-38 kT instead of 0 and no exchange was accepted; it is now 1e-8 kT, as with the GROMACS 2023 patch, and all exchanges are accepted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…le move The GROMACS 2024 patch lost the update of the lambda state passed to PLUMED after an expanded ensemble move, which the GROMACS 2020-2023 patches do. PLUMED then saw the previous state until the next move. In an alanine test with four lambda states, PLUMED's lambda differed from GROMACS's state in 4 of 21 samples; now in none, as with the GROMACS 2023 patch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… with lmc-stats = no The GROMACS 2020 and 2021 patches use the PLUMED bias as the expanded ensemble weights only with lmc-stats = no, and then skip the forced initial pass through the lambda states (lmc-forced-nstart). When the patch was ported to GROMACS 2022, both conditions became "not Wang-Landau": - with PLUMED, its bias replaced the weights of GROMACS for all lmc-stats but Wang-Landau, e.g. with metropolis-transition the weights stayed at zero with a PLUMED input that only printed; - without PLUMED, "!plumedswitch || !EWL()" is always true, so Wang-Landau with lmc-forced-nstart did the forced pass, unlike unpatched GROMACS. The conditions, and the Wang-Landau printout, are now as in the GROMACS 2020 and 2021 patches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

With -hrex and domain decomposition, the patches repartition the atoms for the swapped coordinates and back, but did not do what GROMACS does after repartitioning: resize the update buffers (all versions), and update the long-range nonbonded module, e.g. CPU PME on the PP ranks (2022-2024). The energy of the swapped coordinates could then be wrong. Two identical TIP4P water replicas on 2 ranks each, 20 exchange attempts: the largest energy difference was 55 kT (2022.5), 1.2e3 kT (2023.5) and 2e28 kT, then a crash (2024.3), instead of 0; it is now 0 (round-off). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…d PLUMED The multiple-exchange branch of the replica exchange (-nex) does not add the PLUMED bias to the exchange criterion, nor calls "GREX calculate", which the other ranks of each replica wait for in "GREX prepare". The bias was then ignored, and runs with more than one rank per replica could hang. mdrun now stops with an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…eading them The check that -hrex is not used with the exchange patterns set by PLUMED (e.g. RANDOM_EXCHANGES) read the flag right after setting it to zero, so it never triggered: the patch then computed the HREX energies for the usual neighbour while PLUMED chose another partner. The check now comes after the flag is read from PLUMED. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The rerun patch set isLastStep when PLUMED asked to stop, but GROMACS then overwrote it with the result of reading the next frame, so the rerun went on. It is now kept: a rerun with COMMITTOR committing at the first frame now stops there, instead of going through all 6 frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The GROMACS 2020 patch disables the modular simulator when PLUMED is used, so that md-vv falls back to the legacy simulator, which supports PLUMED. When ported to GROMACS 2021 and 2022, the PLUMED condition was applied to the membed argument of checkUseModularSimulator() instead, inverted; in the GROMACS 2023 and 2024 patches it moved to the membed argument of get_nthreads_mpi(), lifting the single-rank limit of -membed with PLUMED. md-vv with PLUMED then stopped with "PLUMED is not yet compatible with GROMACS new modular simulator" (unless GMX_DISABLE_MODULAR_SIMULATOR was set). The condition is now on the result of checkUseModularSimulator(), as in the GROMACS 2020 patch, and md-vv with PLUMED runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…I ranks The patches keep PLUMED in a global object, which all the thread-MPI ranks of a thread-MPI build share: with more than one rank, PLUMED then gave wrong results. The patch configuration documents this, but mdrun ran anyway; it now stops with an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

PLUMED adds its forces to the CPU force buffer inside do_force(), but GROMACS did not count it as local force work on the CPU. With GPU force buffer operations and nothing else computed on the CPU (e.g. nonbonded, PME and bondeds on the GPU), on a single PP rank, GROMACS then did not add the CPU buffer to the forces reduced on the GPU, and copied those back over it: the forces of PLUMED were lost on all the steps without the virial. PLUMED now counts as local force work on the CPU (in sim_util.cpp, and in decidesimulationworkload.cpp for GROMACS 2024, both now patched for that). TIP4P water, restraint between two oxygens, nonbonded and PME on the GPU, 50 steps (GROMACS 2024.3 with CUDA): the restrained distance ends at 0.283 nm, but at 0.954 nm with GMX_USE_GPU_BUFFER_OPS=1 (1.039 nm without the restraint); now at 0.283 nm in both cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

carlocamilloni · 2026-10-15T07:34:57Z

@invemichele-peptone thanks a lot!

invemichele · 2026-10-15T07:48:27Z

sorry, I did not expect you to merge so quickly, I was still running a final review. I might have a few extra small things to add in a few minutes.

invemichele-peptone · 2026-10-15T10:33:35Z

The extra things are in #1508: the fix for GPU force buffer operations did not work with GROMACS 2020-2023, the constraint virial lacked PLUMED's forces with MTS, plus messages and CHANGES. The ones for the GROMACS 2025/2026 patches are in #1509, and #1507 ports both to master.

🤖 Generated with Claude Code

…with 2020-2023, constraint virial with MTS, messages and CHANGES) (#1508)

invemichele-peptone and others added 18 commits October 14, 2026 16:44

carlocamilloni merged commit c34e5a8 into plumed:v2.9 Oct 15, 2026
11 checks passed

invemichele-peptone mentioned this pull request Oct 15, 2026

GROMACS 2020-2024 patches: follow-up of #1505 (GPU buffer operations with 2020-2023, constraint virial with MTS, messages and CHANGES) #1508

Merged

2 of 7 tasks

invemichele-peptone mentioned this pull request Oct 15, 2026

GROMACS 2025/2026 patches: follow-up of #1506 (constraint virial with MTS, shells in reruns and with ENERGY, rerun stop, docs) #1509

Merged

2 of 7 tasks

carlocamilloni pushed a commit that referenced this pull request Oct 15, 2026

GROMACS 2020-2024 patches: follow-up of #1505 (GPU buffer operations …

8e9e512

…with 2020-2023, constraint virial with MTS, messages and CHANGES) (#1508)

invemichele-peptone deleted the gromacs-legacy-fixes branch October 15, 2026 14:48

Navigation Menu

Repository navigation

GROMACS 2020-2024 patches: fix the pressure, virtual sites, MTS, HREX, COMMITTOR, rerun, energy minimization, expanded ensemble and more - #1505

GROMACS 2020-2024 patches: fix the pressure, virtual sites, MTS, HREX, COMMITTOR, rerun, energy minimization, expanded ensemble and more#1505
carlocamilloni merged 18 commits into
plumed:v2.9from
invemichele-peptone:gromacs-legacy-fixes

invemichele-peptone commented Oct 15, 2026 •

edited

Loading

Uh oh!

carlocamilloni commented Oct 15, 2026

Uh oh!

Uh oh!

invemichele commented Oct 15, 2026

Uh oh!

invemichele-peptone commented Oct 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

invemichele-peptone commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Target release

Type of contribution

Copyright

Tests

Merging into v2.10 and master

Uh oh!

carlocamilloni commented Oct 15, 2026

Uh oh!

Uh oh!

invemichele commented Oct 15, 2026

Uh oh!

invemichele-peptone commented Oct 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

invemichele-peptone commented Oct 15, 2026 •

edited

Loading