Repository navigation
Fix CHARMM PSF topology handling: cross-term width, term order, and topology under selection - #2275
Merged
jamesmkrieger merged 8 commits intoSep 15, 2026
Conversation
A CHARMM `!NCRTERM` record holds the two coupled dihedrals of a CMAP term, so eight atom indices, one term per line. `parsePSF` read four, which consumed only half of the section -- 571 of msbA's 1142 records -- and split each record it did read into two unrelated 4-tuples. `writePSF` then emitted those 4-tuples two to a line, so a file written from a parsed PSF disagreed with its own `!NCRTERM` count. `parsePSF` now reads and reshapes eight per term, `writePSF` writes one term per line, and `setCrossterms` requires the eight columns and says so. `evalCrossterms` takes its term width from the array rather than assuming four, so its neighbour map is built over all eight atoms of a term. Also fix a second way cross-terms went missing: the loop that reads the non-bonded exclusions stops on the first line containing `!`, which has already consumed the header of the section that follows. CHARMM puts `!NGRP` and `!NUMLP` there, but a file -- including one written by `writePSF` -- may go straight to `!NCRTERM`, in which case the scan for that header started past it and every cross-term was silently dropped. The line the loop stopped on is now checked before scanning on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The atom order inside an angle, dihedral, improper or CMAP cross-term is what identifies the term: the middle index of an angle is its vertex, a torsion's four indices are a sequence, and a cross-term's eight are two dihedrals. `setAngles`, `setDihedrals`, `setImpropers` and `setCrossterms` each began by sorting every term's indices, which turns a term into a different one. msbA's PSF gives its first angles as `2 1 3`, `2 1 4`, `2 1 5` -- all three about vertex 1 -- and they were stored as `1 2 3`, `1 2 4`, `1 2 5`, so `Angle.getVectors`, documented as returning the bond vectors that originate from the central atom, took atom 2 as the vertex of all three. Donors and acceptors are ordered pairs too -- a `!NDON` record is a donor and its hydrogen, a `!NACC` record an acceptor and its antecedent -- so they are no longer sorted or deduplicated either. The term-level `argsort` chains go with them: nothing downstream requires a canonical term order, and preserving the order read means a topology parsed from a file is written back out unchanged. `setBonds` and `setNBExclusions` keep their within-pair sort, which is load-bearing rather than cosmetic: those pairs ARE symmetric, so sorting each is what collapses (i, j) and (j, i) in the `np.unique` that follows, and for bonds it is also what guarantees i < j in the '%d %d' keys of `_bondIndex` that `Bond` looks up. Their row-level `argsort` chains are dropped as dead code, since `np.unique(..., axis=0)` already sorts the rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`topology.psf` is a hand-made ten-atom fixture -- two ALA residues -- with every topology section populated, laid out the way CHARMM writes one, and chosen so that no term's indices are already in sorted order. Five of the seven tests fail without the two preceding commits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`evalBonds` and its siblings build `numbonds`, `numangles`, `numdihedrals`, `numimpropers`, `numdonors`, `numacceptors`, `numnbexclusions` and `numcrossterms` with `np.bincount`, which returns an array only as long as the highest atom index that appears in a term. Any atom after the last one taking part in a term of that kind simply had no entry, so the arrays came out shorter than the group -- a ten-atom test topology whose only donor is atoms 1-2 got a `numdonors` of length 2. That breaks the documented use of these arrays. `setBonds` offers `'numbonds 0'` for selecting ions, which are exactly the atoms in no bond and so the ones past the end of a short array, and `Atomic.copy` reads every data label through `data[self._indices]`, which raised `IndexError` on any selection reaching past the end. Each of these functions is already given `n_atoms` and uses it to size the neighbour map; the bincount now gets it too as `minlength`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`AtomSubset._iterAngles` and its siblings had two paths: filter the parent group's terms by membership, or -- when the selection is small relative to the group -- rebuild each term from the neighbour map. The map records which atoms share a term, not where in it they sit, so a rebuilt term came out with the pivot atom first: the angle 2-1-3 arrived as 1-2-3, naming atom 2 as the vertex. Which path ran depended on the ratio of selection size to group size, so the same selection of a small group and of a large one disagreed. Angles, dihedrals, impropers, donors, acceptors and cross-terms are all ordered, so the map cannot be a faithful source for any of them and the shortcut is gone; each now filters the term array, which is what the other path already did. Only bonds and non-bonded exclusions are symmetric, and they keep their map path. `_iterCrossterms` also still assumed four atoms per term and unpacked its rows into three, which raised `ValueError` on any group carrying CMAP terms. Separately, `getBonds` built each `Bond` on the pointer rather than on the parent group, as `iterBonds` does. A `Bond` reads `_bondOrders`, which only an `AtomGroup` has, so `selection.getBonds()` and `selection.numBonds()` raised `AttributeError` for every selection. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Atomic.copy` trimmed and carried over `_bonds` and stopped there, so copying a selection of a CHARMM system -- what cropping a solvated box amounts to -- silently returned an AtomGroup with no angles, dihedrals, impropers, donors, acceptors, non-bonded exclusions or CMAP cross-terms. Nothing warned; the topology was simply gone. All eight sections are now carried, whole for a whole-group copy and trimmed and renumbered for a selection. `trimBonds` already did the renumbering and was only two-atom in the one line that tests membership, so it becomes `trimTerms`, which takes any term width and keeps both the order of the terms and the atom order within each; `trimBonds` stays as its two-atom case. A term only partly inside the selection cannot be renumbered and is dropped, as bonds already were. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The topology arrays hold indices into the AtomGroup. For a selection those are not the positions the atom records are written in -- they run up to the size of the parent group -- and `writePSF` wrote them unchanged, so a PSF written from a selection had bonds pointing past its own `!NATOM` count. Writing four atoms out of ten gave bonds `3 4 / 3 5 / 5 6`; on a 258,469-atom crop of a 720,343-atom system the indices would have run to 720,343. Every section is now mapped onto the 1-based position of its atom in the file being written. A whole-group write is unaffected, its indices already being its positions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ten more tests on the same fixture: that the num* arrays are per-atom, that a selection delegates its terms in parent numbering and in order, that `numBonds` works at all, that `copy()` carries all eight sections renumbered and drops the partly-selected ones, that a written subset's indices stay inside its own `!NATOM` count, and that copy-then-write agrees with writing the selection directly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jamesmkrieger
force-pushed
the
jamesk-claude/psf-topology-order
branch
from
September 10, 2026 17:01
fb22b42 to
87a0fb7
Compare
Contributor
Author
|
the sporadic insty parallel bug is fixed in PR #2277 |
karolamik13
approved these changes
Sep 15, 2026
karolamik13
left a comment
Contributor
There was a problem hiding this comment.
Looks good. I tested on a few PSF files.
Code:
from prody import *
import numpy as np
name = 'test.psf'
p = parsePSF(name)
writePSF(name[:-4]+'_out.psf', p)
p2 = parsePSF(name[:-4]+'_out.psf')
for x in ['bonds', 'angles', 'dihedrals', 'impropers', 'donors', 'acceptors', 'crossterms']:
print(x, np.array_equal(getattr(p, '' + x), getattr(p2, '' + x)))
print(p._crossterms.shape)
print(p2._crossterms.shape)
Output:
bonds True
angles True
dihedrals True
impropers True
donors True
acceptors True
crossterms True
(1012, 8)
(1012, 8)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ProDy cannot currently round-trip a CHARMM PSF:
parsePSFfollowed bywritePSFproduces a file with a different topology from the one it read. Two independent causes, plus a third that hides one of them.1. A CMAP cross-term is eight atom indices, not four
A
!NCRTERMrecord holds the two coupled dihedrals of a CMAP term — eight indices, one term per line. From msbA's PSF:that is φ (20-22-24-31) and ψ (22-24-31-33) of one residue.
parsePSFreadcount=n_crossterms*4, which consumed only the first 571 of the file's 1142 records and split each one into two unrelated 4-tuples — filling the array to the expected count with terms that do not exist.writePSFthen wrote those 4-tuples two to a line, so the output disagreed with its own!NCRTERMcount.Fixed in the parser, the writer,
setCrossterms(which now requires the eight columns and documents what they are), andevalCrossterms, which now takes the term width from the array instead of assuming four, so its neighbour map covers all eight atoms.2. Sorting the indices inside a term changes which term it is
The atom order within an angle, dihedral, improper or cross-term is what identifies it: an angle's middle index is its vertex, a torsion's four are a sequence, a cross-term's eight are two dihedrals. All four setters began with
.sort(1). msbA's PSF gives its first angles as— all three about vertex 1 — and they were stored as
1 2 3,1 2 4,1 2 5, soAngle.getVectors, documented as returning the bond vectors that originate from the central atom, took atom 2 as the vertex of all three.Donors and acceptors are ordered pairs too (
!NDONis a donor and its hydrogen,!NACCan acceptor and its antecedent), so they are no longer sorted or deduplicated either. The term-levelargsortchains go with the sorts: nothing downstream needs a canonical term order, and keeping the order read is what makes the round-trip exact.setBondsandsetNBExclusionskeep their within-pair sort deliberately — those pairs really are symmetric, so sorting each is what collapses(i, j)and(j, i)in thenp.uniquethat follows, and for bonds it is also what guaranteesi < jin the'%d %d'keys of_bondIndexthatBondlooks up. Their row-levelargsortchains are dropped as dead code, sincenp.unique(..., axis=0)already sorts rows.3.
!NCRTERMwas dropped when it directly follows the exclusionsThe loop reading
!NNBstops on the first line containing!— which has already consumed the next section's header. CHARMM puts!NGRPand!NUMLPthere, so real files survived, but a file that goes straight to!NCRTERM— including one written bywritePSF— had the scan start past the header, and every cross-term was silently dropped with no error. The line the loop stopped on is now checked before scanning on.Testing
prody/tests/trajectory/test_psffile.pyand a hand-made ten-atom fixturetopology.psfwith every section populated, laid out the way CHARMM writes one, with no term's indices already sorted. 5 of the 7 tests fail without these fixes.prody/tests/atomic,trajectoryandproteins: 701 passed. The 7 failures are pre-existing and environmental (nommtf, noHpb, network fetches) and are unaffected.Systembuilt from the original PSF and from the ProDy round-tripped PSF give identical potential energy, −9,820,146.3 kJ/mol, with the same 704,223 constraints.Not addressed here:
writePSFstill omits!NGRP,!NUMLPand the per-atom!NNBpointer array. OpenMM tolerates that; CHARMM and NAMD may not.🤖 Generated with Claude Code