Skip to content

Fix MSA files that writeMSA writes and parseMSA cannot read back - #2283

Open
satyaorz wants to merge 1 commit into
prody:mainfrom
satyaorz:fix/msa-write-parse-roundtrip
Open

satyaorz wants to merge 1 commit into
prody:mainfrom
satyaorz:fix/msa-write-parse-roundtrip

Conversation

@satyaorz

@satyaorz satyaorz commented Oct 10, 2026 •

Copy link
Copy Markdown

Disclosing up front: this was put together with help from Claude (Anthropic's coding agent). I reproduced and verified everything below before opening the PR.

writeMSA writes SELEX, Stockholm, CLUSTAL and PIR files that parseMSA cannot read back, or reads back with different labels or sequences. On Python 3 the CLUSTAL and PIR writers do not work at all. Before → after, writing a small MSA and parsing it back:

case master this PR
.aln TypeError: 'float' object cannot be interpreted as an integer (range(n/60)) round-trips
.ali same TypeError round-trips
.slx with a label longer than 31 characters label and sequence are spliced together round-trips
.sth with a 500-character label same, or a segfault in parseSelex for some label sets round-trips
.slx with 12000 columns IOError: failed to parse SELEX/Stockholm file at line 2 (line longer than the fixed buffer) round-trips

SELEX / Stockholm (msaio.c)

  • parseSelex. Previously it took the column where the first sequence starts and read every line at that column. That broke for labels of different lengths, labels longer than the padding, and lines longer than SELEXLINELEN. Now each line is split into a label (everything before the last whitespace-separated item) and a sequence. Lines are read whole, growing the buffer as needed. Alignments split into blocks separated by blank lines are joined, and a block whose labels or row count differ from the first block's raises IOError. Markup lines (#) are skipped, and nothing after // is read.
  • writeSelex. It now always leaves at least one space between a label and its sequence. It also no longer writes labels longer than the padding into a fixed-size buffer, and the label's bytes are released only after they are written (they were Py_DECREFed before use).
  • Labels. Labels are decoded as UTF-8 with replace, so the C path and MSAFile agree on non-ASCII labels.

Python readers and writers (msafile.py)

  • MSAFile. It now reads multi-block SELEX/Stockholm the same way as the C parser, and also reads CLUSTAL and PIR. It can write CLUSTAL (written on close(), since CLUSTAL interleaves sequences) and PIR. A Stockholm writer on a stream now writes the closing // too.
  • parseClustal.
    • Handles blocks, conservation lines and the residue counts clustalw 2 writes.
    • Accepts the headers Biopython accepts (MUSCLE, PROBCONS, MSAPROBS, Kalign).
    • Allows labels that contain spaces.
  • writeClustal. Pads labels to the longest one instead of a fixed 16 characters, and writes a CLUSTAL W header that Biopython reads.
  • parsePIR. Takes the line after each >P1; line as the description, whatever its first field, and no longer keeps the terminal * as a residue.
  • writePIR. Writes the full label by default, so it can be read back.
  • splitSeqLabel. In a label containing /, only a trailing /start-end is split off, so labels like sp/inner/4-11 or x/1/2 keep their full identifier.

Tests

  • New file prody/tests/sequence/test_msa_formats.py. It has round trips for all four formats through writeMSA/parseMSA and through MSAFile, on plain and .gz files and on streams. It also reads fixtures written the way HMMER/Pfam, clustalw 1/2 and MODELLER write them, and checks that mismatched blocks raise IOError. On master these tests fail, and the run segfaults in parseSelex partway through.
  • Fixtures are added as prody/tests/datafiles/msa_formats_*, which the existing datafiles/msa* package-data pattern already covers.
  • Results.
    • prody/tests/sequence: 254 passed. The one failure is testBuildMSAclustalw, which needs the clustalw binary I don't have installed locally.
    • Full suite: no other failures beyond missing optional dependencies (openmm, pdbfixer, mmtf, requests, clustalw).
    • Valgrind is clean on the C parser over a set of malformed inputs (empty files, CRLF, NUL bytes, very long lines and labels, ragged blocks).
  • Interop. Biopython's AlignIO reads the Stockholm, CLUSTAL and PIR files written here, and ProDy reads Biopython's.

SELEX, Stockholm, CLUSTAL and PIR files written by writeMSA or MSAFile
could not be read back, or came back with different labels or sequences:

- parseSelex read every line at the column where the first sequence
  started, so labels of different lengths or longer than the padding were
  spliced into the sequences, and lines longer than the fixed buffer
  failed; it now splits each line into label and sequence, reads whole
  lines, joins blocks and checks that each block repeats the first one
- writeSelex always leaves a space between label and sequence, no longer
  overflows its line buffer, and releases the label only after writing it
- writeClustal and writePIR failed on Python 3 (range of a float)
- MSAFile reads multi-block SELEX/Stockholm, CLUSTAL and PIR, and writes
  CLUSTAL and PIR; a Stockholm writer on a stream ends with //
- parseClustal handles blocks, conservation lines, residue counts, other
  aligners' headers and labels with spaces; the CLUSTAL header is one
  Biopython reads
- parsePIR takes the line after >P1; as the description and drops the
  terminal *
- splitSeqLabel splits only a trailing /start-end from labels with /
- labels are decoded as UTF-8 so the C and Python readers agree
@satyaorz
satyaorz force-pushed the fix/msa-write-parse-roundtrip branch from c90dc0d to fac8c8c Compare October 10, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant