Skip to content

Upgrade fst to 0.4 and swap memmap for memmap2 - #14

Open
bmesuere wants to merge 1 commit into
phase1/ci-and-test-fixfrom
feature/fst-0.4
Open

Upgrade fst to 0.4 and swap memmap for memmap2#14
bmesuere wants to merge 1 commit into
phase1/ci-and-test-fixfrom
feature/fst-0.4

Conversation

@bmesuere

Copy link
Copy Markdown
Member

Clears RUSTSEC-2020-0077. fst 0.3 pulls in memmap 0.6, which is unmaintained; fst 0.4 no longer does its own memory mapping, so we do it with memmap2 instead.

The change

Map::from_path and Map::from_bytes are gone in 0.4, replaced by a generic Map::new over anything AsRef<[u8]>. The two ways of loading an index therefore produce different types, which the old if/else cannot express. That choice moved into utils::load_fst behind a small enum, instead of being repeated in four commands — which also shrinks each call site to one line.

Index compatibility

This is the part worth checking before merging, and it is asymmetric. I built the same TSV with both versions and cross-read them:

reads a 0.3 index reads a 0.4 index
new binary (fst 0.4) yes yes
old binary (fst 0.3) yes noexpected API version 2, got API version 3

So existing index files keep working, including the tryptic.fst and ninemer.fst published on the data server: those are read, never rewritten, by users. What does not work is going back — an index built after this change cannot be read by an older umgap. One-way migration, no action needed unless someone distributes a freshly built index to users on an older version.

Verification

  • 38 tests pass, cargo +nightly fmt --check clean.
  • pept2lca against a 0.3-built index returns correct taxa on both load paths (mmap and -m).
  • buildindex -> printindex round-trips.
  • Advisories in the resolved tree drop from 6 to 5; memmap is out of Cargo.lock entirely.

The remaining five (ansi_term, atty x2, proc-macro-error, structopt) all come from the clap 2 / structopt stack and go together with that upgrade.

Clears RUSTSEC-2020-0077: fst 0.3 pulled in memmap 0.6, which is
unmaintained. fst 0.4 no longer does its own memory mapping, so we do it
ourselves with memmap2.

Map::from_path and Map::from_bytes are gone in 0.4, replaced by a generic
Map::new over anything that is AsRef<[u8]>. The two ways of loading an index
therefore produce different types, so the choice moved into utils::load_fst
behind a small enum, rather than being repeated in four commands.

Index files stay readable: the new code reads indices written by fst 0.3
(format version 2) as well as its own. The reverse does not hold, an index
built after this change cannot be read by an older umgap, since fst 0.4 writes
format version 3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 18, 2026 13:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants