diff --git a/docs/conf.py b/docs/conf.py index a4278b2..eac4147 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -57,7 +57,20 @@ # clicked, rather than the whole tree opening at once. "collapse_navigation": True, "show_toc_level": 2, - "github_url": "https://github.com/SysBioChalmers/RAVEN", + "icon_links": [ + { + "name": "RAVEN (MATLAB) on GitHub", + "url": "https://github.com/SysBioChalmers/RAVEN", + "icon": "fa-brands fa-github", + "type": "fontawesome", + }, + { + "name": "raven-toolbox (Python) on GitHub", + "url": "https://github.com/SysBioChalmers/raven-toolbox", + "icon": "fa-brands fa-github", + "type": "fontawesome", + }, + ], "use_edit_page_button": False, "navbar_end": ["theme-switcher", "navbar-icon-links"], # The wordmark already carries the "RAVEN" text, so no separate navbar diff --git a/docs/guide/index.md b/docs/guide/index.md index 9e0d501..c81e296 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -126,9 +126,10 @@ not yet at parity between the two toolboxes. ```{toctree} :hidden: -:caption: Installation +:caption: Start ../installation/index +../yaml-format ``` ```{toctree} diff --git a/docs/guide/io.md b/docs/guide/io.md index 1db90a7..07bcadc 100644 --- a/docs/guide/io.md +++ b/docs/guide/io.md @@ -56,7 +56,7 @@ makes those annotations unusable as a key for matching metabolites across models. `importModel` accepts SBML Level 3 Version 1 with FBC version 2 and errors -on anything older; see the [RAVEN 3 migration guide](../raven3-migration.md#sbml-io) +on anything older; see the [RAVEN 3 migration guide](../raven3-migration/formats-and-reconstruction.md#sbml-io) if a file is rejected. ::: :::{tab-item} ๐Ÿ Python @@ -233,7 +233,7 @@ RXNS, METS, COMPS, GENES, MODEL `exportToExcelFormat` writes `.xlsx` and errors on any other extension, so a bare directory path is rejected rather than interpreted. The -[RAVEN 3 migration guide](../raven3-migration.md#excel-io) covers scripts +[RAVEN 3 migration guide](../raven3-migration/formats-and-reconstruction.md#excel-io) covers scripts that pass one. There is no matching importer: `curateModelFromTables` reads a curated spreadsheet back by applying tabular edits to an existing model, not by building one from scratch. diff --git a/docs/index.md b/docs/index.md index 8e30a55..4478b37 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,7 +6,7 @@ **Reconstruction, Analysis and Visualization of Metabolic Networks** A toolkit for building, curating, and simulating genome-scale metabolic -models, available as a MATLAB toolbox and a Python package built on cobrapy. +models, available as a **MATLAB** toolbox and a **Python** package built on cobrapy. ## Install @@ -20,6 +20,7 @@ pip install --pre raven-toolbox :::{tab-item} MATLAB (Add-Ons) Home โ†’ Add-Ons โ†’ Get Add-Ons โ†’ search "RAVEN Toolbox". + Then run `checkRaven` to verify. ::: :::{tab-item} Python (git) @@ -143,7 +144,7 @@ Complete function reference for both MATLAB and Python. Set up RAVEN in MATLAB or raven-toolbox in Python with a solver. ::: -:::{grid-item-card} RAVEN and raven-toolbox +:::{grid-item-card} RAVEN vs. raven-toolbox :link: raven3-vs-raven-toolbox :link-type: doc diff --git a/docs/installation/raven.md b/docs/installation/raven.md index d425ca2..eaad891 100644 --- a/docs/installation/raven.md +++ b/docs/installation/raven.md @@ -73,16 +73,25 @@ setRavenSolver('cobra'); `ravenCobraWrapper` converts a model between the RAVEN struct and the COBRA Toolbox structure; see -[RAVEN 3 and raven-toolbox](../raven3-vs-raven-toolbox.md#solvers). +[RAVEN vs. raven-toolbox](../raven3-vs-raven-toolbox.md#solvers). --- ## Install +:::{warning} Getting 3.0.0b1 specifically +This site documents RAVEN **3.0.0b1**, a pre-release. Both the **Add-Ons +manager** and the **Release download**'s normal releases page only offer the +latest stable release (RAVEN 2.x), not 3.0.0b1: neither method can install +what this site documents. Use the **Clone with git** tab, or the direct +archive link in the **Release download** tab below. +::: + ::::{tab-set} :::{tab-item} {octicon}`plug;1em` Add-Ons manager -Installs from within MATLAB, with no separate download. +Installs from within MATLAB, with no separate download. **Installs the +latest stable release, not 3.0.0b1**; see the warning above. 1. Open the **Home** tab and click **Add-Ons โ†’ Get Add-Ons**. 2. Search for **RAVEN Toolbox** and click **Add โ†’ Add to MATLAB**. @@ -93,20 +102,29 @@ If MATLAB does not pick up the toolbox after step 2, run ::: :::{tab-item} {octicon}`download;1em` Release download -Good for offline or managed environments. +Good for offline or managed environments. **The +[RAVEN releases page](https://github.com/SysBioChalmers/RAVEN/releases) +itself only lists stable releases**; 3.0.0b1 is a git tag with no packaged +release, so download it directly instead: -1. Download the latest ZIP from the - [RAVEN releases page](https://github.com/SysBioChalmers/RAVEN/releases). +1. Download + [the 3.0.0b1 archive](https://github.com/SysBioChalmers/RAVEN/archive/refs/tags/3.0.0b1.zip) + directly (this works without git). 2. Extract it to a location of your choice. 3. In MATLAB, add the RAVEN folder to the path (`pathtool`), then [verify](#verify). ::: :::{tab-item} {octicon}`git-branch;1em` Clone with git -Tracks the development branch, and upgrades with a single command. +Clones the repository, then checks out the `3.0.0b1` tag specifically: a +plain `git clone` alone tracks the `main` branch (stable RAVEN 2), not this +site's pre-release. ```bash git clone --depth=1 https://github.com/SysBioChalmers/RAVEN.git +cd RAVEN +git fetch --depth=1 origin tag 3.0.0b1 +git checkout 3.0.0b1 ``` Add the folder to the MATLAB path and [verify](#verify). @@ -185,11 +203,18 @@ release, and run `checkRaven`. ::: :::{tab-item} {octicon}`git-branch;1em` Clone with git +The `3.0.0b1` tag does not move, so there is nothing to pull while staying on +it. To move past this site's documented snapshot onto the actively-developed +branch: + ```bash -git pull origin main +git checkout develop3 +git pull origin develop3 ``` -Then run `checkRaven`. +`develop3` can be ahead of what this page describes; see +[RAVEN 2 to RAVEN 3](../raven3-migration.md) for what changed and when this +document was last checked against it. Then run `checkRaven`. ::: :::: diff --git a/docs/parameter-tuning/index.md b/docs/parameter-tuning/index.md index 098fce1..768fdd8 100644 --- a/docs/parameter-tuning/index.md +++ b/docs/parameter-tuning/index.md @@ -100,18 +100,11 @@ cross-toolbox to-do list. |---|---| | [Parameter benchmarks](benchmarks.md) | `fseof`, `remove_genes`, `connect_blocked_reactions`, `fill_gaps_*`, `check_tasks`, `find_task_essential_reactions` | -## Model file format - -[YAML model format](../yaml-format.md): the format cobrapy, raven-toolbox and -RAVEN MATLAB all read and write, for moving a model file between any of them -or into version control. - ```{toctree} :hidden: ../tuned-parameters benchmarks -../yaml-format ``` ```{toctree} diff --git a/docs/raven3-migration.md b/docs/raven3-migration.md index 4e359a7..bc1fb28 100644 --- a/docs/raven3-migration.md +++ b/docs/raven3-migration.md @@ -8,8 +8,8 @@ offers to fetch any missing on-demand binaries/data, and its checks catch several of the changes below before your scripts do. Beyond that, the two changes most likely to break silently rather than error are homology/KEGG reconstruction returning a different draft model from the same call -([ยง7](#kegg-reconstruction)), and WoLF PSORT-based localization scores having -no in-toolbox migration path ([ยง3](#backward-incompatible-changes)). +([ยง7](raven3-migration/formats-and-reconstruction.md#kegg-reconstruction)), and WoLF PSORT-based localization scores having +no in-toolbox migration path ([ยง3](raven3-migration/backward-incompatible.md#backward-incompatible-changes)). ::: This guide describes what changed between RAVEN 2 and RAVEN 3. It is written @@ -36,13 +36,13 @@ mostly the same. :::{note} This is the MATLAB-to-MATLAB axis This page is about RAVEN 2 โ†’ RAVEN 3, both MATLAB. How the Python package differs from the MATLAB one is a separate question, answered in -[RAVEN and raven-toolbox](raven3-vs-raven-toolbox.md). +[RAVEN vs. raven-toolbox](raven3-vs-raven-toolbox.md). ::: Start with the [upgrade checklist](#upgrade-checklist) below, then read -[Backward-incompatible changes](#backward-incompatible-changes); it lists -everything that can break. The sections after it go into the detail behind -each item. +[Backward-incompatible changes](raven3-migration/backward-incompatible.md#backward-incompatible-changes); it lists +everything that can break. The detail pages after it go into the detail +behind each item. --- @@ -52,466 +52,47 @@ each item. 1. Update to RAVEN 3.0.0 (or later) and run `checkRaven` (replaces `checkInstallation`); it will offer to fetch any missing on-demand binaries/data (ยง11). -2. Search your scripts for the functions listed in [ยง4](#removed-functionality) - (removed) and [ยง5](#renamed-merged-and-consolidated-functions) +2. Search your scripts for the functions listed in [ยง4](raven3-migration/structure-and-renames.md#removed-functionality) + (removed) and [ยง5](raven3-migration/structure-and-renames.md#renamed-merged-and-consolidated-functions) (renamed/merged) and update or remove those calls. 3. If you use `getModelFromKEGG`/`getKEGGModelForOrganism` or `getModelFromHomology`: expect a different draft reconstruction even from an unmodified call: default cutoffs, the KEGG data source, and the homology - best-hit criterion all changed ([ยง7](#kegg-reconstruction)). Rebuild and + best-hit criterion all changed ([ยง7](raven3-migration/formats-and-reconstruction.md#kegg-reconstruction)). Rebuild and re-validate any KEGG- or homology-derived draft models. 4. If you use `.xlsx` curation templates or `importExcelModel`: convert your curation data to the `.tsv` format `curateModelFromTables` expects; there - is no drop-in Excel importer ([ยง6](#excel-io)). + is no drop-in Excel importer ([ยง6](raven3-migration/formats-and-reconstruction.md#excel-io)). 5. If you use WoLF PSORT-based localization scores: switch to one of the modern predictors (DeepLoc2, MULocDeep, COMPARTMENTS, UniProt); there is no - in-toolbox migration path ([ยง10](#new-functionality)). + in-toolbox migration path ([ยง10](raven3-migration/new-and-environment.md#new-functionality)). 6. If your workflows include metabolic-task gap-filling (`fitTasks`, `checkTasks`, `ftINIT`): re-run and check task pass/fail results; a fixed bug means a previously "passing" task can now correctly fail - ([ยง9](#ftinit)). + ([ยง9](raven3-migration/new-and-environment.md#ftinit)). 7. If your model files are under version control (YAML/SBML): expect diffs on the next export even with no content changes (quoting style, EC-code - annotation, ฮ”G fields; [ยง6](#file-formats-and-model-io)). + annotation, ฮ”G fields; [ยง6](raven3-migration/formats-and-reconstruction.md#file-formats-and-model-io)). 8. Re-run your own test suite / validation pipeline and diff key outputs (model size, growth rate, task results) against your RAVEN 2 baseline before trusting RAVEN 3 results in production. --- -## 1. Repository reorganization +## Detail pages -RAVEN 2's top-level layout (`core/`, `external/`, `hpa/`, `io/`, `legacy/`, -`pathway/`, `plotting/`, `solver/`, `struct_conversion/`, `installation/`, -`utils/`) has been replaced with functional folders that group functions by what -they do rather than by legacy history: - -| RAVEN 3 folder | Contents | Mostly came from (RAVEN 2) | -|---|---|---| -| `INIT/` | `ftINIT`, `INIT/tINIT/` for `getINITModel`/`runINIT`, and HPA/expression parsing + gene scoring (`parseHPA`, `parseHPArna`, `scoreModel`) | `INIT/`, `hpa/` | -| `analysis/` | FBA-adjacent analyses: FVA, sampling, FSEOF, OptKnock, robustness, production envelope, flux comparison/tracing | `core/` | -| `annotation/` | MIRIAM editing, SBO term assignment, ฮ”G CSV I/O | `struct_conversion/` (+ new) | -| `biomass/` | Biomass composition fitting/scaling, GAM | `core/` (+ new) | -| `comparison/` | Multi-model comparison, structured model diffing | `core/` (+ new) | -| `conditions/` | Applying a data-driven growth/media condition to a model | *new folder* | -| `conversion/` | RAVENโ‡„COBRA conversion, identifier prefixing, field ordering | `struct_conversion/` | -| `curation/` | Genome-annotation-driven gene/model curation | *new folder* | -| `gapfilling/` | `fillGaps` and its pluggable algorithms, task fitting, leak-metabolite/exchange checks | `core/` (+ new) | -| `installation/` | Path setup, on-demand binary/data download | `installation/` | -| `io/` | Model import/export (SBML, YAML, Excel), FASTA I/O | `io/` | -| `localization/` | Subcellular localization prediction/scoring, compartment assignment | `external/` (+ new) | -| `manipulation/` | Structural edits: add/remove/change rxns, mets, genes; merge, sort, simplify | `core/` | -| `queries/` | Non-mutating lookups: indexes, exchange/transport reactions, stoichiometry construction, model checking | `core/` | -| `reconstruction/` | KEGG- and homology-based draft reconstruction | `external/` | -| `solver/` | LP/MILP solving | `solver/` | -| `tasks/` | Metabolic task list parsing/checking | `core/` | -| `testing/` | `matlab.unittest` suite (`function_tests/`) | `testing/` | -| `utils/` | Generic helpers, including the new argument-parsing/registry infrastructure | `utils/` | -| `deprecated/` | Thin backward-compatible wrappers for merged/renamed functions | *new folder* | - -Entirely removed (no successor folder): `external/metacyc/` (MetaCyc -reconstruction), `pathway/` and `plotting/` (visualization), `legacy/` -(CellDesigner import and other pre-2.0 code), `struct_conversion/`'s Excel-POI -helpers. - -**The reorganization affects fewer scripts than the change list suggests.** -If your scripts call RAVEN -functions by name (the normal usage pattern, with the whole RAVEN folder tree, -including subfolders, added to the MATLAB path via `addRavenToUserPath` or -`addpath(genpath(...))`), moving a file to a new subfolder does not break the -call: MATLAB resolves functions by name across the whole path, not by a -hard-coded relative location. The reorganization only matters if you (a) -hard-coded a path to a specific `.m` file, (b) called one of the functions that -was actually removed or merged (see below), or (c) shadowed a RAVEN function -name with your own; folder names are irrelevant otherwise. - -The full moved-file mapping (RAVEN 2 path โ†’ RAVEN 3 path) for every relocated -function is available on request / in the repository's git history -(`git diff --stat -M develop...develop3`); it is not reproduced in full here -because it is large and, as the point above explains, rarely something you -need to act on. - ---- - -## 2. Calling convention: positional or named arguments - -RAVEN 2 functions took optional arguments strictly positionally -(`f(model, a, b, c)`), which made it easy to lose track of argument order and -impossible to skip an early optional argument to set a later one without also -specifying all of the ones in between. - -RAVEN 3 introduces `utils/parseRAVENargs.m`, used by 112 functions across the -toolbox. Required leading arguments stay as explicit, positional parameters; -everything optional is collected into `varargin` and can be supplied three ways, -interchangeably: - -```matlab -removeReactions(model, rxnList, true, true, true) % positional (RAVEN 2 style, still works) -removeReactions(model, rxnList, "removeUnusedGenes", true) % named -removeReactions(model, rxnList, true, "removeUnusedComps", true) % hybrid -``` - -The rule: parsing scans for the first argument that is a string matching a known -parameter name. Everything before that point is assigned positionally, in the -order the function's `parseRAVENargs` specification declares; everything from -that point on must be valid name-value pairs. - -**This means old positional calls keep working, if the function's new -parameter order still matches the old positional order.** In every function -spot-checked for this guide (`importModel`, `exportModel`, -`exportToExcelFormat`, `removeReactions`, `ftINIT`, `fillGaps`, `fitTasks`, -`getModelFromHomology`, `addIdentifierPrefix`/`removeIdentifierPrefix`, and -`exportForGit`), new optional parameters were appended at the end of the list, -so old positional calls keep working unchanged. - -One caveat from the parser's own documentation: if a positional argument you're -passing is itself a string that happens to match one of the function's parameter -names, use the explicit name-value form for that argument to avoid ambiguity. - ---- - -(backward-incompatible-changes)= -## 3. Backward-incompatible changes - -This is the consolidated list of everything a RAVEN 2 script could plausibly -break on, or silently produce different output from, when run against RAVEN 3. -Items are grouped by how they manifest. - -### Will error / undefined function - -| RAVEN 2 usage | What happens in RAVEN 3 | +| | | |---|---| -| `importExcelModel(...)` | Removed. Use `curation/curateModelFromTables.m` (different input format: `.tsv`, not `.xlsx`; see [ยง6](#excel-io)) or convert your template to YAML/SBML and use `importModel`. | -| `importModel(...)` on SBML Level < 3, Level 3 without FBC v2, or any non-FBC SBML | Errors. Only SBML **Level 3 Version 1, FBC package version 2** is accepted (see [ยง6](#sbml-io)). | -| `exportToExcelFormat(model, fileName)` where `fileName` is a bare path (no `.xlsx`) | No longer falls back to tab-delimited text export; errors with "only export to xlsx format is supported". | -| `exportToTabDelimited(...)` | Removed outright (was a dead-code path anyway). | -| `getWoLFScores(...)` / `parseScores(..., 'wolf', ...)` | WoLF PSORT support is fully removed. `parseScores` now errors on `'predictor','wolf'` with a message pointing to `'deeploc'`, `'mulocdeep'`, `'compartments'`, or `'uniprot'`. **No in-toolbox migration path for existing WoLF PSORT output files.** | -| `qMOMA(...)`, `solveQP(...)` | Removed (RAVEN's last `quadprog`/Optimization Toolbox dependency). `findGeneDeletions`'s `analysisType`/`refModel`/`oeFactor` arguments (MOMA-based over-expression analysis) were removed with them; only FBA-based single/double gene deletion (`sgd`/`dgd`) remains. | -| `checkRxn(...)` | Removed, no direct successor. For debugging why a reaction can't carry flux, use `gapfilling/findLeakMetabolite.m` / `gapfilling/canExchange.m`, or `gapfilling/checkProduction.m`. | -| `dispEM(...)`, `followFluxes(...)`, `followChanged(...)`, `printModel(...)`, `mapCompartments(...)`, `getExpressionStructure(...)`, `getMetsInComp(...)`, `parallelPoolRAVEN(...)` | All removed (see [ยง4](#removed-functionality) for what replaces each, where anything does). | -| Anything under `external/metacyc/`, `pathway/`, `plotting/`, `legacy/` | Entire subsystems removed: MetaCyc-based reconstruction and all visualization/plotting functionality (`drawMap`, `drawPathway`, `markPathwayWithFluxes`, `setOmicDataToRxns`, etc.). No in-toolbox replacement; use COBRA Toolbox or Escher for visualization. | -| `getGenesFromKEGG`, `getMetsFromKEGG`, `getRxnsFromKEGG`, `constructMultiFasta`, `getWSLpath`, `getBlastFromExcel` | Removed along with the old local-KEGG-FTP-dump workflow (see [ยง7](#kegg-reconstruction)). | -| `getToolboxVersion(...)` | Removed as a standalone function; folded into an internal helper of `exportForGit`. | - -### Will silently produce different results - -Same call, same input model, different output. Grouped by function; see the -linked section for the full explanation of each. - -| Function | New behavior (default, unless noted) | -|---|---| -| `getModelFromHomology`, `strictness=3` | Best-hit tie-break now uses **bitscore** instead of E-value (`"scoreBy","evalue"` restores the old criterion); [ยง7](#kegg-reconstruction) | -| `getModelFromHomology`, `minLen` | Default lowered from `200` to `100`; [ยง7](#kegg-reconstruction) | -| `getModelFromHomology`, GPR merging | Gene substitution is exact-token-based instead of regex/`strrep`-based; [ยง7](#kegg-reconstruction) | -| `getKEGGModelForOrganism`, `cutOff` | Default lowered from `10^-50` to `10^-30`; [ยง7](#kegg-reconstruction) | -| `getKEGGModelForOrganism`, `minScoreRatioG` | Default raised from `0.8` to `0.9`; [ยง7](#kegg-reconstruction) | -| `removeGenes`, `expandModel`, `findPotentialErrors`, `addRxns` | grRules are now parsed correctly (exact gene-token matching, structure-aware isozyme/complex handling) instead of via substring/regex matching; [ยง8](#gpr-parsing) | -| `changeGrRules(..., 'replace', false)` onto a gene-less reaction | Produces a valid grRule instead of an unparseable one; [ยง8](#gpr-parsing) | -| `simplifyModel` (used by task-based gap-filling) | No longer drops a task's boundary constraint on a metabolite with no reactions: a task that used to silently pass may now correctly fail; [ยง9](#ftinit) | -| `ftINITInternalAlg` | `allowExcretion` constraint sign corrected | -| `ravenCobraWrapper` | Gene-ID-to-MIRIAM matching no longer breaks on gene IDs containing regex metacharacters; [ยง6](#raven-cobra-interoperability) | -| `permuteModel`, `randomSampling`, `optimizeProb`, `getFluxZ`, `mergeCompartments`, `convertToIrrev`, `replaceMets`, `setExchangeBounds`, `fitParameters`, `guessComposition` | Each had a pre-existing bug fixed; output differs only for models/inputs that triggered it | - -None of these are API changes; the same call with the same arguments now -returns a different, corrected result. - -### Will warn, but keep working - -- **Error/warning identifiers changed.** `dispEM` (RAVEN 2's universal - error/warning function) always raised errors with an **empty** identifier - (`MException('', ...)`), so no RAVEN 2 script could have legitimately caught - on `err.identifier`. RAVEN 3 replaces `dispEM` with native `error()`/ - `warning()` calls, most (not all) carrying real identifiers, - predominantly `RAVEN:badInput`, plus a handful of specific ones - (`RAVEN:modelError`, `RAVEN:badGrRule`, `RAVEN:sampling`, `RAVEN:infeasible`, - `RAVEN:grRuleTooComplex`). Message text is largely unchanged (many messages - are byte-identical to RAVEN 2's), so `try/catch` blocks that pattern-match on - message text keep working; scripts can newly (optionally) match on - `err.identifier` instead. -- **`getINITModel`/`runINIT` now emit a one-time "legacy" notice** pointing at - `ftINIT`, silenceable with `warning('off','RAVEN:legacyMethod')`. This is - explicitly *not* a deprecation; RAVEN's own documentation states these - functions "remain supported" indefinitely, to reproduce previously published - Human-GEM-derived models. See [ยง9](#ftinit). -- **`makeSomething`, `consumeSomething`, `canProduce`, `canConsume`** now live in - a new `deprecated/` folder as thin wrappers around `findLeakMetabolite`/ - `canExchange`, and emit a one-time `RAVEN:deprecated` warning stating they - "will be removed in the next major release." Unlike the legacy-notice - functions above, these are on an actual removal track; migrate call sites - before the next major release. See [ยง9](#ftinit). -- **`checkInstallation` is now `checkRaven`**, moved from `installation/` to the - repository root, with an unchanged signature - (`[currVer, installType] = checkRaven(developMode, checkBinaries)`). Existing - calls do **not** break: `checkInstallation` survives as a forwarding wrapper - that passes every argument and output straight through, after emitting a - `RAVEN:deprecated` warning naming `checkRaven`. It is on the same removal - track as the `deprecated/` functions above, so switch at the next opportunity. - -### Environment / installation changes - -KEGG HMM libraries and external command-line binaries (BLAST+, DIAMOND, HMMER, -cd-hit, MAFFT, WoLF PSORT) are no longer committed to the repository. RAVEN 3 -fetches them on demand from the `raven-data` GitHub release the first time a -function needs one, which means **internet access is required on first use**, -not just at clone/install time. See [ยง11](#toolbox-deps) for what's needed for -an air-gapped install, and the full dependency comparison. - ---- - -(removed-functionality)= -## 4. Removed functionality - -Beyond the items already listed as backward-incompatible above, RAVEN 3 removed -a number of functions and whole subsystems as part of a deliberate effort to -make the codebase smaller. None of these have call sites left in RAVEN 3 itself. - -| Removed | What it did | Replacement / notes | -|---|---|---| -| `pathway/*`, `plotting/*` (16 functions: `drawMap`, `drawPathway`, `colorPathway`, `colorSubsystem`, `markPathwayWithFluxes`, `markPathwayWithExpression`, `setOmicDataToRxns`, `plotAdditionalInfo`, etc.) | Metabolic map/pathway visualization | None in RAVEN core. Use COBRA Toolbox, Escher, or export flux/omics data for external plotting. | -| `external/metacyc/*` (7 functions: `getModelFromMetaCyc`, `getRxnsFromMetaCyc`, `getMetsFromMetaCyc`, `getEnzymesFromMetaCyc`, `addSpontaneousRxns`, `linkMetaCycKEGGRxns`, `combineMetaCycKEGGModels`) | MetaCyc-based draft reconstruction | None; KEGG- and homology-based reconstruction remain the supported paths. | -| `legacy/*` (CellDesigner import, `changeGeneAssoc`, the bundled `xml_toolbox`) | Pre-2.0 compatibility code | None; considered dead. | -| `io/importExcelModel.m`, `io/loadWorkbook.m`, `io/loadSheet.m`, `io/writeSheet.m`, `io/SBMLFromExcel.m`, `io/addJavaPaths.m`, `io/startup.m` | Apache-POI-based Excel model I/O | `io/writeExcel.m` (dependency-free OOXML writer) for export; `curation/curateModelFromTables.m` for tabular curation. No full Excel-template *import* path remains; see [ยง6](#excel-io). | -| `io/exportModelToSIF.m`, `io/exportToTabDelimited.m` | SIF graph export; tab-delimited model export | Dead/orphaned code with no callers; removed outright. | -| `io/getFullPath.m`, `io/getMD5Hash.m`, `io/getToolboxVersion.m` | Path canonicalization; file MD5 hashing; toolbox git-version lookup | Inlined where still needed (e.g. MD5 via Java's `MessageDigest` directly inside `getBlast`), or folded into `exportForGit`. | -| `core/dispEM.m` | Universal error/warning display | Native `warning()`/`error()` + new `utils/ravenList.m` for formatted item lists. See [ยง3](#backward-incompatible-changes). | -| `core/followFluxes.m`, `core/followChanged.m` | Flux-change reporting relative to a reference solution | `queries/printFluxes.m` covers the cutoff-filtering half; nothing replaces the reference-flux comparison directly. | -| `core/printModel.m` | Printed reactions to screen/file | `queries/printModelStats.m` / `queries/printFluxes.m` cover related summary/flux printing; no direct one-line-per-reaction dump remains. | -| `core/getMetsInComp.m` | Returned metabolite indices in a given compartment | Inline it as `model.metComps == compIndex`. | -| `core/mapCompartments.m` | Remapped compartment labels in a localization-score structure | Superseded by the new predictor-based localization workflow and `localization/defaultCompartmentMap.m`. | -| `core/getExpressionStructure.m` | Loaded an expression-experiment structure from an ad hoc Excel format | Superseded by `INIT/parseHPA.m`/`INIT/parseHPArna.m` for the supported expression-data workflow. | -| `core/checkRxn.m` | Per-reaction reactant/product synthesizability debugging | No direct successor; use `gapfilling/findLeakMetabolite.m`, `gapfilling/canExchange.m`, or `gapfilling/checkProduction.m`. | -| `core/parallelPoolRAVEN.m` | Parallel Computing Toolbox pool setup | Renamed/rewritten as `utils/parallelWorkersRAVEN.m`. | -| `external/getWoLFScores.m` | WoLF PSORT score parsing | None; see [ยง3](#backward-incompatible-changes). | -| `external/kegg/{getGenesFromKEGG,getMetsFromKEGG,getRxnsFromKEGG,constructMultiFasta,getWSLpath}.m`, `external/getBlastFromExcel.m` | Local-KEGG-FTP-dump parsing; per-KO multi-FASTA assembly; WSL path translation; Excel-based BLAST import | Superseded by the raven-data-artifact-based KEGG pipeline. See [ยง7](#kegg-reconstruction). | -| `solver/qMOMA.m`, `solver/solveQP.m` | MOMA-based analyses (quadratic programming) | Removed; no in-toolbox QP-based replacement. See [ยง3](#backward-incompatible-changes) for `findGeneDeletions` impact. | -| `testing/unit_tests/*.m` (script-based tests) | Ad hoc test scripts | `testing/function_tests/*.m`, a `matlab.unittest`-class-based suite. Contributor-facing only; `runRAVENtests` usage is unchanged for end users. | - ---- - -(renamed-merged-and-consolidated-functions)= -## 5. Renamed, merged, and consolidated functions - -RAVEN 3 introduces a formal **`deprecated/` folder convention**: functions that -were merged into a more general replacement stay callable under their old name, -as a thin wrapper that emits a one-time `RAVEN:deprecated` warning ("will be -removed in the next major release") and forwards to the new function. - -| Old function(s) | New function | Notes | -|---|---|---| -| `makeSomething(model, ignoreMets, isNames, minNrFluxes, allowExcretion, params, ignoreIntBounds)` | `findLeakMetabolite(model, 'produce', ...)` | Argument order maps 1:1; the wrapper forwards `varargin` unchanged. | -| `consumeSomething(model, ignoreMets, isNames, minNrFluxes, params, ignoreIntBounds)` | `findLeakMetabolite(model, 'consume', ...)` | **Not** a verbatim forward: `consumeSomething`'s old positional order has no `allowExcretion` slot, so the wrapper explicitly remaps arguments by name to avoid corrupting `params`/`ignoreIntBounds`. | -| `canProduce(model, mets)` | `canExchange(model, 'produce', 'mets', mets)` | | -| `canConsume(model, mets)` | `canExchange(model, 'consume', 'mets', mets)` | | -| `compareRxnsGenesMetsComps` | folded into `comparison/compareMultipleModels.m` | Its rxn/gene/met/comp overlap output is now one part of `compareMultipleModels`'s output; no separate function remains. | -| `INIT/scoreComplexModel.m` | `INIT/scoreModel.m` | **Merged, not renamed.** RAVEN 2 had *both* `INIT/scoreComplexModel.m` and `hpa/scoreModel.m`; RAVEN 3 folded them into one `scoreModel`, called with different settings from tINIT than from ftINIT. A RAVEN 2 call to `scoreModel` is unaffected; a call to `scoreComplexModel` has to move to `scoreModel` *and* pass the settings that reproduce the old behaviour (a single operator for both `and`/`or`, `dataPrecedence` `'reaction'`). | -| `INIT/ftINITFillGapsMILP.m` | folded into `solver/getMinNrFluxes.m` | Exposed now via `getMinNrFluxes`'s `resolveTies`/`proveAbsGap` options. | -| `INIT/ftINITFillGapsForAllTasks.m` | folded into `gapfilling/ftINITFillGaps.m` and `fitTasks`'s `'preMerged'` gap-fill mode | | -| `getGenesFromKEGG`/`getMetsFromKEGG`/`getRxnsFromKEGG` | `reconstruction/kegg/readKEGGTable.m` + the raven-data KEGG artifacts | Not a direct signature-compatible rename: the whole data source changed; see [ยง7](#kegg-reconstruction). | -| `checkInstallation` | `checkRaven` | Moved from `installation/checkInstallation.m` to repo-root `checkRaven.m`; identical calling signature. `checkInstallation` remains as a deprecated forwarding wrapper, so old calls keep working with a `RAVEN:deprecated` warning. | - -Separately, `getINITModel` and `runINIT` (now under `INIT/tINIT/`) are marked -**legacy but indefinitely supported** (not on the `deprecated/` removal track) -via `utils/legacyMethodNotice.m`, which fires a one-time, silenceable -`RAVEN:legacyMethod` warning pointing at `ftINIT`. Use `ftINIT` for new work; -keep `getINITModel`/`runINIT` only to reproduce models originally built with -them. - ---- - -(file-formats-and-model-io)= -## 6. File formats and model I/O - -(sbml-io)= -### SBML I/O - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| `exportModel` output format | SBML Level 3 Version 1, FBC v2 | unchanged | -| `importModel` accepted input | any SBML level/version; L2-style notes-based gene associations (`GENE_ASSOCIATION`/`PROTEIN_ASSOCIATION`) also read; FBC fields optional | only SBML **Level 3 Version 1, FBC v2**; anything else errors | -| MIRIAM prefixes read on import | `urn:miriam:`, `http://identifiers.org/`, `https://identifiers.org/` | unchanged | - -Older SBML files must be converted first (e.g. re-exported from COBRApy/COBRA -Toolbox) before `importModel` will accept them. - -(excel-io)= -### Excel I/O - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| Excel engine | Apache POI (Java), bundled | dependency-free OOXML writer (`io/writeExcel.m`) | -| Full-model `.xlsx` import | `importExcelModel.m` (`RXNS`/`METS`/`COMPS`/`GENES`/`MODEL` sheets โ†’ complete model) | removed, no equivalent importer | -| Nearest replacement for curation | no equivalent | `curation/curateModelFromTables.m`: curates/extends an *existing* model from separate `.tsv` files (mets by name+compartment, reactions by stoichiometry, genes by name), not a from-scratch importer | -| `exportToExcelFormat` on a bare path (no `.xlsx`) | fell back to tab-delimited text export | errors, `.xlsx` only | - -If your pipeline depends on the old Excel curation template, either convert it -to the `.tsv` shape `curateModelFromTables` expects, or build the model via -`importModel`/YAML and use `curateModelFromTables` for incremental edits. - -### YAML I/O - -`readYAMLmodel`/`writeYAMLmodel` are pure-MATLAB in both branches; model YAML -I/O has no Python dependency in RAVEN 2 or RAVEN 3. (`io/parseYAML.m` is an -unrelated, new, pure-MATLAB generic YAML parser used only by -`conditions/applyCondition.m` to read condition files; it doesn't affect -model I/O.) Output format changes, visible as diffs in version-controlled -model files: - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| Quote style | single quotes | double quotes | -| Reaction EC codes | top-level `eccodes` field only | also written under `annotation: ec-code` | -| Gene-less model | `genes:` section omitted | `genes: []` written explicitly | -| Empty `gene_reaction_rule` | written as an empty string | omitted | -| `objective_coefficient` on import | ignored | honored | - -(raven-cobra-interoperability)= -### RAVENโ‡„COBRA interoperability - -`conversion/ravenCobraWrapper.m` keeps its signature -(`newModel = ravenCobraWrapper(model)`) unchanged. Two correctness fixes change -output for affected models: a stale-variable bug that could reuse the wrong -MIRIAM data during gene-annotation conversion, and gene IDs containing regex -metacharacters (e.g. `ENSG00000141510.16`) that could be mis-matched during -annotation conversion; both fixed. - ---- - -(kegg-reconstruction)= -## 7. Reconstruction: KEGG and homology pipelines - -### KEGG-based reconstruction (`getModelFromKEGG`, `getKEGGModelForOrganism`) - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| Reaction/gene database source | local KEGG FTP dump (license required), parsed by `getGenesFromKEGG`/`getMetsFromKEGG`/`getRxnsFromKEGG` into a cached `keggModel.mat` | prebuilt artifacts (a gene-free reference model plus KO/reaction/organism tables) fetched on demand from the `raven-data` GitHub release, assembled by `buildGlobalKEGGModel`/`buildGlobalGPR`/`readKEGGTable` | -| `getModelFromKEGG`'s `keggPath` argument | required | removed, not needed | -| Ortholog HMM library | one `.hmm` file per KO (self-built via CD-HIT + MAFFT + `hmmbuild`, or downloaded); queried with one `hmmsearch` call per KO, via WSL on Windows | one concatenated per-domain library (`kegg118_eukaryotes.hmm` / `kegg118_prokaryotes.hmm`); one `hmmsearch` call against the whole proteome, no WSL | -| `dataDir` | arbitrary local folder | must be `kegg118_eukaryotes` or `kegg118_prokaryotes` | -| `nSequences`, `seqIdentity` (CD-HIT clustering controls) | configurable | removed, no local clustering step | -| `cutOff` default | `10^-50` | `10^-30` | -| `minScoreRatioG` default | `0.8` | `0.9` | -| Calling convention | positional | positional-or-named (`parseRAVENargs`); old positional order preserved | - -### Homology-based reconstruction (`getModelFromHomology`) - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| `strictness` parameter | `1`/`2`/`3` | unchanged | -| Best-hit tie-break at `strictness=3` | E-value | bitscore by default (`"scoreBy","evalue"` restores the old criterion) | -| `minLen` default | `200` | `100` | -| GPR gene substitution when merging homology hits | regex/`strrep` substitution (could corrupt IDs that were substrings of one another, or contained regex metacharacters) | exact-token-based substitution | - -### Shared infrastructure - -External binaries (BLAST+, DIAMOND, HMMER) and the KEGG data artifacts -themselves are fetched on demand rather than bundled with the repository; see -[ยง11](#toolbox-deps). - -**New:** `curation/downloadGenomeData.m`, `curation/getGeneData.m`, and -`curation/processProteinFastaFile.m` give a new on-ramp from a bare NCBI genome -accession (`GCF_`/`GCA_`) straight to a model-ready proteome and gene ID -mapping table. - ---- - -(gpr-parsing)= -## 8. Gene-reaction-rule (GPR) parsing - -RAVEN 3 adds a proper grRule tokenizer/parser: `utils/parseGrRule.m` builds a -parse tree (`type`: `'gene'|'and'|'or'`, with `children`); `utils/grRuleToDNF.m` -converts it to disjunctive normal form (RAVEN's canonical "OR of AND-clauses" -shape for enzyme-complex/isozyme structure); `utils/isDnfGrRule.m` checks -whether a rule is already in that form; `utils/grRuleToString.m` renders a tree -back to a canonical string. These are used internally by four functions, -they are **not** applied automatically to every loaded model; a grRule keeps -its original text unless one of these functions is called on it. - -| Function | RAVEN 2 parsing | RAVEN 3 parsing | -|---|---|---| -| `removeGenes` | substring match on gene ID (`"10"` also matched `"100"`) | exact gene-token match | -| `expandModel` | naive split on `' or '` (could drop an `AND`-ed subunit inside a nested rule) | parse-tree-based isozyme expansion, preserves nested `AND`/`OR` | -| `findPotentialErrors` | string-pattern check for `") and ("` etc. (false positives on valid rules) | parse-tree-based non-DNF detection | -| `addRxns` (new gene IDs from a grRule) | split on the substrings "and"/"or" anywhere (could shred an ID like `"band1"`) | split only on space-bounded `' and '`/`' or '` tokens | -| `comparison/diffModels.m` | (function didn't exist) | grRules compared as logical DNF equality, not string equality | - -The accepted grRule *syntax* itself is unchanged: `' and '`/`' or '` -(case-insensitive) still work exactly as before, and -`manipulation/standardizeGrRules.m` (which runs on import) uses the same -string-splitting logic it always did. - ---- - -(ftinit)= -## 9. Gap-filling and context-specific model extraction - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| `fillGaps` algorithm | one built-in MILP connectivity-maximization strategy | same strategy is still the default; `algorithm` option adds `'fastLP'`/`'swiftLP'` (FASTCORE/SWIFTCORE-style LP relaxation), `'gapfillMILP'` (growth-floor MILP with directionality repair), `'topological'` (BFS producibility pre-screen, no model modification) | -| `fillGaps` positional arguments | `allowNetProduction, useModelConstraints, supressWarnings, rxnScores` | unchanged; new options are name-value-only | -| `ftINIT` tie-breaking | depends on solver seed | optional `resolveTies` pins degenerate MILP optima to a deterministic answer | -| `ftINIT` MILP optimality proof | escalating relative-gap search only | optional `proveAbsGap` proves each stage to a fixed absolute gap in one solve | -| `ftINIT` positional arguments | preserved | unchanged; `resolveTies`/`proveAbsGap` are name-value-only, default off | -| `getINITModel`/`runINIT` | supported, no runtime notice | supported, now under `INIT/tINIT/`; one-time "legacy, use `ftINIT`" notice (see [ยง5](#renamed-merged-and-consolidated-functions)) | -| `fitTasks` gap-fill mode | one behavior (equivalent to today's `'merge'`) | `gapFillMode`: `'merge'` (default, same as before) or `'preMerged'` (faster, ftINIT-style) | -| `fitTasks` outputs | `[outModel, addedRxns]` | `[outModel, addedRxns, failedTasks]` | -| Task requiring net production of a metabolite with no reactions | boundary constraint silently dropped before gap-filling; task could incorrectly report as passing | constraint preserved; task correctly reports failing if the reference model can't produce it | - ---- - -(new-functionality)= -## 10. New functionality - -All additive; none of this replaces or changes behavior of existing RAVEN 2 -functions, except where noted. - -| Area | New function(s) | What it adds | -|---|---|---| -| Flux sampling (`analysis/`) | `sampleACHR`, `sampleCHRR`, `sampleMaxVolEllipse`, `sampleChebyshevCenter`, `sampleWarmupPoints` | `randomSampling` becomes a dispatcher (`'method'` option) across RAVEN 2's original random-objective method (now `'randomObjective'`) plus two new near-uniform MCMC samplers: ACHR (hit-and-run) and CHRR (hit-and-run with rounding, better mixing on ill-conditioned polytopes) | -| Flux analysis / QC | `compareFluxes`, `getMinimalMedium`, `traceFluxPath`, `walkFluxes`, `modelSummary` | structured flux-vector diffing; MILP-based minimal-medium search; highest-flux-fraction path tracing between two reactions; interactive flux-neighborhood navigation; model/flux summary printing | -| Model comparison (`comparison/`) | `diffModels` | structured, ID-keyed diff between two models: stoichiometry, bounds, objective, grRule (as logical DNF equality), EC codes, metabolite properties; distinct from the existing overlap-style `compareMultipleModels` | -| Manipulation | `findDuplicateRxns`, `findPotentialErrors` | standalone duplicate-stoichiometry query; parse-tree-based non-DNF grRule detection (see [ยง8](#gpr-parsing)) | -| Genome-based curation (`curation/`, new folder) | `downloadGenomeData`, `getGeneData`, `processProteinFastaFile`, `renameModelGenes`, `curateModelFromTables` | pipeline from an NCBI genome accession to a gene ID mapping table to bulk gene renaming in a model; `curateModelFromTables` bulk-curates mets/rxns/genes from `.tsv` files against an existing model | -| Biomass composition (`biomass/`) | `getBiomassFractions`, `scaleBiomassFraction`, `scaleBiomassPseudoreaction`, `setGAM` | organism-agnostic biomass-composition and growth-associated-maintenance (GAM) calibration; `tutorial/tutorial7.m` demonstrates the workflow | -| Localization (`localization/`) | `assignCompartments`, `defaultCompartmentMap`, `getUniProtScores` | deterministic MILP compartment assignment (alternative to the heuristic `predictLocalization`); `parseScores` gains `'deeploc'`/`'cello'`/`'mulocdeep'`/`'compartments'`/`'uniprot'` predictors (replacing WoLF PSORT; see [ยง3](#backward-incompatible-changes)) | -| Annotation (`annotation/`) | `assignSBOterms`, `loadDeltaGCSV`/`saveDeltaGCSV` | organism-agnostic SBO term assignment; ฮ”G thermodynamic data CSV round-trip | -| Conditions (`conditions/`, new folder) | `applyCondition` | applies a data-driven growth/media condition (a YAML file, not code) to a model in a fixed sequence: exchange bounds, optional cofactor-pseudoreaction edits, optional biomass-stoichiometry deltas, per-reaction bound overrides | -| Internal infrastructure | `ravenModelFields` | central registry of every 1-D model field (entity type, default value), backing `permuteModel`/`removeReactions` internally | - ---- - -(toolbox-deps)= -## 11. Toolbox dependencies and installation - -| | RAVEN 2 | RAVEN 3 | -|---|---|---| -| MATLAB version | no pinned/documented minimum (guidance: avoid very new MATLAB syntax, to accommodate older installs) | CI is tested against **R2024b** | -| Statistics Toolbox | required (`combnk`) | not required (`nchoosek`, base MATLAB) | -| Bioinformatics Toolbox | required (`fastaread`/`fastawrite`) | not required (`io/readFasta.m`/`io/writeFasta.m`) | -| Optimization Toolbox | required for `qMOMA`/`solveQP` | not required (both removed) | -| Excel I/O | bundled Apache POI (Java) | dependency-free OOXML writer | -| LP/MILP solver | required (bundled `glpk`, or Gurobi/SCIP/COBRA) | unchanged, still required | -| External binaries (BLAST+, DIAMOND, HMMER, cd-hit, MAFFT) | committed to the repository, available offline immediately after clone | fetched on demand from the `raven-data` GitHub release on first use (per-platform ZIPs); an offline `*-full`/`*-binaries` release bundle is available for air-gapped installs | -| KEGG HMM libraries / reference data | manual download or local build required | fetched on demand from `raven-data`, cached locally after first use | -| Installation self-check | `installation/checkInstallation.m`; ran the full test suite to populate its pass/fail table | renamed `checkRaven`, moved to the repo root; does fast, targeted checks (a small import/export round-trip, a trivial LP per solver) instead of invoking the full test classes, and proactively offers to download missing binaries | - -Net effect: a bare RAVEN 3 clone is far smaller and installs with fewer MATLAB -toolbox prerequisites. The trade-off: it needs network access the first time -KEGG- or homology-based reconstruction functions actually run. - ---- - -## 12. Error and warning handling - -RAVEN 2's `dispEM` always raised errors with an empty identifier, so no RAVEN 2 -`try/catch` could have matched on `err.identifier`; RAVEN 3's move to native -`error()`/`warning()` with real (mostly `RAVEN:*`) identifiers is additive from -that angle, not breaking. Message text is preserved for most existing checks, -so text-matching `try/catch` blocks are unaffected either way. See -[ยง3](#backward-incompatible-changes) for the details and the new -`utils/ravenList.m` message-formatting helper. - -One related, purely additive change: `queries/checkModelStruct.m`, called *with* -an output argument (`issues = checkModelStruct(model)`), now returns a -structured array of findings (`category`/`target`/`message`) instead of only -throwing or printing, useful for programmatic model QC. The no-output-argument -calling convention (throws/warns directly) is unchanged. +| [ยง1โ€“5. What moved, what's gone, and what's renamed](raven3-migration/structure-and-renames.md) | The folder reorganization, every removed function and subsystem, and every renamed or merged function. | +| [ยง2โ€“3. Backward-incompatible changes](raven3-migration/backward-incompatible.md) | The positional-or-named calling convention, and the full punch list: what will error, what will silently differ, what will warn, and what changed about installation. | +| [ยง6โ€“8. File formats, reconstruction, and GPR parsing](raven3-migration/formats-and-reconstruction.md) | SBML, Excel and YAML I/O; the KEGG and homology reconstruction pipelines; the new grRule parser. | +| [ยง9โ€“12. New functionality, dependencies, and error handling](raven3-migration/new-and-environment.md) | Gap-filling and ftINIT changes, everything purely additive, the toolbox dependency comparison, and error/warning handling. | + +```{toctree} +:hidden: + +raven3-migration/structure-and-renames +raven3-migration/backward-incompatible +raven3-migration/formats-and-reconstruction +raven3-migration/new-and-environment +``` diff --git a/docs/raven3-migration/backward-incompatible.md b/docs/raven3-migration/backward-incompatible.md new file mode 100644 index 0000000..b55bda8 --- /dev/null +++ b/docs/raven3-migration/backward-incompatible.md @@ -0,0 +1,124 @@ +# Backward-incompatible changes + +## 1. Calling convention: positional or named arguments + +RAVEN 2 functions took optional arguments strictly positionally +(`f(model, a, b, c)`), which made it easy to lose track of argument order and +impossible to skip an early optional argument to set a later one without also +specifying all of the ones in between. + +RAVEN 3 introduces `utils/parseRAVENargs.m`, used by 112 functions across the +toolbox. Required leading arguments stay as explicit, positional parameters; +everything optional is collected into `varargin` and can be supplied three ways, +interchangeably: + +```matlab +removeReactions(model, rxnList, true, true, true) % positional (RAVEN 2 style, still works) +removeReactions(model, rxnList, "removeUnusedGenes", true) % named +removeReactions(model, rxnList, true, "removeUnusedComps", true) % hybrid +``` + +The rule: parsing scans for the first argument that is a string matching a known +parameter name. Everything before that point is assigned positionally, in the +order the function's `parseRAVENargs` specification declares; everything from +that point on must be valid name-value pairs. + +**This means old positional calls keep working, if the function's new +parameter order still matches the old positional order.** In every function +spot-checked for this guide (`importModel`, `exportModel`, +`exportToExcelFormat`, `removeReactions`, `ftINIT`, `fillGaps`, `fitTasks`, +`getModelFromHomology`, `addIdentifierPrefix`/`removeIdentifierPrefix`, and +`exportForGit`), new optional parameters were appended at the end of the list, +so old positional calls keep working unchanged. + +One caveat from the parser's own documentation: if a positional argument you're +passing is itself a string that happens to match one of the function's parameter +names, use the explicit name-value form for that argument to avoid ambiguity. + +--- + +(backward-incompatible-changes)= +## 2. Backward-incompatible changes + +This is the consolidated list of everything a RAVEN 2 script could plausibly +break on, or silently produce different output from, when run against RAVEN 3. +Items are grouped by how they manifest. + +### Will error / undefined function + +| RAVEN 2 usage | What happens in RAVEN 3 | +|---|---| +| `importExcelModel(...)` | Removed. Use `curation/curateModelFromTables.m` (different input format: `.tsv`, not `.xlsx`; see [ยง6](../raven3-migration/formats-and-reconstruction.md#excel-io)) or convert your template to YAML/SBML and use `importModel`. | +| `importModel(...)` on SBML Level < 3, Level 3 without FBC v2, or any non-FBC SBML | Errors. Only SBML **Level 3 Version 1, FBC package version 2** is accepted (see [ยง6](../raven3-migration/formats-and-reconstruction.md#sbml-io)). | +| `exportToExcelFormat(model, fileName)` where `fileName` is a bare path (no `.xlsx`) | No longer falls back to tab-delimited text export; errors with "only export to xlsx format is supported". | +| `exportToTabDelimited(...)` | Removed outright (was a dead-code path anyway). | +| `getWoLFScores(...)` / `parseScores(..., 'wolf', ...)` | WoLF PSORT support is fully removed. `parseScores` now errors on `'predictor','wolf'` with a message pointing to `'deeploc'`, `'mulocdeep'`, `'compartments'`, or `'uniprot'`. **No in-toolbox migration path for existing WoLF PSORT output files.** | +| `qMOMA(...)`, `solveQP(...)` | Removed (RAVEN's last `quadprog`/Optimization Toolbox dependency). `findGeneDeletions`'s `analysisType`/`refModel`/`oeFactor` arguments (MOMA-based over-expression analysis) were removed with them; only FBA-based single/double gene deletion (`sgd`/`dgd`) remains. | +| `checkRxn(...)` | Removed, no direct successor. For debugging why a reaction can't carry flux, use `gapfilling/findLeakMetabolite.m` / `gapfilling/canExchange.m`, or `gapfilling/checkProduction.m`. | +| `dispEM(...)`, `followFluxes(...)`, `followChanged(...)`, `printModel(...)`, `mapCompartments(...)`, `getExpressionStructure(...)`, `getMetsInComp(...)`, `parallelPoolRAVEN(...)` | All removed (see [ยง4](../raven3-migration/structure-and-renames.md#removed-functionality) for what replaces each, where anything does). | +| Anything under `external/metacyc/`, `pathway/`, `plotting/`, `legacy/` | Entire subsystems removed: MetaCyc-based reconstruction and all visualization/plotting functionality (`drawMap`, `drawPathway`, `markPathwayWithFluxes`, `setOmicDataToRxns`, etc.). No in-toolbox replacement; use COBRA Toolbox or Escher for visualization. | +| `getGenesFromKEGG`, `getMetsFromKEGG`, `getRxnsFromKEGG`, `constructMultiFasta`, `getWSLpath`, `getBlastFromExcel` | Removed along with the old local-KEGG-FTP-dump workflow (see [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction)). | +| `getToolboxVersion(...)` | Removed as a standalone function; folded into an internal helper of `exportForGit`. | + +### Will silently produce different results + +Same call, same input model, different output. Grouped by function; see the +linked section for the full explanation of each. + +| Function | New behavior (default, unless noted) | +|---|---| +| `getModelFromHomology`, `strictness=3` | Best-hit tie-break now uses **bitscore** instead of E-value (`"scoreBy","evalue"` restores the old criterion); [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction) | +| `getModelFromHomology`, `minLen` | Default lowered from `200` to `100`; [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction) | +| `getModelFromHomology`, GPR merging | Gene substitution is exact-token-based instead of regex/`strrep`-based; [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction) | +| `getKEGGModelForOrganism`, `cutOff` | Default lowered from `10^-50` to `10^-30`; [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction) | +| `getKEGGModelForOrganism`, `minScoreRatioG` | Default raised from `0.8` to `0.9`; [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction) | +| `removeGenes`, `expandModel`, `findPotentialErrors`, `addRxns` | grRules are now parsed correctly (exact gene-token matching, structure-aware isozyme/complex handling) instead of via substring/regex matching; [ยง8](../raven3-migration/formats-and-reconstruction.md#gpr-parsing) | +| `changeGrRules(..., 'replace', false)` onto a gene-less reaction | Produces a valid grRule instead of an unparseable one; [ยง8](../raven3-migration/formats-and-reconstruction.md#gpr-parsing) | +| `simplifyModel` (used by task-based gap-filling) | No longer drops a task's boundary constraint on a metabolite with no reactions: a task that used to silently pass may now correctly fail; [ยง9](../raven3-migration/new-and-environment.md#ftinit) | +| `ftINITInternalAlg` | `allowExcretion` constraint sign corrected | +| `ravenCobraWrapper` | Gene-ID-to-MIRIAM matching no longer breaks on gene IDs containing regex metacharacters; [ยง6](../raven3-migration/formats-and-reconstruction.md#raven-cobra-interoperability) | +| `permuteModel`, `randomSampling`, `optimizeProb`, `getFluxZ`, `mergeCompartments`, `convertToIrrev`, `replaceMets`, `setExchangeBounds`, `fitParameters`, `guessComposition` | Each had a pre-existing bug fixed; output differs only for models/inputs that triggered it | + +None of these are API changes; the same call with the same arguments now +returns a different, corrected result. + +### Will warn, but keep working + +- **Error/warning identifiers changed.** `dispEM` (RAVEN 2's universal + error/warning function) always raised errors with an **empty** identifier + (`MException('', ...)`), so no RAVEN 2 script could have legitimately caught + on `err.identifier`. RAVEN 3 replaces `dispEM` with native `error()`/ + `warning()` calls, most (not all) carrying real identifiers, + predominantly `RAVEN:badInput`, plus a handful of specific ones + (`RAVEN:modelError`, `RAVEN:badGrRule`, `RAVEN:sampling`, `RAVEN:infeasible`, + `RAVEN:grRuleTooComplex`). Message text is largely unchanged (many messages + are byte-identical to RAVEN 2's), so `try/catch` blocks that pattern-match on + message text keep working; scripts can newly (optionally) match on + `err.identifier` instead. +- **`getINITModel`/`runINIT` now emit a one-time "legacy" notice** pointing at + `ftINIT`, silenceable with `warning('off','RAVEN:legacyMethod')`. This is + explicitly *not* a deprecation; RAVEN's own documentation states these + functions "remain supported" indefinitely, to reproduce previously published + Human-GEM-derived models. See [ยง9](../raven3-migration/new-and-environment.md#ftinit). +- **`makeSomething`, `consumeSomething`, `canProduce`, `canConsume`** now live in + a new `deprecated/` folder as thin wrappers around `findLeakMetabolite`/ + `canExchange`, and emit a one-time `RAVEN:deprecated` warning stating they + "will be removed in the next major release." Unlike the legacy-notice + functions above, these are on an actual removal track; migrate call sites + before the next major release. See [ยง9](../raven3-migration/new-and-environment.md#ftinit). +- **`checkInstallation` is now `checkRaven`**, moved from `installation/` to the + repository root, with an unchanged signature + (`[currVer, installType] = checkRaven(developMode, checkBinaries)`). Existing + calls do **not** break: `checkInstallation` survives as a forwarding wrapper + that passes every argument and output straight through, after emitting a + `RAVEN:deprecated` warning naming `checkRaven`. It is on the same removal + track as the `deprecated/` functions above, so switch at the next opportunity. + +### Environment / installation changes + +KEGG HMM libraries and external command-line binaries (BLAST+, DIAMOND, HMMER, +cd-hit, MAFFT, WoLF PSORT) are no longer committed to the repository. RAVEN 3 +fetches them on demand from the `raven-data` GitHub release the first time a +function needs one, which means **internet access is required on first use**, +not just at clone/install time. See [ยง11](../raven3-migration/new-and-environment.md#toolbox-deps) for what's needed for +an air-gapped install, and the full dependency comparison. diff --git a/docs/raven3-migration/formats-and-reconstruction.md b/docs/raven3-migration/formats-and-reconstruction.md new file mode 100644 index 0000000..3041aa8 --- /dev/null +++ b/docs/raven3-migration/formats-and-reconstruction.md @@ -0,0 +1,122 @@ +# File formats, reconstruction, and GPR parsing + +(file-formats-and-model-io)= +## 1. File formats and model I/O + +(sbml-io)= +### SBML I/O + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| `exportModel` output format | SBML Level 3 Version 1, FBC v2 | unchanged | +| `importModel` accepted input | any SBML level/version; L2-style notes-based gene associations (`GENE_ASSOCIATION`/`PROTEIN_ASSOCIATION`) also read; FBC fields optional | only SBML **Level 3 Version 1, FBC v2**; anything else errors | +| MIRIAM prefixes read on import | `urn:miriam:`, `http://identifiers.org/`, `https://identifiers.org/` | unchanged | + +Older SBML files must be converted first (e.g. re-exported from COBRApy/COBRA +Toolbox) before `importModel` will accept them. + +(excel-io)= +### Excel I/O + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| Excel engine | Apache POI (Java), bundled | dependency-free OOXML writer (`io/writeExcel.m`) | +| Full-model `.xlsx` import | `importExcelModel.m` (`RXNS`/`METS`/`COMPS`/`GENES`/`MODEL` sheets โ†’ complete model) | removed, no equivalent importer | +| Nearest replacement for curation | no equivalent | `curation/curateModelFromTables.m`: curates/extends an *existing* model from separate `.tsv` files (mets by name+compartment, reactions by stoichiometry, genes by name), not a from-scratch importer | +| `exportToExcelFormat` on a bare path (no `.xlsx`) | fell back to tab-delimited text export | errors, `.xlsx` only | + +If your pipeline depends on the old Excel curation template, either convert it +to the `.tsv` shape `curateModelFromTables` expects, or build the model via +`importModel`/YAML and use `curateModelFromTables` for incremental edits. + +### YAML I/O + +`readYAMLmodel`/`writeYAMLmodel` are pure-MATLAB in both branches; model YAML +I/O has no Python dependency in RAVEN 2 or RAVEN 3. (`io/parseYAML.m` is an +unrelated, new, pure-MATLAB generic YAML parser used only by +`conditions/applyCondition.m` to read condition files; it doesn't affect +model I/O.) Output format changes, visible as diffs in version-controlled +model files: + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| Quote style | single quotes | double quotes | +| Reaction EC codes | top-level `eccodes` field only | also written under `annotation: ec-code` | +| Gene-less model | `genes:` section omitted | `genes: []` written explicitly | +| Empty `gene_reaction_rule` | written as an empty string | omitted | +| `objective_coefficient` on import | ignored | honored | + +(raven-cobra-interoperability)= +### RAVENโ‡„COBRA interoperability + +`conversion/ravenCobraWrapper.m` keeps its signature +(`newModel = ravenCobraWrapper(model)`) unchanged. Two correctness fixes change +output for affected models: a stale-variable bug that could reuse the wrong +MIRIAM data during gene-annotation conversion, and gene IDs containing regex +metacharacters (e.g. `ENSG00000141510.16`) that could be mis-matched during +annotation conversion; both fixed. + +--- + +(kegg-reconstruction)= +## 2. Reconstruction: KEGG and homology pipelines + +### KEGG-based reconstruction (`getModelFromKEGG`, `getKEGGModelForOrganism`) + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| Reaction/gene database source | local KEGG FTP dump (license required), parsed by `getGenesFromKEGG`/`getMetsFromKEGG`/`getRxnsFromKEGG` into a cached `keggModel.mat` | prebuilt artifacts (a gene-free reference model plus KO/reaction/organism tables) fetched on demand from the `raven-data` GitHub release, assembled by `buildGlobalKEGGModel`/`buildGlobalGPR`/`readKEGGTable` | +| `getModelFromKEGG`'s `keggPath` argument | required | removed, not needed | +| Ortholog HMM library | one `.hmm` file per KO (self-built via CD-HIT + MAFFT + `hmmbuild`, or downloaded); queried with one `hmmsearch` call per KO, via WSL on Windows | one concatenated per-domain library (`kegg118_eukaryotes.hmm` / `kegg118_prokaryotes.hmm`); one `hmmsearch` call against the whole proteome, no WSL | +| `dataDir` | arbitrary local folder | must be `kegg118_eukaryotes` or `kegg118_prokaryotes` | +| `nSequences`, `seqIdentity` (CD-HIT clustering controls) | configurable | removed, no local clustering step | +| `cutOff` default | `10^-50` | `10^-30` | +| `minScoreRatioG` default | `0.8` | `0.9` | +| Calling convention | positional | positional-or-named (`parseRAVENargs`); old positional order preserved | + +### Homology-based reconstruction (`getModelFromHomology`) + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| `strictness` parameter | `1`/`2`/`3` | unchanged | +| Best-hit tie-break at `strictness=3` | E-value | bitscore by default (`"scoreBy","evalue"` restores the old criterion) | +| `minLen` default | `200` | `100` | +| GPR gene substitution when merging homology hits | regex/`strrep` substitution (could corrupt IDs that were substrings of one another, or contained regex metacharacters) | exact-token-based substitution | + +### Shared infrastructure + +External binaries (BLAST+, DIAMOND, HMMER) and the KEGG data artifacts +themselves are fetched on demand rather than bundled with the repository; see +[ยง11](../raven3-migration/new-and-environment.md#toolbox-deps). + +**New:** `curation/downloadGenomeData.m`, `curation/getGeneData.m`, and +`curation/processProteinFastaFile.m` give a new on-ramp from a bare NCBI genome +accession (`GCF_`/`GCA_`) straight to a model-ready proteome and gene ID +mapping table. + +--- + +(gpr-parsing)= +## 3. Gene-reaction-rule (GPR) parsing + +RAVEN 3 adds a proper grRule tokenizer/parser: `utils/parseGrRule.m` builds a +parse tree (`type`: `'gene'|'and'|'or'`, with `children`); `utils/grRuleToDNF.m` +converts it to disjunctive normal form (RAVEN's canonical "OR of AND-clauses" +shape for enzyme-complex/isozyme structure); `utils/isDnfGrRule.m` checks +whether a rule is already in that form; `utils/grRuleToString.m` renders a tree +back to a canonical string. These are used internally by four functions, +they are **not** applied automatically to every loaded model; a grRule keeps +its original text unless one of these functions is called on it. + +| Function | RAVEN 2 parsing | RAVEN 3 parsing | +|---|---|---| +| `removeGenes` | substring match on gene ID (`"10"` also matched `"100"`) | exact gene-token match | +| `expandModel` | naive split on `' or '` (could drop an `AND`-ed subunit inside a nested rule) | parse-tree-based isozyme expansion, preserves nested `AND`/`OR` | +| `findPotentialErrors` | string-pattern check for `") and ("` etc. (false positives on valid rules) | parse-tree-based non-DNF detection | +| `addRxns` (new gene IDs from a grRule) | split on the substrings "and"/"or" anywhere (could shred an ID like `"band1"`) | split only on space-bounded `' and '`/`' or '` tokens | +| `comparison/diffModels.m` | (function didn't exist) | grRules compared as logical DNF equality, not string equality | + +The accepted grRule *syntax* itself is unchanged: `' and '`/`' or '` +(case-insensitive) still work exactly as before, and +`manipulation/standardizeGrRules.m` (which runs on import) uses the same +string-splitting logic it always did. diff --git a/docs/raven3-migration/new-and-environment.md b/docs/raven3-migration/new-and-environment.md new file mode 100644 index 0000000..78d31ae --- /dev/null +++ b/docs/raven3-migration/new-and-environment.md @@ -0,0 +1,76 @@ +# New functionality, dependencies, and error handling + +(ftinit)= +## 1. Gap-filling and context-specific model extraction + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| `fillGaps` algorithm | one built-in MILP connectivity-maximization strategy | same strategy is still the default; `algorithm` option adds `'fastLP'`/`'swiftLP'` (FASTCORE/SWIFTCORE-style LP relaxation), `'gapfillMILP'` (growth-floor MILP with directionality repair), `'topological'` (BFS producibility pre-screen, no model modification) | +| `fillGaps` positional arguments | `allowNetProduction, useModelConstraints, supressWarnings, rxnScores` | unchanged; new options are name-value-only | +| `ftINIT` tie-breaking | depends on solver seed | optional `resolveTies` pins degenerate MILP optima to a deterministic answer | +| `ftINIT` MILP optimality proof | escalating relative-gap search only | optional `proveAbsGap` proves each stage to a fixed absolute gap in one solve | +| `ftINIT` positional arguments | preserved | unchanged; `resolveTies`/`proveAbsGap` are name-value-only, default off | +| `getINITModel`/`runINIT` | supported, no runtime notice | supported, now under `INIT/tINIT/`; one-time "legacy, use `ftINIT`" notice (see [ยง5](../raven3-migration/structure-and-renames.md#renamed-merged-and-consolidated-functions)) | +| `fitTasks` gap-fill mode | one behavior (equivalent to today's `'merge'`) | `gapFillMode`: `'merge'` (default, same as before) or `'preMerged'` (faster, ftINIT-style) | +| `fitTasks` outputs | `[outModel, addedRxns]` | `[outModel, addedRxns, failedTasks]` | +| Task requiring net production of a metabolite with no reactions | boundary constraint silently dropped before gap-filling; task could incorrectly report as passing | constraint preserved; task correctly reports failing if the reference model can't produce it | + +--- + +(new-functionality)= +## 2. New functionality + +All additive; none of this replaces or changes behavior of existing RAVEN 2 +functions, except where noted. + +| Area | New function(s) | What it adds | +|---|---|---| +| Flux sampling (`analysis/`) | `sampleACHR`, `sampleCHRR`, `sampleMaxVolEllipse`, `sampleChebyshevCenter`, `sampleWarmupPoints` | `randomSampling` becomes a dispatcher (`'method'` option) across RAVEN 2's original random-objective method (now `'randomObjective'`) plus two new near-uniform MCMC samplers: ACHR (hit-and-run) and CHRR (hit-and-run with rounding, better mixing on ill-conditioned polytopes) | +| Flux analysis / QC | `compareFluxes`, `getMinimalMedium`, `traceFluxPath`, `walkFluxes`, `modelSummary` | structured flux-vector diffing; MILP-based minimal-medium search; highest-flux-fraction path tracing between two reactions; interactive flux-neighborhood navigation; model/flux summary printing | +| Model comparison (`comparison/`) | `diffModels` | structured, ID-keyed diff between two models: stoichiometry, bounds, objective, grRule (as logical DNF equality), EC codes, metabolite properties; distinct from the existing overlap-style `compareMultipleModels` | +| Manipulation | `findDuplicateRxns`, `findPotentialErrors` | standalone duplicate-stoichiometry query; parse-tree-based non-DNF grRule detection (see [ยง8](../raven3-migration/formats-and-reconstruction.md#gpr-parsing)) | +| Genome-based curation (`curation/`, new folder) | `downloadGenomeData`, `getGeneData`, `processProteinFastaFile`, `renameModelGenes`, `curateModelFromTables` | pipeline from an NCBI genome accession to a gene ID mapping table to bulk gene renaming in a model; `curateModelFromTables` bulk-curates mets/rxns/genes from `.tsv` files against an existing model | +| Biomass composition (`biomass/`) | `getBiomassFractions`, `scaleBiomassFraction`, `scaleBiomassPseudoreaction`, `setGAM` | organism-agnostic biomass-composition and growth-associated-maintenance (GAM) calibration; `tutorial/tutorial7.m` demonstrates the workflow | +| Localization (`localization/`) | `assignCompartments`, `defaultCompartmentMap`, `getUniProtScores` | deterministic MILP compartment assignment (alternative to the heuristic `predictLocalization`); `parseScores` gains `'deeploc'`/`'cello'`/`'mulocdeep'`/`'compartments'`/`'uniprot'` predictors (replacing WoLF PSORT; see [ยง3](../raven3-migration/backward-incompatible.md#backward-incompatible-changes)) | +| Annotation (`annotation/`) | `assignSBOterms`, `loadDeltaGCSV`/`saveDeltaGCSV` | organism-agnostic SBO term assignment; ฮ”G thermodynamic data CSV round-trip | +| Conditions (`conditions/`, new folder) | `applyCondition` | applies a data-driven growth/media condition (a YAML file, not code) to a model in a fixed sequence: exchange bounds, optional cofactor-pseudoreaction edits, optional biomass-stoichiometry deltas, per-reaction bound overrides | +| Internal infrastructure | `ravenModelFields` | central registry of every 1-D model field (entity type, default value), backing `permuteModel`/`removeReactions` internally | + +--- + +(toolbox-deps)= +## 3. Toolbox dependencies and installation + +| | RAVEN 2 | RAVEN 3 | +|---|---|---| +| MATLAB version | no pinned/documented minimum (guidance: avoid very new MATLAB syntax, to accommodate older installs) | CI is tested against **R2024b** | +| Statistics Toolbox | required (`combnk`) | not required (`nchoosek`, base MATLAB) | +| Bioinformatics Toolbox | required (`fastaread`/`fastawrite`) | not required (`io/readFasta.m`/`io/writeFasta.m`) | +| Optimization Toolbox | required for `qMOMA`/`solveQP` | not required (both removed) | +| Excel I/O | bundled Apache POI (Java) | dependency-free OOXML writer | +| LP/MILP solver | required (bundled `glpk`, or Gurobi/SCIP/COBRA) | unchanged, still required | +| External binaries (BLAST+, DIAMOND, HMMER, cd-hit, MAFFT) | committed to the repository, available offline immediately after clone | fetched on demand from the `raven-data` GitHub release on first use (per-platform ZIPs); an offline `*-full`/`*-binaries` release bundle is available for air-gapped installs | +| KEGG HMM libraries / reference data | manual download or local build required | fetched on demand from `raven-data`, cached locally after first use | +| Installation self-check | `installation/checkInstallation.m`; ran the full test suite to populate its pass/fail table | renamed `checkRaven`, moved to the repo root; does fast, targeted checks (a small import/export round-trip, a trivial LP per solver) instead of invoking the full test classes, and proactively offers to download missing binaries | + +Net effect: a bare RAVEN 3 clone is far smaller and installs with fewer MATLAB +toolbox prerequisites. The trade-off: it needs network access the first time +KEGG- or homology-based reconstruction functions actually run. + +--- + +## 4. Error and warning handling + +RAVEN 2's `dispEM` always raised errors with an empty identifier, so no RAVEN 2 +`try/catch` could have matched on `err.identifier`; RAVEN 3's move to native +`error()`/`warning()` with real (mostly `RAVEN:*`) identifiers is additive from +that angle, not breaking. Message text is preserved for most existing checks, +so text-matching `try/catch` blocks are unaffected either way. See +[ยง3](../raven3-migration/backward-incompatible.md#backward-incompatible-changes) for the details and the new +`utils/ravenList.m` message-formatting helper. + +One related, purely additive change: `queries/checkModelStruct.m`, called *with* +an output argument (`issues = checkModelStruct(model)`), now returns a +structured array of findings (`category`/`target`/`message`) instead of only +throwing or printing, useful for programmatic model QC. The no-output-argument +calling convention (throws/warns directly) is unchanged. diff --git a/docs/raven3-migration/structure-and-renames.md b/docs/raven3-migration/structure-and-renames.md new file mode 100644 index 0000000..6a3b87e --- /dev/null +++ b/docs/raven3-migration/structure-and-renames.md @@ -0,0 +1,113 @@ +# What moved, what's gone, and what's renamed + +## 1. Repository reorganization + +RAVEN 2's top-level layout (`core/`, `external/`, `hpa/`, `io/`, `legacy/`, +`pathway/`, `plotting/`, `solver/`, `struct_conversion/`, `installation/`, +`utils/`) has been replaced with functional folders that group functions by what +they do rather than by legacy history: + +| RAVEN 3 folder | Contents | Mostly came from (RAVEN 2) | +|---|---|---| +| `INIT/` | `ftINIT`, `INIT/tINIT/` for `getINITModel`/`runINIT`, and HPA/expression parsing + gene scoring (`parseHPA`, `parseHPArna`, `scoreModel`) | `INIT/`, `hpa/` | +| `analysis/` | FBA-adjacent analyses: FVA, sampling, FSEOF, OptKnock, robustness, production envelope, flux comparison/tracing | `core/` | +| `annotation/` | MIRIAM editing, SBO term assignment, ฮ”G CSV I/O | `struct_conversion/` (+ new) | +| `biomass/` | Biomass composition fitting/scaling, GAM | `core/` (+ new) | +| `comparison/` | Multi-model comparison, structured model diffing | `core/` (+ new) | +| `conditions/` | Applying a data-driven growth/media condition to a model | *new folder* | +| `conversion/` | RAVENโ‡„COBRA conversion, identifier prefixing, field ordering | `struct_conversion/` | +| `curation/` | Genome-annotation-driven gene/model curation | *new folder* | +| `gapfilling/` | `fillGaps` and its pluggable algorithms, task fitting, leak-metabolite/exchange checks | `core/` (+ new) | +| `installation/` | Path setup, on-demand binary/data download | `installation/` | +| `io/` | Model import/export (SBML, YAML, Excel), FASTA I/O | `io/` | +| `localization/` | Subcellular localization prediction/scoring, compartment assignment | `external/` (+ new) | +| `manipulation/` | Structural edits: add/remove/change rxns, mets, genes; merge, sort, simplify | `core/` | +| `queries/` | Non-mutating lookups: indexes, exchange/transport reactions, stoichiometry construction, model checking | `core/` | +| `reconstruction/` | KEGG- and homology-based draft reconstruction | `external/` | +| `solver/` | LP/MILP solving | `solver/` | +| `tasks/` | Metabolic task list parsing/checking | `core/` | +| `testing/` | `matlab.unittest` suite (`function_tests/`) | `testing/` | +| `utils/` | Generic helpers, including the new argument-parsing/registry infrastructure | `utils/` | +| `deprecated/` | Thin backward-compatible wrappers for merged/renamed functions | *new folder* | + +Entirely removed (no successor folder): `external/metacyc/` (MetaCyc +reconstruction), `pathway/` and `plotting/` (visualization), `legacy/` +(CellDesigner import and other pre-2.0 code), `struct_conversion/`'s Excel-POI +helpers. + +**The reorganization affects fewer scripts than the change list suggests.** +If your scripts call RAVEN +functions by name (the normal usage pattern, with the whole RAVEN folder tree, +including subfolders, added to the MATLAB path via `addRavenToUserPath` or +`addpath(genpath(...))`), moving a file to a new subfolder does not break the +call: MATLAB resolves functions by name across the whole path, not by a +hard-coded relative location. The reorganization only matters if you (a) +hard-coded a path to a specific `.m` file, (b) called one of the functions that +was actually removed or merged (see below), or (c) shadowed a RAVEN function +name with your own; folder names are irrelevant otherwise. + +The full moved-file mapping (RAVEN 2 path โ†’ RAVEN 3 path) for every relocated +function is available on request / in the repository's git history +(`git diff --stat -M develop...develop3`); it is not reproduced in full here +because it is large and, as the point above explains, rarely something you +need to act on. + +--- + +(removed-functionality)= +## 2. Removed functionality + +Beyond the items already listed as backward-incompatible above, RAVEN 3 removed +a number of functions and whole subsystems as part of a deliberate effort to +make the codebase smaller. None of these have call sites left in RAVEN 3 itself. + +| Removed | What it did | Replacement / notes | +|---|---|---| +| `pathway/*`, `plotting/*` (16 functions: `drawMap`, `drawPathway`, `colorPathway`, `colorSubsystem`, `markPathwayWithFluxes`, `markPathwayWithExpression`, `setOmicDataToRxns`, `plotAdditionalInfo`, etc.) | Metabolic map/pathway visualization | None in RAVEN core. Use COBRA Toolbox, Escher, or export flux/omics data for external plotting. | +| `external/metacyc/*` (7 functions: `getModelFromMetaCyc`, `getRxnsFromMetaCyc`, `getMetsFromMetaCyc`, `getEnzymesFromMetaCyc`, `addSpontaneousRxns`, `linkMetaCycKEGGRxns`, `combineMetaCycKEGGModels`) | MetaCyc-based draft reconstruction | None; KEGG- and homology-based reconstruction remain the supported paths. | +| `legacy/*` (CellDesigner import, `changeGeneAssoc`, the bundled `xml_toolbox`) | Pre-2.0 compatibility code | None; considered dead. | +| `io/importExcelModel.m`, `io/loadWorkbook.m`, `io/loadSheet.m`, `io/writeSheet.m`, `io/SBMLFromExcel.m`, `io/addJavaPaths.m`, `io/startup.m` | Apache-POI-based Excel model I/O | `io/writeExcel.m` (dependency-free OOXML writer) for export; `curation/curateModelFromTables.m` for tabular curation. No full Excel-template *import* path remains; see [ยง6](../raven3-migration/formats-and-reconstruction.md#excel-io). | +| `io/exportModelToSIF.m`, `io/exportToTabDelimited.m` | SIF graph export; tab-delimited model export | Dead/orphaned code with no callers; removed outright. | +| `io/getFullPath.m`, `io/getMD5Hash.m`, `io/getToolboxVersion.m` | Path canonicalization; file MD5 hashing; toolbox git-version lookup | Inlined where still needed (e.g. MD5 via Java's `MessageDigest` directly inside `getBlast`), or folded into `exportForGit`. | +| `core/dispEM.m` | Universal error/warning display | Native `warning()`/`error()` + new `utils/ravenList.m` for formatted item lists. See [ยง3](../raven3-migration/backward-incompatible.md#backward-incompatible-changes). | +| `core/followFluxes.m`, `core/followChanged.m` | Flux-change reporting relative to a reference solution | `queries/printFluxes.m` covers the cutoff-filtering half; nothing replaces the reference-flux comparison directly. | +| `core/printModel.m` | Printed reactions to screen/file | `queries/printModelStats.m` / `queries/printFluxes.m` cover related summary/flux printing; no direct one-line-per-reaction dump remains. | +| `core/getMetsInComp.m` | Returned metabolite indices in a given compartment | Inline it as `model.metComps == compIndex`. | +| `core/mapCompartments.m` | Remapped compartment labels in a localization-score structure | Superseded by the new predictor-based localization workflow and `localization/defaultCompartmentMap.m`. | +| `core/getExpressionStructure.m` | Loaded an expression-experiment structure from an ad hoc Excel format | Superseded by `INIT/parseHPA.m`/`INIT/parseHPArna.m` for the supported expression-data workflow. | +| `core/checkRxn.m` | Per-reaction reactant/product synthesizability debugging | No direct successor; use `gapfilling/findLeakMetabolite.m`, `gapfilling/canExchange.m`, or `gapfilling/checkProduction.m`. | +| `core/parallelPoolRAVEN.m` | Parallel Computing Toolbox pool setup | Renamed/rewritten as `utils/parallelWorkersRAVEN.m`. | +| `external/getWoLFScores.m` | WoLF PSORT score parsing | None; see [ยง3](../raven3-migration/backward-incompatible.md#backward-incompatible-changes). | +| `external/kegg/{getGenesFromKEGG,getMetsFromKEGG,getRxnsFromKEGG,constructMultiFasta,getWSLpath}.m`, `external/getBlastFromExcel.m` | Local-KEGG-FTP-dump parsing; per-KO multi-FASTA assembly; WSL path translation; Excel-based BLAST import | Superseded by the raven-data-artifact-based KEGG pipeline. See [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction). | +| `solver/qMOMA.m`, `solver/solveQP.m` | MOMA-based analyses (quadratic programming) | Removed; no in-toolbox QP-based replacement. See [ยง3](../raven3-migration/backward-incompatible.md#backward-incompatible-changes) for `findGeneDeletions` impact. | +| `testing/unit_tests/*.m` (script-based tests) | Ad hoc test scripts | `testing/function_tests/*.m`, a `matlab.unittest`-class-based suite. Contributor-facing only; `runRAVENtests` usage is unchanged for end users. | + +--- + +(renamed-merged-and-consolidated-functions)= +## 3. Renamed, merged, and consolidated functions + +RAVEN 3 introduces a formal **`deprecated/` folder convention**: functions that +were merged into a more general replacement stay callable under their old name, +as a thin wrapper that emits a one-time `RAVEN:deprecated` warning ("will be +removed in the next major release") and forwards to the new function. + +| Old function(s) | New function | Notes | +|---|---|---| +| `makeSomething(model, ignoreMets, isNames, minNrFluxes, allowExcretion, params, ignoreIntBounds)` | `findLeakMetabolite(model, 'produce', ...)` | Argument order maps 1:1; the wrapper forwards `varargin` unchanged. | +| `consumeSomething(model, ignoreMets, isNames, minNrFluxes, params, ignoreIntBounds)` | `findLeakMetabolite(model, 'consume', ...)` | **Not** a verbatim forward: `consumeSomething`'s old positional order has no `allowExcretion` slot, so the wrapper explicitly remaps arguments by name to avoid corrupting `params`/`ignoreIntBounds`. | +| `canProduce(model, mets)` | `canExchange(model, 'produce', 'mets', mets)` | | +| `canConsume(model, mets)` | `canExchange(model, 'consume', 'mets', mets)` | | +| `compareRxnsGenesMetsComps` | folded into `comparison/compareMultipleModels.m` | Its rxn/gene/met/comp overlap output is now one part of `compareMultipleModels`'s output; no separate function remains. | +| `INIT/scoreComplexModel.m` | `INIT/scoreModel.m` | **Merged, not renamed.** RAVEN 2 had *both* `INIT/scoreComplexModel.m` and `hpa/scoreModel.m`; RAVEN 3 folded them into one `scoreModel`, called with different settings from tINIT than from ftINIT. A RAVEN 2 call to `scoreModel` is unaffected; a call to `scoreComplexModel` has to move to `scoreModel` *and* pass the settings that reproduce the old behaviour (a single operator for both `and`/`or`, `dataPrecedence` `'reaction'`). | +| `INIT/ftINITFillGapsMILP.m` | folded into `solver/getMinNrFluxes.m` | Exposed now via `getMinNrFluxes`'s `resolveTies`/`proveAbsGap` options. | +| `INIT/ftINITFillGapsForAllTasks.m` | folded into `gapfilling/ftINITFillGaps.m` and `fitTasks`'s `'preMerged'` gap-fill mode | | +| `getGenesFromKEGG`/`getMetsFromKEGG`/`getRxnsFromKEGG` | `reconstruction/kegg/readKEGGTable.m` + the raven-data KEGG artifacts | Not a direct signature-compatible rename: the whole data source changed; see [ยง7](../raven3-migration/formats-and-reconstruction.md#kegg-reconstruction). | +| `checkInstallation` | `checkRaven` | Moved from `installation/checkInstallation.m` to repo-root `checkRaven.m`; identical calling signature. `checkInstallation` remains as a deprecated forwarding wrapper, so old calls keep working with a `RAVEN:deprecated` warning. | + +Separately, `getINITModel` and `runINIT` (now under `INIT/tINIT/`) are marked +**legacy but indefinitely supported** (not on the `deprecated/` removal track) +via `utils/legacyMethodNotice.m`, which fires a one-time, silenceable +`RAVEN:legacyMethod` warning pointing at `ftINIT`. Use `ftINIT` for new work; +keep `getINITModel`/`runINIT` only to reproduce models originally built with +them. diff --git a/docs/raven3-vs-raven-toolbox.md b/docs/raven3-vs-raven-toolbox.md index c2ad747..3ee2472 100644 --- a/docs/raven3-vs-raven-toolbox.md +++ b/docs/raven3-vs-raven-toolbox.md @@ -1,4 +1,4 @@ -# RAVEN and raven-toolbox +# RAVEN vs. raven-toolbox :::{note} Looking for a specific function? The [MATLAB vs Python](matlab-vs-python.md) table pairs every function that @@ -58,164 +58,12 @@ Check the [mapping table](matlab-vs-python.md) rather than guessing. It also records what cobrapy covers instead of raven-toolbox, and what is deliberately absent. -## What only raven-toolbox has +## Detail pages -### Built on cobrapy - -The largest difference is not a feature but a foundation: a raven-toolbox model -**is** a `cobra.Model`. Every cobrapy tool, and everything in the wider COBRA -Python ecosystem, works on it without conversion. RAVEN's own model is a MATLAB -struct, and `ravenCobraWrapper` converts between that and the COBRA Toolbox -format as an explicit step. - -The practical consequence: FBA, FVA, knockouts, media handling and the rest come -from cobrapy rather than from raven-toolbox, which is why so many RAVEN -functions have a *cobrapy* row rather than a Python counterpart in the mapping -table. - -### KEGG artefact generation - -raven-toolbox can build the KEGG reference artefacts themselves, parsing a KEGG -release into reaction and compound tables, assembling the reference model, -building the per-KO FASTA sets and HMM libraries, and deriving the phylogenetic -distance matrix. RAVEN consumes pre-built artefacts; it does not produce them. - -This is what keeps the KEGG route reproducible against a stated KEGG release -rather than against whichever artefact is currently distributed. - -### Smaller additions - -- **Growth conditions**: apply a named, versioned growth condition to a model. -- **Batch curation**: apply a table of curation edits to a model in one pass. -- **ฮ”G and SBO annotation**: load and save thermodynamic data through CSV, and - assign SBO terms. -- **Biomass helpers**: sum a biomass composition, rescale a pseudoreaction, and - scale a fraction to a measured value. -- **Checksummed provisioning**: both toolboxes now fetch BLAST+, DIAMOND, - HMMER and the KEGG artefacts on demand rather than bundling them, from the - same release. What is Python-only is the baked registry that pins a given - release to the exact assets it was tested against; RAVEN resolves from the - published release each time. See - [Download data and binaries](installation/data-and-binaries.md). - -## What only RAVEN has - -### tINIT - -`getINITModel` and `runINIT` are the original tINIT implementation. RAVEN keeps -them for the models already built with them; raven-toolbox, a new -implementation with no such installed base, carries ftINIT alone. A tINIT model -has to be reproduced in MATLAB; see -[14. Context-specific models](guide/init.md). - -### COBRA Toolbox conversion - -`ravenCobraWrapper` converts between the RAVEN and COBRA Toolbox model -structures. There is nothing to convert in Python: the model is already a -`cobra.Model`, so no equivalent exists or is needed. - -### Dynamic FBA - -`runDynamicFBA` has no Python counterpart, deliberately. Several maintained -Python packages already cover dynamic FBA well -([dfba](https://pypi.org/project/dfba/), -[reframed](https://pypi.org/project/reframed/), -[mewpy](https://pypi.org/project/mewpy/)), and reimplementing it would add a -second-rate version of something that already exists. - -### MATLAB-specific plumbing - -A large share of RAVEN's function count is MATLAB housekeeping with nothing to -map to: path management (`addRavenToUserPath`, `findRAVENroot`), argument -handling (`parseRAVENargs`, `convertCharArray`), progress and printing -(`setRavenProgress`, `printOrange`), and the solver abstraction -(`optimizeProb`, `setRavenSolver`), which in Python is cobrapy's solver -interface via optlang. - -## Same function, different answer - -Where the two agree on the job but differ in what they hand back, how they order -it, or what they do to the model on the way. Deliberately short: an entry is -added only once the difference has been confirmed in both sources. - -:::{admonition} Not a complete list -:class: info -Absence from this section is not a guarantee of identical behaviour. It only -means the difference has not been confirmed and written up yet. -::: - -### Duplicate reactions: gene associations are not merged - -`contractModel` merges duplicate reactions **and their gene associations**: when -it collapses a set of duplicates it joins the distinct `grRules` with `or`, so -every gene that pointed at any of the duplicates still points at the survivor. - -`remove_duplicate_reactions` keeps one reaction of each duplicate set and removes -the rest, without merging gene associations. A gene that was associated *only* -with a removed duplicate is no longer associated with anything. - -The stoichiometric network is the same either way; the geneโ€“reaction mapping is -not. If you are contracting a draft assembled from several templates, where the -same reaction commonly arrives with different gene associations, check the GPRs -of the survivors afterwards. - -### Metabolic tasks: the same result, a different runtime - -`checkTasks` rebuilds the working model from the original for each task. -`check_tasks` instead applies each task's constraints to one model inside a -`with model:` block and reverts them afterwards, restoring by hand the one kind -of edit cobra's context manager does not track (direct mass-balance bound -changes). - -The pass and fail results are the same. The runtime is not: at genome scale, -copying the model for each task dominates the MATLAB runtime, which is why the -Python version reuses a single model. This changes runtime comparisons, not -results. - -### Gap-filling: `fillGaps` splits into three functions in raven-toolbox - -`fillGaps` does several different jobs through one function call. -raven-toolbox splits them, so porting a `fillGaps` call means choosing: - -| What you were doing | Use | +| | | |---|---| -| Connecting blocked reactions against template models | `connect_blocked_reactions` | -| Fast LP-based filling of a large candidate set | `fill_gaps_fast_lp` | -| MILP filling with explicit weights | `fill_gaps_kumar_milp` | -| Only *finding* the gaps (`canExchange`, `checkProduction`, `getAllSubGraphs`, `haveFlux`) | `analyse_topology` | - -The choice changes both the reaction set added and the runtime, because each -of the three functions uses a different algorithm; they are not the same -algorithm under three different names. - -### Anything solved by MILP - -ftINIT extraction, gap-filling, and compartment assignment all solve -mixed-integer problems that routinely have **several optima of equal objective -value**. Two runs can return different reaction sets and both be correct; across -languages, across solvers, and in some configurations across runs of the same -solver. - -Do not compare these outputs for identity. Compare them for overlap, and -expect a range of acceptable values, not one exact number. - -### Elemental balance: an unknown result is not the same as balanced - -Elemental balance checks whether a reaction has the same count of each -chemical element (carbon, hydrogen, oxygen, and so on) on both sides, the way -a correct chemical equation must. `getElementalBalance` and -`get_elemental_balance` each report one of three results per reaction, not -two: `balanced`, `unbalanced`, or `unknown`. A reaction is `unknown` when one -of its metabolites has no chemical formula recorded, so the element counts -cannot be computed at all; that is different from `unbalanced`, which means -the counts were computed and did not match. The two functions agree on this -three-way result. - -That third case is lost if you switch to plain cobrapy: its -`check_mass_balance` has no separate `unknown` result, so a reaction with a -missing formula and a reaction that is genuinely unbalanced can look the -same. raven-toolbox keeps its own function instead of using cobrapy's -directly, specifically to keep that distinction. +| [What each has that the other doesn't](raven3-vs-raven-toolbox/what-each-has.md) | Capabilities that exist on only one side: cobrapy's foundation and KEGG artefact generation on the Python side; tINIT, COBRA Toolbox conversion, dynamic FBA, and MATLAB-specific plumbing on the MATLAB side. | +| [Same function, different answer](raven3-vs-raven-toolbox/same-function-different-answer.md) | Where both sides do the same job but hand back a different result: gene-association merging, task-checking runtime, gap-filling's three-way split, MILP alternate optima, and elemental balance. | ## Solvers @@ -229,3 +77,10 @@ Genome-scale MILP work (ftINIT extraction in particular) is where the choice matters most; see [Installation](installation/index.md) for the full solver matrix. +```{toctree} +:hidden: + +raven3-vs-raven-toolbox/what-each-has +raven3-vs-raven-toolbox/same-function-different-answer +``` + diff --git a/docs/raven3-vs-raven-toolbox/same-function-different-answer.md b/docs/raven3-vs-raven-toolbox/same-function-different-answer.md new file mode 100644 index 0000000..5147d16 --- /dev/null +++ b/docs/raven3-vs-raven-toolbox/same-function-different-answer.md @@ -0,0 +1,84 @@ +# Same function, different answer + +Where the two agree on the job but differ in what they hand back, how they order +it, or what they do to the model on the way. Deliberately short: an entry is +added only once the difference has been confirmed in both sources. + +:::{admonition} Not a complete list +:class: info +Absence from this section is not a guarantee of identical behaviour. It only +means the difference has not been confirmed and written up yet. +::: + +## Duplicate reactions: gene associations are not merged + +`contractModel` merges duplicate reactions **and their gene associations**: when +it collapses a set of duplicates it joins the distinct `grRules` with `or`, so +every gene that pointed at any of the duplicates still points at the survivor. + +`remove_duplicate_reactions` keeps one reaction of each duplicate set and removes +the rest, without merging gene associations. A gene that was associated *only* +with a removed duplicate is no longer associated with anything. + +The stoichiometric network is the same either way; the geneโ€“reaction mapping is +not. If you are contracting a draft assembled from several templates, where the +same reaction commonly arrives with different gene associations, check the GPRs +of the survivors afterwards. + +## Metabolic tasks: the same result, a different runtime + +`checkTasks` rebuilds the working model from the original for each task. +`check_tasks` instead applies each task's constraints to one model inside a +`with model:` block and reverts them afterwards, restoring by hand the one kind +of edit cobra's context manager does not track (direct mass-balance bound +changes). + +The pass and fail results are the same. The runtime is not: at genome scale, +copying the model for each task dominates the MATLAB runtime, which is why the +Python version reuses a single model. This changes runtime comparisons, not +results. + +## Gap-filling: `fillGaps` splits into three functions in raven-toolbox + +`fillGaps` does several different jobs through one function call. +raven-toolbox splits them, so porting a `fillGaps` call means choosing: + +| What you were doing | Use | +|---|---| +| Connecting blocked reactions against template models | `connect_blocked_reactions` | +| Fast LP-based filling of a large candidate set | `fill_gaps_fast_lp` | +| MILP filling with explicit weights | `fill_gaps_kumar_milp` | +| Only *finding* the gaps (`canExchange`, `checkProduction`, `getAllSubGraphs`, `haveFlux`) | `analyse_topology` | + +The choice changes both the reaction set added and the runtime, because each +of the three functions uses a different algorithm; they are not the same +algorithm under three different names. + +## Anything solved by MILP + +ftINIT extraction, gap-filling, and compartment assignment all solve +mixed-integer problems that routinely have **several optima of equal objective +value**. Two runs can return different reaction sets and both be correct; across +languages, across solvers, and in some configurations across runs of the same +solver. + +Do not compare these outputs for identity. Compare them for overlap, and +expect a range of acceptable values, not one exact number. + +## Elemental balance: an unknown result is not the same as balanced + +Elemental balance checks whether a reaction has the same count of each +chemical element (carbon, hydrogen, oxygen, and so on) on both sides, the way +a correct chemical equation must. `getElementalBalance` and +`get_elemental_balance` each report one of three results per reaction, not +two: `balanced`, `unbalanced`, or `unknown`. A reaction is `unknown` when one +of its metabolites has no chemical formula recorded, so the element counts +cannot be computed at all; that is different from `unbalanced`, which means +the counts were computed and did not match. The two functions agree on this +three-way result. + +That third case is lost if you switch to plain cobrapy: its +`check_mass_balance` has no separate `unknown` result, so a reaction with a +missing formula and a reaction that is genuinely unbalanced can look the +same. raven-toolbox keeps its own function instead of using cobrapy's +directly, specifically to keep that distinction. diff --git a/docs/raven3-vs-raven-toolbox/what-each-has.md b/docs/raven3-vs-raven-toolbox/what-each-has.md new file mode 100644 index 0000000..d145a5a --- /dev/null +++ b/docs/raven3-vs-raven-toolbox/what-each-has.md @@ -0,0 +1,75 @@ +# What each has that the other doesn't + +## What only raven-toolbox has + +### Built on cobrapy + +The largest difference is not a feature but a foundation: a raven-toolbox model +**is** a `cobra.Model`. Every cobrapy tool, and everything in the wider COBRA +Python ecosystem, works on it without conversion. RAVEN's own model is a MATLAB +struct, and `ravenCobraWrapper` converts between that and the COBRA Toolbox +format as an explicit step. + +The practical consequence: FBA, FVA, knockouts, media handling and the rest come +from cobrapy rather than from raven-toolbox, which is why so many RAVEN +functions have a *cobrapy* row rather than a Python counterpart in the mapping +table. + +### KEGG artefact generation + +raven-toolbox can build the KEGG reference artefacts themselves, parsing a KEGG +release into reaction and compound tables, assembling the reference model, +building the per-KO FASTA sets and HMM libraries, and deriving the phylogenetic +distance matrix. RAVEN consumes pre-built artefacts; it does not produce them. + +This is what keeps the KEGG route reproducible against a stated KEGG release +rather than against whichever artefact is currently distributed. + +### Smaller additions + +- **Growth conditions**: apply a named, versioned growth condition to a model. +- **Batch curation**: apply a table of curation edits to a model in one pass. +- **ฮ”G and SBO annotation**: load and save thermodynamic data through CSV, and + assign SBO terms. +- **Biomass helpers**: sum a biomass composition, rescale a pseudoreaction, and + scale a fraction to a measured value. +- **Checksummed provisioning**: both toolboxes now fetch BLAST+, DIAMOND, + HMMER and the KEGG artefacts on demand rather than bundling them, from the + same release. What is Python-only is the baked registry that pins a given + release to the exact assets it was tested against; RAVEN resolves from the + published release each time. See + [Download data and binaries](../installation/data-and-binaries.md). + +## What only RAVEN has + +### tINIT + +`getINITModel` and `runINIT` are the original tINIT implementation. RAVEN keeps +them for the models already built with them; raven-toolbox, a new +implementation with no such installed base, carries ftINIT alone. A tINIT model +has to be reproduced in MATLAB; see +[14. Context-specific models](../guide/init.md). + +### COBRA Toolbox conversion + +`ravenCobraWrapper` converts between the RAVEN and COBRA Toolbox model +structures. There is nothing to convert in Python: the model is already a +`cobra.Model`, so no equivalent exists or is needed. + +### Dynamic FBA + +`runDynamicFBA` has no Python counterpart, deliberately. Several maintained +Python packages already cover dynamic FBA well +([dfba](https://pypi.org/project/dfba/), +[reframed](https://pypi.org/project/reframed/), +[mewpy](https://pypi.org/project/mewpy/)), and reimplementing it would add a +second-rate version of something that already exists. + +### MATLAB-specific plumbing + +A large share of RAVEN's function count is MATLAB housekeeping with nothing to +map to: path management (`addRavenToUserPath`, `findRAVENroot`), argument +handling (`parseRAVENargs`, `convertCharArray`), progress and printing +(`setRavenProgress`, `printOrange`), and the solver abstraction +(`optimizeProb`, `setRavenSolver`), which in Python is cobrapy's solver +interface via optlang. diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 6a60e42..1c32655 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -27,7 +27,7 @@ folder of the RAVEN repository. `checkRaven` passes; see [Installation](../installation/raven.md). - Tutorials 2โ€“4 involve editing models in **RAVEN-compatible Excel format**, using `importExcelModel`. RAVEN 3 removed `importExcelModel`, so this step - does not run as written on RAVEN 3; see [Excel I/O](../raven3-migration.md#excel-io) + does not run as written on RAVEN 3; see [Excel I/O](../raven3-migration/formats-and-reconstruction.md#excel-io) for the replacement, `curateModelFromTables`, which curates an existing model from `.tsv` files rather than importing a full model from `.xlsx`. - To run a section of a script in MATLAB, highlight it, right-click, and choose @@ -42,7 +42,7 @@ raven-toolbox function name; some RAVEN functions map to cobrapy instead (`solveLP` โ†’ `model.optimize()`), and some have no Python counterpart at all (there is no Excel reader). Look each one up in the [API reference](../api/index.md), and see -[RAVEN 3 and raven-toolbox](../raven3-vs-raven-toolbox.md) for what maps where. +[RAVEN vs. raven-toolbox](../raven3-vs-raven-toolbox.md) for what maps where. ::: ```{toctree} diff --git a/docs/yaml-format.md b/docs/yaml-format.md index 7b21afe..db12aa4 100644 --- a/docs/yaml-format.md +++ b/docs/yaml-format.md @@ -1,4 +1,4 @@ -# RAVEN / cobrapy YAML model format +# YAML file format This document describes the YAML format produced and consumed by diff --git a/scripts/curated_pairs.yml b/scripts/curated_pairs.yml index c1ab863..3aedf4c 100644 --- a/scripts/curated_pairs.yml +++ b/scripts/curated_pairs.yml @@ -75,6 +75,20 @@ aliases: getIndexes: parse_name_comp exportToExcelFormat: export_to_excel + # Gap-filling and analysis + getEssentialRxns: find_task_essential_reactions + deleteUnusedGenes: remove_genes + gapFillFastLP: fill_gaps_fast_lp + gapFillMILP: fill_gaps_kumar_milp + gapFillTopological: analyse_topology + makeFakeBlastStructure: make_ortholog_hits + getModelFromKEGG: build_reference_model + sampleACHR: random_sampling + sampleCHRR: random_sampling + gapFillFastCore: fill_gaps_fast_lp + gapFillSwiftCore: fill_gaps_fast_lp + replaceMets: replace_metabolite + cobrapy: # RAVEN function -> the cobrapy object or function that replaces it. importModel: cobra.io.read_sbml_model @@ -91,8 +105,29 @@ cobrapy: runPhenotypePhasePlane: cobra.flux_analysis.production_envelope modelSummary: cobra.Model.summary setRavenSolver: cobra.Configuration + addMets: cobra.Metabolite + addGenesRaven: cobra.Reaction.gene_reaction_rule + addExchangeRxns: cobra.Model.add_boundary + getAllowedBounds: cobra.flux_analysis.flux_variability_analysis + printModelStats: cobra.Model + addIdentifierPrefix: cobra.io.read_sbml_model + removeIdentifierPrefix: cobra.io.write_sbml_model + getMinimalMedium: cobra.medium.minimal_medium + optimizeProb: cobra.Model.solver + parseGrRule: cobra.core.gene.GPR + grRuleToString: cobra.core.gene.GPR + getGenesFromGrRules: cobra.core.gene.GPR + getAllRxnsFromGenes: cobra.Gene.reactions not_ported: + getINITModel: >- + The original tINIT implementation. RAVEN keeps it for models already + built with it; raven-toolbox, a new implementation with no such + installed base, carries ftINIT alone. + runINIT: >- + The original tINIT implementation. RAVEN keeps it for models already + built with it; raven-toolbox, a new implementation with no such + installed base, carries ftINIT alone. runDynamicFBA: >- Dynamic FBA is covered by maintained Python packages (dfba, reframed, mewpy). ravenCobraWrapper: >- @@ -107,3 +142,132 @@ not_ported: printFluxes: >- No equivalent yet. Use cobra's model.summary(), or the solution's fluxes Series directly. + + # Internal helpers, subsumed into an already-mapped public function + buildEquation: >- + Internal helper for building an equation string from raw metabolites and + coefficients; the equivalent is inside add_reactions_from_equations and + cobra.Reaction.build_reaction_string, both already mapped. + constructS: >- + Internal helper for building a stoichiometric matrix from equation + strings; the equivalent is inside add_reactions_from_equations, already + mapped from addRxns. + parseRxnEqu: >- + Internal helper for extracting metabolite names from equation strings; + the equivalent is inside add_reactions_from_equations, already mapped + from addRxns. + buildGlobalGPR: >- + Internal to assembling the global KEGG reference model; the equivalent + is inside build_reference_model, already mapped from getModelFromKEGG. + buildGlobalKEGGModel: >- + Internal to assembling the global KEGG reference model; the equivalent + is inside build_reference_model, already mapped from getModelFromKEGG. + keggDataVersion: >- + Internal to assembling the global KEGG reference model; the equivalent + is inside build_reference_model, already mapped from getModelFromKEGG. + parseFormulas: >- + Internal formula-to-elemental-composition parser; the equivalent is + inside get_elemental_balance, not exposed separately. + writeExcel: >- + Internal low-level .xlsx writer; the equivalent is inside + export_to_excel, already mapped from exportToExcelFormat. + cleanSheet: >- + Internal to the old Excel-import pipeline (importExcelModel), which + RAVEN 3 removed; raven-toolbox never had an Excel reader either. + sampleChebyshevCenter: >- + Internal rounding-step subroutine of RAVEN's own sampleCHRR; raven-toolbox's + random_sampling (already mapped from sampleACHR/sampleCHRR) handles + rounding and warmup internally without exposing separate functions. + sampleMaxVolEllipse: >- + Internal rounding-step subroutine of RAVEN's own sampleCHRR; raven-toolbox's + random_sampling (already mapped from sampleACHR/sampleCHRR) handles + rounding and warmup internally without exposing separate functions. + sampleWarmupPoints: >- + Internal warmup-point generator for RAVEN's own sampleACHR; + raven-toolbox's random_sampling (already mapped from + sampleACHR/sampleCHRR) handles warmup internally. + + # MATLAB-specific plumbing, with no Python equivalent needed + convertCharArray: >- + MATLAB char-array/string-array normalisation; Python has one string type. + parseRAVENargs: >- + Resolves RAVEN's positional-or-named optional arguments; Python + functions take native keyword arguments directly. + parseYAML: >- + A generic YAML parser for condition files; Python's standard YAML + libraries (PyYAML) cover this directly, with no toolbox wrapper needed. + printOrange: >- + Prints coloured text to the MATLAB Command Window; not applicable + outside the MATLAB GUI. + ravenList: >- + Formats a header and item list for a MATLAB warning()/error() call; + Python's own string formatting covers this directly. + deprecationWarning: >- + Emits RAVEN's own one-time deprecated-function notice; Python's + warnings.warn covers this directly. + legacyMethodNotice: >- + Emits RAVEN's own one-time legacy-method notice; Python's warnings.warn + covers this directly. + parallelWorkersRAVEN: >- + Returns a parfor worker count for the Parallel Computing Toolbox; + raven-toolbox's parallelism is configured directly through + cobra.Configuration().processes. + setRavenProgress: >- + Sets a preference for how RAVEN reports progress in the MATLAB GUI or + console; no equivalent concept in Python. + checkFileExistence: >- + A generic file-existence check; Python's own pathlib/os.path cover this + directly. + checkSolution: >- + Checks whether a Mosek solution is feasible and optimal; Mosek is not a + solver either toolbox supports through this site's documented setup. + splitProbForConditioning: >- + Improves LP numerical conditioning by splitting wide-ratio columns + before solving; cobrapy and the solvers it wraps (through optlang) + handle numerical scaling internally, with no equivalent public option. + permuteModel: >- + Reorders the parallel arrays of a RAVEN MATLAB struct; a cobra.Model has + no comparable field-order concept to reorder. + standardizeModelFieldOrder: >- + Orders the fields of a RAVEN MATLAB struct to a canonical layout; a + cobra.Model has no comparable field-order concept. + ravenModelFields: >- + A registry of RAVEN's own MATLAB struct fields, used internally by + permuteModel and removeReactions; a cobra.Model has no comparable + struct-field concept. + runRAVENtests: >- + Runs RAVEN's own MATLAB unit test suite. raven-toolbox's tests run + through pytest, the standard Python testing workflow, not a toolbox + function. + defaultCompartmentMap: >- + raven-toolbox exposes the same table as a module-level constant, + localization.DEFAULT_COMPARTMENT_MAP, not a function. + + # No raven-toolbox equivalent yet + findPotentialErrors: >- + Detects grRules that are not in disjunctive normal form. cobrapy's own + GPR class represents a parsed rule directly, so raven-toolbox has no + equivalent "detect an invalid legacy string" step. + getTransportRxns: >- + Classifies which reactions move a metabolite between compartments. + raven-toolbox has no equivalent classifier yet. + removeBadRxns: >- + Iteratively removes reactions that let the model make or consume + something from nothing. raven-toolbox has no equivalent yet; see + find_leak_metabolite (mapped from makeSomething/consumeSomething) for + finding the same kind of issue. + reverseRxns: >- + Negates a reaction's stoichiometry and bounds to reverse its direction. + Simple enough to do directly against a cobra.Reaction's own API; no + dedicated function exists. + readFasta: >- + raven-toolbox has no public FASTA reader; the closest relative is + write_fasta (already auto-matched from writeFasta), the read side stays + internal to the functions that need it. + runRobustnessAnalysis: >- + Robustness analysis for one reaction against an objective. + raven-toolbox has no equivalent yet. + runSimpleOptKnock: >- + A simple, brute-force OptKnock implementation. raven-toolbox has no + equivalent; strain-design algorithms like OptKnock are explicitly out of + scope for both toolboxes, see RAVEN 3 and raven-toolbox.