Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions analysis/2026-10-03_mucoromycotina-mat-campaign.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Mucoromycotina MAT campaign: calls, locus size, synteny, sexP/sexM trees (2026-10-03)

Status: running (HMG-box tree and RAxML-NG check not in yet); figures are drafts.

## Question
Across BFD, LCG and Jena Mucoromycotina genomes: which genomes carry a sexP
(Plus) or sexM (Minus) MAT locus, how large is the locus per genus and
idiomorph, how is gene order kept across genera, and do sexP and sexM form
separate clades in a rooted gene tree?

## Ruling
J. Stajich, 2026-10-03: run Mucoromycotina first; locus size = inner ends of
the flanking genes; root the gene tree with a non-MAT HMG-box outgroup; a
synteny figure of about 20-30 pruned loci.

## Data
- 974 genomes: BFD Mucorales + Umbelopsidales 289 (288 after suppress), LCG 621,
Jena 64. LCG and Jena are held-out sets (no training, curation or classifier
use). Code: v0.6.0 (`run-7c7ed99`).
- Taxon overrides (`db/taxon_overrides.tsv`): confirmed rows are counted under
the likely genus; `hold out` and `unconfirmed` rows are dropped from locus
size and the tree set.

## Method
See `results/2026-10-03_mucoromycotina_mat/NOTE.md` for every step and file.

## Results
- Detection: 865 of 973 genomes called; 911 loci (Plus 482, Minus 425,
undetermined 4). BFD 256/288, LCG 548/621, Jena 61/64. 45 genomes have more
than one locus; 34 are flagged `two_idiomorphs`.
- Locus size (579 loci, 23 genera, polished flanks on both sides). Largest:
Umbelopsis (median Plus 13,063 / Minus 12,256 bp) and Absidia (9,611 / 9,249).
Intermediate: Phycomyces (6,531 / 4,166), Backusella (6,113 / 6,057), Pilaira
(4,726 / 2,968). Most Mucoraceae and Rhizopus: about 1.4-2.6 kb. The flank
pair is not the same in every lineage, so sizes compare one flank interval per
lineage, not one fixed gene set.
- Synteny: 26 loci (Plus and Minus for 13 genera). Figure:
`synteny/mucoromycotina_MAT_synteny.png`; interactive:
`mucoromycotina_MAT_synteny.clinker.html.zst`.
- Full-length gene tree (526 sequences x 526 sites, JTT+F+R8, UFBoot 1000):
- sexP clade: 163 tips, 0 sexM tips inside, UFBoot 40.
- sexM clade: 173 tips, 0 sexP tips inside, UFBoot 93.
- 3 outgroup HMG genes from Mucorales genomes fall inside these clades (2 in
sexP, 1 in sexM); they may be MAT-gene paralogs away from the locus.
- The outgroup is not monophyletic; 132 outgroup tips fall between sexP and
sexM after rooting. This tree does not support or reject sexP + sexM as
sister groups.

## Decision
None yet. Figures are drafts for the curator.

## Open
- HMG-box tree (job 29389343) and RAxML-NG check.
- Compare locus size within one flank pair, or show all with the pair marked?
- May held-out LCG/Jena genomes appear in publication figures? (They are used
here for description only, not for any build or score.)
- LCG called 548 genomes on v0.6.0 versus 536 on 52b3ff9; not yet compared
call by call.
- Identity of the 3 outgroup tips inside the sexP/sexM clades.
131 changes: 131 additions & 0 deletions analysis/2026-10-04_basidiomycota-ascomycota-v060-campaign.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Basidiomycota and Ascomycota campaign on v0.6.0 (2026-10-03/04)

Status: decided (runs complete); R. toruloides HD locus open.

## Question
Can MATPredict v0.6.0 run over every BFD Basidiomycota and Ascomycota genome?
What call rates, run times and failures does it give, and what changed against
the 2026-09-26 Basidiomycota run?

## Ruling
J. Stajich, 2026-10-03: run a regression check, then a pilot per phylum, then
all Basidiomycota, then all Ascomycota in waves, on the `exfab` partition.
2026-10-04: fix the code-26 failure on a branch with a PR and v0.6.1; commit
these results with the Mucoromycotina campaign.

## Data
- Code: frozen worktree `run-7c7ed99`. Its `src/` and `db/` are identical to
tag `v0.6.0`.
- Genomes: BFD `samples.csv`, every row of the phylum with a taxid and a genome
file. Routing comes from the taxid (lineage routing).
- Partition `exfab`. Accounts: `stajichlab` (384 CPUs, 1 TB shared with all
other jobs of the user) and `exfab` (32 CPUs, 256 GB).

## Method
1. Regression: the fixed panel (166 genomes) plus Zygo 23 on v0.6.0, diffed
against the last checked candidate (2026-10-01 scope-only run).
2. Pilots: 163 Basidiomycota genomes, stratified by order (57 orders, >= 2
each); 246 Ascomycota genomes, stratified by class (30 classes, 92 orders).
`random.seed(20261003)`.
3. Full runs: Basidiomycota as 3 jobs of 64 CPUs (genomes < 500 Mb) and 1 job
of 32 CPUs (58 genomes >= 500 Mb, 4 h per-genome limit). Ascomycota as 9
waves of about 2,157 genomes, 64 CPUs each, 1 h per-genome limit.

## Results

### Regression
0 changed loci in all five panels. Zygo 23: 23/23 on scaffold and contig
inputs. `results/2026-10-03_regression_v060/diff/summary.md`.

### Basidiomycota (3,275 listed)
- 3,270 reports; 5 on the suppress list; 0 failures; 0 timeouts.
- Wall time: 3 jobs of about 1 h, 1 job of 32 min. Median per genome 101 s
(< 50 Mb) to 716 s (> 500 Mb).

| Subphylum | Genomes | Called v0.6.0 | Without PR-only genomes | Called 2026-09-26 |
|---|---|---|---|---|
| Agaricomycotina | 2,417 | 2,231 (92.3%) | 2,021 (83.6%) | 2,007 (83.0%) |
| Pucciniomycotina | 486 | 391 (80.5%) | 371 (76.3%) | 64 (13.2%) |
| Ustilaginomycotina | 315 | 308 (97.8%) | 301 (95.6%) | 301 (95.6%) |
| Wallemiomycotina | 51 | 51 (100%) | 51 (100%) | 0 |

- Gains come from the records curated since 2026-09-26: `redHD`/`redPR`
(Sporidiobolales, about 246 genomes), `rustHD` (107), `wallMAT` (51).
- `Basidiomycota:PR` is called in 1,111 more genomes. 1,360 PR calls carry
`verification: unverified` (admitted only through the strict-CAAX scan). In
phylum-fallback orders most calls are PR only: Sebacinales 12/12 PR-only,
Cantharellales 73/90, Trichosporonales 50/60, Auriculariales 18/19. Do not
count these as MAT loci without the CAAX caveat.
- Apparent losses are mostly merges of one interval into one call: HD into
Aalpha 96, HD into redHD 15, Schizophyllum Balpha + Bbeta into one B call 40.
- 26 Sporidiobolales (R. toruloides) genomes lost a generic-HD call (HD1/HD2,
often with MIP1, on the PR contig, about 200 kb from PR). v0.6.0 calls
`redHD` on a different contig, without MIP1. Which one is the MAT-linked HD
pair is open (see open questions).
- 2 genomes lost every call: GCA_030280995.1 (Pucciniales, bLocus) and
GCA_054785265.1 (Kriegeriales, HD).
- Tables: `results/2026-10-03_basidiomycota_v060/genomes.tsv`, `loci.tsv`,
`by_order.tsv`, `by_order_nonPR.tsv`, `misses_by_order.tsv`, `size_bins.tsv`.

### Ascomycota (19,415 listed)
- 19,380 reports; 15 on the suppress list; 20 failures. All 20 are Alaninales
(genetic code 26): exonerate has no table 26 and exits 1, which ended the
genome. Fixed in v0.6.1 (PR #13). Re-run of the 20 on v0.6.1
(`results/2026-10-04_alaninales_v061/`): 20/20 reports, 0 failures, 30-35 s
each, miniprot-only models. 4 called (P. tannophilus GCA_001661245.1: MATsc
MATa, MTL alpha, MAT MAT1-1; 3 others one MATsc call each), 16 uncalled. All
route `phylum_fallback`: no Alaninales record exists (reference gap).
Curator (2026-10-04): miniprot models genes less well than exonerate, so pass
exonerate the NCBI table as a 64-letter string (accepted by stock exonerate
2.4.0) instead of skipping it. Re-run on that code (`run-b04a77b`,
`results/2026-10-04_alaninales_v061_exostring/`): 20/20 reports, **15/20
called** (miniprot-only 4/20); median 457 s per genome (34 s). Regression vs
v0.6.0: 0 changed loci (`results/2026-10-04_regression_v061s/`). Family labels
under phylum fallback are not reliable (MATsc and MTL in one genome).
- 17,314 of 19,380 genomes have >= 1 call (89.3%). 848 CPU-h; median 162 s per
genome; maximum 3,095 s.

| Class | Genomes | Called | Routing |
|---|---|---|---|
| Sordariomycetes | 6,017 | 93.0% | lineage |
| Eurotiomycetes | 3,221 | 96.5% | lineage |
| Pichiomycetes | 2,967 | 89.9% | lineage 2,367 / fallback 575 |
| Dothideomycetes | 2,731 | 89.4% | lineage |
| Saccharomycetes | 2,648 | 85.1% | lineage 2,281 / fallback 364 |
| Lecanoromycetes | 479 | 86.2% | lineage |
| Leotiomycetes | 435 | 92.2% | lineage |
| Dipodascomycetes | 398 | 28.7% | mostly fallback |
| Pezizomycetes | 172 | 79.7% | lineage |
| Orbiliomycetes | 67 | 9.0% | fallback |
| Lipomycetes | 59 | 94.9% | fallback |
| Taphrinomycetes | 22 | 22.7% | lineage |

- 21,011 loci: Ascomycota:MAT 12,628; MATsc 4,954 (includes HML/HMR silent
copies); MTL 2,799; MATtub 200; PM 138; MATyl 123; mat1 100; mat2 69.
- Confidence: high 13,740, medium 7,112, low 159. Strict pass 20,695, relaxed
316. `homothallic_candidate` loci: 357.
- Ascomycota:MAT genomes: MAT1-1 only 6,269; MAT1-2 only 5,692; both 204.
- Tables: `results/2026-10-03_ascomycota_v060/genomes.tsv`, `loci.tsv`,
`by_class.tsv`.

## Decision
Results kept as the v0.6.0 baseline for both phyla. Code-26 fix shipped as
v0.6.1 (PR #13).

## Open
- R. toruloides HD: old generic-HD locus (PR contig) versus v0.6.0 `redHD`
locus (other contig). Examples: GCA_000222205.2 (AEVR02000017.1:693,823-734,526
old; AEVR02000008.1:1,252,021-1,268,991 new), GCA_001542265.1,
GCA_000258745.1. Curator ruling pending.
- Classes with low call rates under phylum fallback (Dipodascomycetes,
Orbiliomycetes, Trigonopsidomycetes) are reference gaps, not measured
absence.
- The genome with 11 Ascomycota loci has not been examined.

## Files
- `results/2026-10-03_regression_v060/`
- `results/2026-10-03_basidiomycota_pilot/`, `results/2026-10-03_ascomycota_pilot/`
- `results/2026-10-03_basidiomycota_v060/`, `results/2026-10-03_ascomycota_v060/`
(`reports_all.tar.zst` = every `detection_report.yaml` and `wall_seconds`;
per-genome GFF3 and FASTA are not committed)
- `results/2026-10-04_alaninales_v061/`
5 changes: 4 additions & 1 deletion analysis/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,17 @@ in yet; superseded = replaced by a later study. `results/` = main checkout.
| 09-29 | [Classifier builds: aligner, determinism, gate, P1](2026-09-29_classifier-builds.md) | decided; shipped 9c39e2c | no aligner more accurate; mafft --auto = L-INS-i; ClipKIT 16 wrong; builds byte-identical; P1 withholds 26, reveals 5 |
| 09-29 | [sexM-like paralogs, strain labels, Absidia](2026-09-29_paralogs-and-strain-labels.md) | decided | labels 10/7/4 (n=21); P1 in both mating types; Absidia flanks off-scaffold |
| 09-29 | [Two idiomorphs and homothallism](2026-09-29_two-idiomorphs-and-homothallism.md) | decided (report-only); causes open | 36 unlinked; literature signal 13/16 vs 5/9; Syzygites both idiomorphs |
| 10-03 | [Mucoromycotina MAT campaign](2026-10-03_mucoromycotina-mat-campaign.md) | running (HMG tree) | 865/973 genomes, 911 loci; locus size 579 loci/23 genera; full-length tree sexP 0 sexM inside (UFBoot 40), sexM 0 sexP inside (UFBoot 93) |

## Basidiomycota
| Date | Report | Status | Key numbers |
| 10-01 | [Branch merges and scope-only families](2026-10-01_basidio-merges-and-scope-only.md) | decided | net 14 gained in-lineage, 2 CAAX-only PR lost, 0 label changes |
|---|---|---|---|
| 10-01 | [Branch merges and scope-only families](2026-10-01_basidio-merges-and-scope-only.md) | decided | net 14 gained in-lineage, 2 CAAX-only PR lost, 0 label changes |
| 09-26 | [Anchors, full run, cap-off](2026-09-26_basidiomycota.md) | decided | 3,269 genomes; Agaricomycotina 83.0% |
| 09-26 | [Cryptococcus SXI slot](2026-09-26_cryptococcus-sxi.md) | decided | 243 genomes, 0 lost, 53 gained |
| 09-26 | [Pucciniomycotina, Wallemia, Rhodotorula](2026-09-26_pucciniomycotina-wallemia-rhodotorula.md) | decided | Wallemia 51/51; Rhodotorula P/R 61/62 held-out |
| 09-27 | [Receptor (B/PR) loci](2026-09-27_receptor-loci.md) | decided (review later) | CAAX: 6/9 vs 0/25; Agaricales PR 14->132 |
| 10-04 | [Basidiomycota + Ascomycota v0.6.0 campaign](2026-10-04_basidiomycota-ascomycota-v060-campaign.md) | decided; R. toruloides HD open | Basidio 2,982/3,270 (2,745 without PR-only); Asco 17,314/19,380; 20 code-26 failures -> v0.6.1 |

## Ascomycota (incl. Serinales, Dothideomycetes)
| Date | Report | Status | Key numbers |
Expand All @@ -33,6 +35,7 @@ in yet; superseded = replaced by a later study. `results/` = main checkout.
| 09-26 | [Serinales, C. albicans, C. auris](2026-09-26_serinales-candida.md) | decided | C. albicans 8/11 collapsed |
| 09-26 | [Dothideomycetes curation](2026-09-26_dothideomycetes.md) | decided | 90/99; C. kikuchii homothallism candidate |
| 09-27 | [Confidence rules](2026-09-27_confidence-rules.md) | decided | 86 risers; no fallback floor |
| 10-04 | [Basidiomycota + Ascomycota v0.6.0 campaign](2026-10-04_basidiomycota-ascomycota-v060-campaign.md) | decided | 19,415 genomes; 89.3% called; Dipodascomycetes 28.7%, Orbiliomycetes 9.0% (fallback) |

## Validation and held-out sets
| Date | Report | Status | Key numbers |
Expand Down
10 changes: 10 additions & 0 deletions analysis/open-questions.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,13 @@ not match the genome's name:
| Item | Next step | Where |
|---|---|---|
| S. pombe P and Yarrowia A/B idiomorph class (unassigned) | protein-domain evidence (Pc vs alpha box; MATA/MATB proteins) | `db/Ascomycota/order.yml` (B9) |

## Opened 2026-10-04 (campaign runs)

| Item | Next step | Where |
|---|---|---|
| R. toruloides HD: 2026-09-26 generic-HD locus (PR contig, ~200 kb from PR, with MIP1) vs v0.6.0 `redHD` locus (other contig, no MIP1); 26 genomes | curator ruling on which is the MAT-linked HD pair | `2026-10-04_basidiomycota-ascomycota-v060-campaign.md` |
| Basidiomycota PR calls: 1,360 unverified (strict-CAAX only); most phylum-fallback calls are PR only | keep the caveat in any summary; review with the receptor-loci study | same |
| Mucoromycotina locus size: flank pair differs by lineage | compare within one pair, or plot all with the pair marked | `2026-10-03_mucoromycotina-mat-campaign.md` |
| Held-out LCG/Jena genomes in publication figures | curator ruling | same |
| 3 outgroup HMG genes inside the sexP/sexM clades (Dicele1 h5, GCA_016758965.1 h6, Mycafr1 h1) | check locus and identity | same |
7 changes: 7 additions & 0 deletions results/2026-10-03_ascomycota_pilot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Ascomycota pilot on v0.6.0 (2026-10-03)

246 BFD Ascomycota genomes, stratified by CLASS (30 classes, 92 orders): about 200 x class
share, at least 3 per class, round-robin over orders inside a class (random.seed 20261003).
Lists: pilot_list.tsv, pilot_meta.tsv. Code: frozen worktree run-7c7ed99 (= v0.6.0).
Partition exfab, 32-way, GENOME_TIMEOUT 3600 (no genome > 500 Mb in the pick).
Purpose: per-genome runtime and call rate per class before the 19,415-genome run.
1 change: 1 addition & 0 deletions results/2026-10-03_ascomycota_pilot/jobs.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
29387476 pilot 246
31 changes: 31 additions & 0 deletions results/2026-10-03_ascomycota_pilot/pilot_by_stratum.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
stratum genomes reports called routing wall_median_s wall_p90_s wall_max_s
Sordariomycetes 62 62 52 lineage:62 159 213 277
Eurotiomycetes 33 33 29 lineage:33 156 211 466
Pichiomycetes 31 21 13 lineage:11; phylum_fallback:10 52 324 425
Dothideomycetes 28 27 21 lineage:27 112 184 229
Saccharomycetes 27 27 18 phylum_fallback:20; lineage:7 219 345 494
Lecanoromycetes 5 5 4 lineage:5 139 164 164
Dipodascomycetes 4 4 2 phylum_fallback:2; direct:2 139 358 358
Leotiomycetes 4 4 4 lineage:4 197 219 219
Alloascoideomycetes 3 3 3 phylum_fallback:3 181 311 311
Arthoniomycetes 3 3 2 lineage:3 103 170 170
Candelariomycetes 3 3 3 lineage:3 150 238 238
Geoglossomycetes 3 3 2 lineage:3 171 183 183
Lichinomycetes 3 3 3 lineage:3 170 186 186
Lipomycetes 3 3 3 phylum_fallback:3 549 581 581
NA 3 3 3 phylum_fallback:2; lineage:1 317 624 624
Orbiliomycetes 3 3 0 phylum_fallback:3 175 203 203
Pezizomycetes 3 3 2 lineage:3 33 50 50
Pneumocystomycetes 3 3 3 lineage:3 5 5 5
Schizosaccharomycetes 3 3 3 direct:3 11 34 34
Sporopachydermiomycetes 3 3 3 phylum_fallback:3 278 303 303
Taphrinomycetes 3 3 1 lineage:3 5 5 5
Trigonopsidomycetes 3 3 0 phylum_fallback:3 250 322 322
Coniocybomycetes 2 2 2 lineage:2 124 132 132
Saitoellomycetes 2 2 0 lineage:2 7 8 8
Laboulbeniomycetes 1 1 0 lineage:1 52 52 52
Neolectomycetes 1 1 0 lineage:1 5 5 5
Novakomycetes 1 1 0 lineage:1 3 3 3
Sareomycetes 1 1 1 lineage:1 171 171 171
Xylobotryomycetes 1 1 1 lineage:1 83 83 83
Xylonomycetes 1 1 1 lineage:1 123 123 123
Loading