Skip to content

Add cau mudata-builder: the two count matrices as one MuData - #14

Merged
emattei merged 2 commits into
mainfrom
feat/mudata-builder
Oct 15, 2026
Merged

emattei merged 2 commits into
mainfrom
feat/mudata-builder

Conversation

@emattei

@emattei emattei commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

Why

The design steps end at guide_metadata.tsv. Nothing took a sequenced
channel and produced the object an analysis works in, so that step was a
hand-written script every time.

What it does

Assembles one channel's gene and guide counts into a MuData laid out like the
IGVF CRISPR pipeline's own object, so the two can be compared field by field:
modalities gene and guide, a cell index of {barcode}_{measurement_set},
raw counts untouched, and each feature's annotation in var.

cau mudata-builder run \
    --rna counts_unfiltered/adata.h5ad \
    --guide counts_unfiltered_modified/adata.h5ad \
    --guide-metadata results/igvf/guide_metadata.tsv \
    --store gencode.v43.annotation.parquet \
    --measurement-set IGVFDS8842PXCC \
    --gtf-accession IGVFFI9561BASO \
    --outdir results/

What it deliberately does not do

No statistics, no QC, no filtering, no doublet calls, no guide assignment, no
pairs to test. Each is a judgement, and a step that makes judgements while
assembling hides them.

Three decisions worth reviewing

Cells are the union, not the intersection, with obsm recording which
modality holds each. On a real channel that is 1,022,982 barcodes against
207,274 seen by both: an intersection would discard four fifths of the object
before anyone looked at it.

Every gene type is annotated, not only protein-coding. The counts come
from a transcriptome built on the whole GTF: 20,066 protein-coding against
42,691 lncRNA, pseudogenes and small RNAs, which would otherwise be nameless.

One GENCODE, checked. A build whose store differs from the one the guide
metadata was written against is refused, so genes and guides cannot describe
different genomes. uns['reference'] then carries the GTF and genome by name,
md5 and accession, so the object answers "which GENCODE" without its run
record beside it. Only names are stored, never paths: the object is shared,
and a path carries the filesystem it was built on.

Two traps the tests pin

PAR genes appear twice -- 47 features with a _PAR_Y suffix and a chrX twin
of identical sequence -- and are carried through untouched rather than summed
or de-duplicated.

kb indexes genes as ENSG00000290825.1 while the store drops the version.
The join uses the store's own rule, which strips a trailing .N and leaves
_PAR_Y alone; splitting on the first dot instead would hand the chrY copy
the chrX gene's coordinates with nothing complaining.

Also

anndata and mudata become runtime dependencies, imported inside functions
like pysam so cau --help loads neither. Both lockfiles regenerated.

The README and two module docstrings still described location as
exon/intron/intergenic, which stopped being true in #13. Corrected here.

Verified end to end on channel IGVFDS8842PXCC: 62,757 genes and 43,766 guides,
none unannotated on either side.

🤖 Generated with Claude Code

emattei and others added 2 commits October 15, 2026 10:27
The design steps end at guide_metadata.tsv; nothing took a sequenced channel
and produced the object an analysis works in. This assembles one: modalities
gene and guide, a cell index of {barcode}_{measurement_set}, raw counts, and
each feature's annotation in var, laid out like the IGVF pipeline's own
object so the two can be compared field by field.

Assembly only. Nothing is derived from the counts, no cell is dropped, no
guide is called and no pair is proposed. QC and filtering are later steps,
and each is a judgement this one has no business making.

Three decisions worth stating. Cells are the union of the two matrices, not
their intersection, with obsm recording which modality holds each: on a real
channel that is 1,022,982 barcodes against 207,274 seen by both, so an
intersection would discard most of the object before anyone looked at it.
Every gene type is annotated rather than only the coding ones, because the
counts come from a transcriptome built on the whole GTF and two thirds of the
features would otherwise be nameless. And uns['reference'] carries the GTF
and genome by name, md5 and accession, so the object answers "which GENCODE"
without its run record beside it; a build whose store differs from the one
the guide metadata was written against is refused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The docs page covers what the object holds, why cells are the union, why
every gene type is annotated, the two traps that cost real debugging (PAR
genes counted twice, and the version rule that must be the store's own), and
what uns['reference'] records.

The README's annotation line and two module docstrings still said `location`
was exon/intron/intergenic, which stopped being true when the axes were
split. Docs accuracy is a hard rule here, and these were the last three
places that had not caught up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@emattei
emattei merged commit 076c160 into main Oct 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant