Skip to content

Support non-perturbed scRNA-seq and mouse data - #17

Closed
OlgaPushkarev wants to merge 5 commits into
mainfrom
nonperturbed-mouse-support
Closed

OlgaPushkarev wants to merge 5 commits into
mainfrom
nonperturbed-mouse-support

Conversation

@OlgaPushkarev

Copy link
Copy Markdown

Summary

Lets the pipeline run on scRNA-seq data without perturbations (no guide information) and on mouse data. Human perturbation runs are unchanged.

Non-perturbed data (no guides)

  • K-selection plots: new --no_perturbation and --no_trait flags skip the perturbation and trait inputs and their plots. The regulator row shared by both panel functions is moved into one helper and hidden when there is no perturbation data.
  • Program analysis: a missing uns['guide_targets'] is handled; perturbed-gene annotations are left empty, with a message.
  • Excel summary: the Targets Summary sheet (and the specificity scores it writes) is skipped when there are no guides. The gene-symbol lookup falls back to var_names when the --gene_names_key column is absent, as compute_kd_efficiency already does.
  • Evaluation: --Perform_perturbation without guide information now fails early with a clear message.

Mouse data

  • Curated ortholog mapping (new src/Stage2_Evaluation/A_Metrics/src/orthologs.py): mouse genes are matched to the human Enrichr libraries and to human gene summaries through orthologs (the union of MGI HOM_MouseHumanSequence and Ensembl BioMart homologs), instead of upper-casing symbols. Upper-casing misses orthologs with different names (e.g. Zfp451 → ZNF451, mouse H2-* → human HLA-*) and sometimes picks a non-orthologous human gene. Details:
    • The table is downloaded once and cached in src/Stage2_Evaluation/Resources/; --ortholog_table pins a fixed copy.
    • Mouse genes without a human ortholog are excluded from enrichment.
    • A gene with several human orthologs contributes to all of them; a human gene with several mouse orthologs keeps the highest loading.
    • MSigDB mouse gene sets keep mouse symbols.
    • organism values other than human/mouse now raise a clear error.
  • Excel summary: --organism mouse annotates each gene with its human orthologs' MyGene summaries and a new Human ortholog column.
  • Evaluation: trait (GWAS) enrichment is human-only (OpenTargets) and is skipped with a message for other organisms.
  • Plotting utilities: the MyGene helpers now honor their species argument.

Other changes

  • ensure_umap reuses embeddings stored as obsm['UMAP'] / obsm['PCA'] (e.g. Seurat conversions) instead of computing a different UMAP.
  • A one-time ruff format of all tracked .py files, as a separate format-only commit, listed in .git-blame-ignore-revs so git blame skips it. Reviewing commit by commit (or git diff -w) is easiest: the feature changes are in the last two commits.
  • A study-specific example removed from a --cell_type help string.
  • README section "Non-perturbed and mouse data" (per-stage flags), plus parameter-table entries for the new options.

Testing

  • New tests (synthetic data, offline):

    • tests/Script/test_non_perturbed.py (7)
    • tests/Script/Stage2_Evaluation/test_orthologs.py (9)
    • tests/Script/Stage3_Interpretation/A_Plotting/Program/test_ensure_umap.py (3)

    All pass. Against the previous code, the tests for new behavior fail and the tests for unchanged behavior pass.

  • Regression: the existing Stage 2 metrics and Stage 3 suites (146 tests) give identical per-test outcomes before and after, on the same torch-cNMF test inference output. On human test data, the evaluation outputs match the previous code exactly (same rows and p-values), except gene order within the Genes column and last-digit rounding of Combined Score.

  • End to end: a 4,000-cell subsample of a public mouse endothelial scRNA-seq dataset with no perturbations ran through inference, evaluation, K-selection, program analysis and the Excel summary. With ortholog mapping:

    • An MHC class II program (top genes Cd74, H2-Ab1, H2-Aa, H2-Eb1, Ciita) is correctly annotated as antigen presentation. Upper-casing had dropped its H2-* genes and returned "regulation of cell migration".
    • 90% of the top program genes get a gene summary.

Pre-existing issues (not changed here)

  • 6 Stage 3 tests and 1 torch inference test already fail on main because they are stale (load_perturbation_data(samples=...), plot_top_program_per_gene error expectations, and inference output files that are no longer written).
  • gseapy's Biomart client currently fails ("Method Not Allowed") because www.ensembl.org redirects. get_idconversion uses it for Ensembl-ID-to-symbol conversion and falls back silently. The new ortholog code queries an Ensembl mirror directly and is not affected.
  • tools/check_no_lab_specific_content.py reports pre-existing hits on main (mostly user and study paths in .sh runners).

🤖 Generated with Claude Code

OlgaPushkarev and others added 5 commits October 8, 2026 17:08
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Applies 'ruff format' (default black-compatible style, line length 88) to
every tracked .py file so later changes have small, reviewable diffs.
Notebooks are untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Non-perturbed data (no guide information):
- K-selection: new --no_trait / --no_perturbation flags skip the trait and
  perturbation inputs and plots; the shared regulator row is factored into
  _plot_regulator_row and hidden when there is no perturbation data.
- Program analysis: tolerate a missing uns['guide_targets'].
- Excel summary: skip the Targets Summary sheet (and the specificity
  scores it writes) when there are no guides; fall back to var_names when
  the gene symbol column is absent.
- Evaluation: fail early with a clear message when --Perform_perturbation
  is requested without guide information.

Mouse data:
- Mouse genes are matched to human gene-set libraries and gene summaries
  through curated orthologs (union of MGI and Ensembl, new orthologs.py)
  instead of upper-casing symbols. The table is downloaded once and cached
  in src/Stage2_Evaluation/Resources/; --ortholog_table pins a fixed copy.
  Genes without a human ortholog are excluded from enrichment; one-to-many
  orthologs keep all human genes. MSigDB mouse gene sets keep mouse symbols.
- Excel summary: --organism mouse annotates genes with their human
  orthologs' MyGene summaries and a 'Human ortholog' column.
- Evaluation: trait (GWAS) enrichment is human-only and is skipped with a
  message for other organisms.
- Plotting utilities: honor the species argument of the MyGene helpers.

Human runs are unchanged: existing Stage 2/3 tests give identical
per-test outcomes before and after, and human evaluation outputs match
apart from gene order within the Genes column and last-digit rounding.

Tests: tests/Script/test_non_perturbed.py and
tests/Script/Stage2_Evaluation/test_orthologs.py (synthetic data, offline).
Docs: README section on non-perturbed and mouse data, parameter tables.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Data converted from Seurat stores its embeddings as obsm['UMAP'] /
obsm['PCA'] (often as DataFrames), which scanpy does not see, so program
and perturbed-gene plots recomputed a different UMAP. ensure_umap now
copies such embeddings to 'X_umap' (first two columns) / 'X_pca' before
deciding whether to compute one. Existing X_umap/X_pca entries and the
original keys are left untouched; files without these keys behave as
before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@OlgaPushkarev

Copy link
Copy Markdown
Author

Superseded by #18 (lab-content cleanup, so CI passes), #19 (ruff format, format-only) and #20 (non-perturbed and mouse support, including the changes from this PR plus review fixes). Please merge those in order.

@OlgaPushkarev
OlgaPushkarev deleted the nonperturbed-mouse-support branch October 10, 2026 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant