This repository was archived by the owner on Sep 24, 2026. It is now read-only.
Repository navigation
docs: split scanpy/api/performance into per-topic directories, and clean up outdated docs - #561
Merged
Merged
Conversation
…tories
The three largest docs (4.7k, 3.9k and 5.5k lines) are now directories of
per-topic pages, each with a README index and a quick start:
- docs/scanpy/ 20 pages: loading, backed mode, lazy preprocessing,
conversion, external annotations, file operations, and
one page per accelerator family.
- docs/api/ 20 pages: one per Python surface, per Rust crate, the
conversion references, and the CLI.
- docs/performance/ 18 pages: storage, conversion, memory, CPU/GPU
accelerators, the training loader, cloud, comparisons.
Section text moved verbatim (headings promoted, relative links rewritten), so
every existing section anchor survives on its new page. docs/scanpy.md,
docs/api.md and docs/performance.md stay as redirect stubs mapping each old
section to its new page, and are excluded from the Sphinx build.
Inbound references are retargeted to the specific page: markdown links across
the repo, code comments, docstrings and two runtime error strings. The tests
that parse these docs now read the pages that hold the parsed content
(test_accel_doc_signatures -> api/python-accel.md, test_experiment_stub_coverage
-> api/python-experiment.md and scanpy/loading.md, test_doublet_import ->
scanpy/external-annotations.md, test_floor_reachability -> performance/
conversion.md, test_docs_anchors' D1 referrers).
docs/index.md gets one toctree per directory; myst_heading_anchors goes to 5
so scanpy/loading.md's H5 "Filter Expression Compatibility" section, linked
from several pages, has an anchor on the rendered site.
Also removes ROADMAP.md and the references to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RwRKstgr4TNkXhLhkQwmjA
- skills/scx-usage: SparseCellSetDataset, obs/var/cellbender import, the
nbglm / hvg / pydeseq2 extras, --force on destination-writing subcommands,
in-place build-csc, 0.19.0 wheel names, and the prefer_format="auto" /
cpu_csc_nnz DE routing.
- .claude/skills/scx-dev: current version examples, the hdf5 test commands in
the pre-release checklist, and the editable-install version check.
- .claude/skills/scx-user: --force on re-runs, the csc="auto" ingest default,
and checking adata.uns["scx_accel"] routes.
- docs/{architecture,format,migrating-from-h5ad,operations}.md: conversion
builds the CSC sidecar by default (csc="auto"), and build-csc appends in
place and is rollback-able.
- README.md: Harmony2 line wording.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RwRKstgr4TNkXhLhkQwmjA
…hey described Removes stale material from the docs and the comprehensive benchmark harness: out-of-date performance write-ups and comparison tables, the harness code, report sections, gates and env entries that only existed to produce them, and the matching rows in the recorded baselines. - The benchmarks that remain (grouped_read, ooc_rss_boundary, ml_loader) keep every SCX arm and every other format unchanged. - Baseline snapshots drop the retired rows; each MANIFEST.sha256 is recomputed and verified, one snapshot directory is renamed to v0.11.0-recapture, and the baselines README row counts are updated. - Docs keep SCX's own numbers wherever they stand alone; links, anchors and the Sphinx toctree are updated for the removed page. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RwRKstgr4TNkXhLhkQwmjA
Dropping ROADMAP.md left `for top in ["AGENTS.md"]`, which clippy's single_element_loop rejects under `--all-targets -D warnings`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RwRKstgr4TNkXhLhkQwmjA
There was a problem hiding this comment.
Code Review
This pull request restructures the project's documentation by splitting the monolithic api.md, scanpy.md, and performance.md files into modular, multi-page directories (docs/api/, docs/scanpy/, and docs/performance/) with dedicated index files, updating all internal and external cross-references. Additionally, it completely retires the third-party comparison formats shardad and cellstream by removing their runners, dependencies, benchmarks, and baseline entries from the comprehensive benchmark suite. Minor updates are also introduced to pre-release checklists, CLI/Python documentation notes, and skills files. I have no feedback to provide as there are no review comments to assess.
nick-youngblut
added a commit
that referenced
this pull request
Sep 24, 2026
All 16 versioned workspace members, `pyscx/pyproject.toml` and `rscx/DESCRIPTION`; `tests/scx-integration-tests` stays at `0.0.0`. README's `scx-cli` download snippet follows. (ROADMAP.md no longer exists, so there is no date stamp to move.) 0.20.0 collects the merged PRs since 0.19.0 (#544-#561): - CSC sidecar series: one-pass bucketed CSC builder (#555); `build-csc` as an in-place, rollback-able append (#556); same-pass CSC build on ingest and carry-through on rewrite ops (#557); parallel CSC build and `csc="auto"` as the ingest default (#558); CSC dispatch for row-indexed transforms and row-filtered / gene-subset handles (#553, #554); CSC route coverage and the exact-nnz Wilcoxon kernel as the 1-vs-rest CSC default (#559) - Correctness: Leiden parallel local moving keeps decliners eligible (#544); five CPU-accelerator parity / convergence fixes (#548); GPU cuVS stream sync, reduction determinism, decoder bounds and VRAM accounting (#549); CLI destination-overwrite safety and MTX multimodal / streaming / integer integrity (#550); unscoped whole-matrix reads of a multimodal file refused (#551); dictionary (categorical) output from append / merge / merge --sort-by (#546) and `scx sort`'s obs spill path (#547) - Benchmarks and docs: four community analytical benchmarks (#552), laptop-test recapture (#560), README refresh (#545), docs split into per-topic directories (#561) Behaviour changes worth calling out in the release notes: - **`csc="auto"` is the ingest default** on every entry point and preset (`from_mudata` keeps `off`): files with `n_obs >= 50000` and `n_vars >= 5000` now get a CSC sidecar, costing ingest wall time and +42-71 % on disk. `--csc off` / `csc="off"` opts out (#558). - **Rewrite ops carry the CSC sidecar by default** (`compact`, `merge`, `optimize`, `sort`, `subset`), rebuilt from the output's own shards; `--csc carry|always|off`. `--rebuild-csc` / `rebuild_csc=True` are deprecated aliases for `always` (#557). - **`scx build-csc` appends in place** rather than rewriting the file, and `scx rollback` removes the sidecar (#556). - **Ten CLI subcommands no longer silently overwrite an existing destination**; `--force` is required (convert, merge, subset, query --output, upgrade, cloud-optimize, explode, pack, pull; compact / optimize / sort migrated onto the same guard) and is refused where nothing is written (#550). - **MTX**: a declared-`integer` MTX with a value past 2^24, a non-integral value or `nan`/`inf` is refused on ingest (`--allow-lossy` restores the old behaviour); multimodal MTX export requires a modality (`pyscx.to_mtx` gains `modality=`); export streams one shard at a time (#550). - **An unscoped whole-matrix read of a multimodal file raises** `MultimodalRequiresModality` instead of folding every modality into one `n_obs x n_modalities`-row answer — `to_anndata()`, `to_memory()`, `read_all_csr_shards*`, `scx pull --filter` (#551). - **Numerical changes**: UMAP init scale and `random_init` now match umap-learn, the kNN sigma search, and NB-GLM dispersion shrinkage (the `above_min_disp` residual filter always runs; new `disp_outlier_sd=2.0` carve-out, `None` disables only the carve-out) (#548); parallel Leiden labels change (#544). - **Wilcoxon on the CPU CSC route** defaults to the exact-nnz kernel (route id `cpu_csc_nnz`); `SCX_ACCEL_WILCOXON_NNZ=0` restores densify (`cpu_csc`), and `reference=` / `rankby_abs=True` still use densify (#559). - **Categorical obs columns** keep their declared categories, unused levels and `ordered` bit through append / merge / sort (#546, #547). No benchmark recapture for this release. Pre-release gate on this tree: `cargo fmt --check`; `cargo clippy --workspace --exclude rscx --all-targets -D warnings`; `cargo test --workspace --exclude rscx` (4,220 passed, 0 failed); `cargo test -p scx-convert --features hdf5` (474 passed); `cargo test -p scx-cli --features hdf5 -- --test-threads=1` (253 passed); `maturin develop --release` + `pytest tests/` (3,247 passed, 140 skipped, 2 xfailed; `pyscx.__version__ == "0.20.0"`). Not run: rscx tests, `--features cloud`, GPU pytest, fuzzing, benchmarks. Claude-Session: https://claude.ai/code/session_01RwRKstgr4TNkXhLhkQwmjA Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three commits:
docs/scanpy.md(4.7k lines),docs/api.md(3.9k) anddocs/performance.md(5.5k) becomedocs/scanpy/(20 pages),docs/api/(20 pages) anddocs/performance/(18 pages). Each has a README index and a quick start, and the pages cross-link. Section text moved verbatim (headings promoted, relative links rewritten), so every existing section anchor still resolves on its new page. The old file paths remain as short redirect stubs that map each old section to its new page; they are excluded from the Sphinx build.scx-gpu/src/shard_validate.rs,pyscx/src/convert/to_anndata.rs).test_accel_doc_signatures,test_experiment_stub_coverage,test_doublet_import,test_floor_reachability(the two pinned table checks), andtest_docs_anchors(the D1 referrers).docs/index.mdgets one toctree per directory.myst_heading_anchorsgoes from 4 to 5 so the H5 "Filter Expression Compatibility" section, linked from several pages, gets an anchor on the rendered site.ROADMAP.mdand the references to it.scx-usage,scx-devandscx-userskills, plus thecsc="auto"default and in-placebuild-cscwording inarchitecture.md,format.md,migrating-from-h5ad.mdandoperations.md.MANIFEST.sha256is recomputed and verified. One snapshot directory is renamed tov0.11.0-recapture, and the baselines README row counts are updated.Verification
test_docs_anchors).benchmarks/comprehensive/tests+pyscx/tests/test_docs_anchors.py: 541 passed, 2 skipped.test_floor_reachability.pytable pins,cargo fmt --all --check, andcargo clippy --workspace --exclude rscx -- -D warnings.python_api.rstexcluded, because its autodoc does not import in the local env) shows no toctree or anchor warnings on the new pages. The remaining warnings are links out ofdocs/intobenchmarks/andskills/, a kind the build already reports for existing pages. The RTD build itself was not run.Notes for review
--no-verify. The pre-commit hook runsgit add -u, which would have swept unrelated working-tree edits into it.cargo fmt --checkandcargo clippy -D warningswere run by hand and are clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01RwRKstgr4TNkXhLhkQwmjA