Skip to content

Clean up usage of _is_na, _is_true and _is_false - #736

Draft
grst wants to merge 1 commit into
mainfrom
cleanup-is-na-380
Draft

Clean up usage of _is_na, _is_true and _is_false#736
grst wants to merge 1 commit into
mainfrom
cleanup-is-na-380

Conversation

@grst

@grst grst commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Since #356, the data types in .obsm["airr"] are guaranteed, so downstream
code no longer needs to guard against string representations of NaN,
True and False. Replace those checks with pandas.isnull/notnull,
which is both faster and easier to read. The only remaining occurrences are
in scirpy.io, where input data is sanitized before it is stored.

To make that guarantee hold for arbitrary input, AirrCell.add_chain now
also casts strings to the type declared in the AIRR rearrangement schema.
Previously, building AirrCell objects from data that spelled out booleans
or numbers as strings (e.g. productive="True") produced an
.obsm["airr"] array with union types that pp.index_chains could not
process.

Two follow-ups fell out of this:

  • pl.vdj_usage relied on astype(str) turning missing gene calls into
    the string "nan", which no longer happens with pandas' nullable string
    dtypes. As a result the none segment was silently dropped from the
    plot. Missing values are now replaced with "none" explicitly.
  • Two test fixtures spelled a missing clonotype as the string "NaN" in
    obs. Those now use None.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

Closes #380

  • CHANGELOG.md updated
  • Tests added (For bug fixes or new features)
  • Tutorial updated (if necessary)

Since #356, the data types in `.obsm["airr"]` are guaranteed, so downstream
code no longer needs to guard against string representations of `NaN`,
`True` and `False`. Replace those checks with `pandas.isnull`/`notnull`,
which is both faster and easier to read. The only remaining occurrences are
in `scirpy.io`, where input data is sanitized before it is stored.

To make that guarantee hold for arbitrary input, `AirrCell.add_chain` now
also casts strings to the type declared in the AIRR rearrangement schema.
Previously, building `AirrCell` objects from data that spelled out booleans
or numbers as strings (e.g. `productive="True"`) produced an
`.obsm["airr"]` array with union types that `pp.index_chains` could not
process.

Two follow-ups fell out of this:

* `pl.vdj_usage` relied on `astype(str)` turning missing gene calls into
  the string `"nan"`, which no longer happens with pandas' nullable string
  dtypes. As a result the `none` segment was silently dropped from the
  plot. Missing values are now replaced with `"none"` explicitly.
* Two test fixtures spelled a missing clonotype as the string `"NaN"` in
  `obs`. Those now use `None`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.24%. Comparing base (02ad919) to head (f558cf6).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #736      +/-   ##
==========================================
+ Coverage   78.20%   78.24%   +0.04%     
==========================================
  Files          51       51              
  Lines        4620     4625       +5     
==========================================
+ Hits         3613     3619       +6     
+ Misses       1007     1006       -1     
Files with missing lines Coverage Δ
src/scirpy/io/_datastructures.py 90.00% <100.00%> (ø)
src/scirpy/io/_util.py 97.91% <100.00%> (+0.69%) ⬆️
src/scirpy/ir_dist/__init__.py 92.00% <100.00%> (+0.08%) ⬆️
src/scirpy/pl/_vdj_usage.py 87.50% <100.00%> (+0.99%) ⬆️
src/scirpy/tests/conftest.py 100.00% <ø> (ø)
src/scirpy/tl/_chain_qc.py 98.80% <100.00%> (ø)
src/scirpy/tl/_clonal_expansion.py 96.07% <100.00%> (ø)
src/scirpy/tl/_clonotype_modularity.py 85.98% <100.00%> (ø)
src/scirpy/tl/_diversity.py 89.79% <100.00%> (ø)
src/scirpy/tl/_group_abundance.py 90.62% <100.00%> (ø)
... and 3 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

clean up usage of _is_na, _is_true and _is_false

1 participant