Skip to content

Fix silent correctness bugs; improve SIF build tooling and job provenance - #9

Merged
juliendrapeau merged 5 commits into
devfrom
sync
May 11, 2026
Merged

juliendrapeau merged 5 commits into
devfrom
sync

Conversation

@juliendrapeau

Copy link
Copy Markdown
Owner

Summary

  • Fix two silent correctness bugs: global RNG state mutation in sweep config generation, and NaN
    values from empty CSV cells silently corrupting Snakemake rule params
  • Refactor build_sif.py: fast local-apptainer path, SSH safety check, BuildKit cache, remove
    tqdm and third-party importlib_metadata
  • Extend job_utils.py with full SLURM metadata capture and automatic container provenance

Resolves #7.

Changes

Fix global RNG state in sweeper.py

Config.default_metadata was calling random.seed(seed) and np.random.seed(seed) as a side
effect of generating a config. This mutated global state on every Config instantiation, causing
non-obvious test interference and making independent RNG streams impossible. Replaced with
np.random.default_rng().integers(2**32) for seed generation (no global state). Callers needing
reproducibility should use np.random.default_rng(metadata["seed"]) locally.

NaN sanitization in Snakefile

pd.read_csv represents empty CSV cells as NaN (float). Without sanitization, any optional
parameter left blank in the sweep CSV arrives at the Snakemake rule as a truthy float and silently
passes incorrect values downstream. Added _clean() helper that maps NaN → None when building
params_by_path.

build_sif.py refactored

  • Fast path: when apptainer/singularity is installed locally and building for the host
    platform, uses docker buildx build --load + apptainer build docker-daemon:// instead of
    writing an intermediate OCI archive — substantially faster for native builds.
  • SSH safety: _ssh_flag() only passes --ssh default when ssh-add -l confirms keys are
    loaded; previously it was always passed, causing failures on machines without an SSH agent.
  • BuildKit cache: OCI fallback path now uses --cache-from/--cache-to to avoid rebuilding
    unchanged layers.
  • Stdlib metadata: replaced third-party importlib_metadata with stdlib importlib.metadata;
    removed importlib-metadata and tqdm from dev dependencies.

SLURM metadata and container provenance in job_utils.py

capture_job_metadata() now records SLURM env vars (SLURM_JOB_ID, SLURM_JOB_PARTITION,
SLURM_CPUS_PER_TASK, SLURM_MEM_PER_CPU, SLURM_NODELIST, SLURM_TIMELIMIT,
SLURM_ARRAY_TASK_ID) — previously none of this was captured, making it impossible to correlate
results with cluster allocation details after the fact. All fields are None on local runs.

capture_run_metadata() now reads the container path from APPTAINER_CONTAINER/SINGULARITY_CONTAINER
(set automatically when running inside Apptainer) rather than the manually-set CONTAINER_DIGEST
env var, and computes a SHA-256 digest of the SIF file for exact reproducibility.

Version

Bumped to 0.4.0.

Replace random.randint + np.random.seed/random.seed with
np.random.default_rng().integers() for seed generation.
Callers needing reproducibility should use the stored seed with
np.random.default_rng(metadata["seed"]).

Ported from sgcode/dev PR #40.
pd.read_csv represents empty cells as NaN (float). Without
sanitization, optional None params (e.g. degree bounds) arrive
as truthy floats and silently corrupt downstream logic. Added
_clean() helper that maps NaN → None.

Ported from sgcode/dev PR #42.
build_sif.py:
- Add fast path when apptainer is available locally (docker --load +
  apptainer build docker-daemon://) — skips intermediate OCI archive
- Add _ssh_flag() that only passes --ssh default when agent has keys
- Add BuildKit layer cache in the OCI fallback path
- Replace importlib_metadata (third-party) with stdlib importlib.metadata
- Remove tqdm; use simple labelled print() instead
- Remove importlib-metadata dev dependency

job_utils.py:
- capture_job_metadata() now records SLURM_JOB_ID, SLURM_JOB_PARTITION,
  SLURM_CPUS_PER_TASK, SLURM_MEM_PER_CPU, SLURM_NODELIST,
  SLURM_TIMELIMIT, SLURM_ARRAY_TASK_ID (all None on local runs)
- capture_run_metadata() reads container path from APPTAINER_CONTAINER /
  SINGULARITY_CONTAINER env vars instead of manual CONTAINER_DIGEST;
  computes SHA-256 digest of the SIF file; records container_image name
@github-actions

Copy link
Copy Markdown

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
59 55 93% 90% 🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 5e4f574 by action🐍

@juliendrapeau
juliendrapeau merged commit 6740d1f into dev May 11, 2026
2 checks passed
@juliendrapeau
juliendrapeau deleted the sync branch May 11, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add more comprehensive metadata

1 participant