Repository navigation
Unify experiment layout; reorganize hpc/config/; add plots and sweeper improvements - #10
Merged
Merged
Conversation
Adds workflow/plots/ package: common.py with use_style(), read_results_csv(), and style constants; four mplstyles (default, one_column, two_column, presentation).
- use_hash=True uses each config's content hash as the directory name instead of a human-readable slug, guaranteeing uniqueness regardless of slugify_structure. - After assigning paths, raises ValueError listing all duplicate short_paths so collisions are caught at sweep-generation time rather than silently overwriting configs. Both options threaded through generate_and_save.
- Move config_example.yaml and submit_example.yaml into hpc/config/. - Add hpc/config/clusters_example.yaml — users copy to clusters.yaml (gitignored) and fill in SSH host → scratch-root mappings. - Replace hardcoded CLUSTER_ROOTS dict in lifecycle.py with _load_cluster_roots() that reads from hpc/config/clusters.yaml, so cluster paths are no longer baked into tracked code. - Update run_snakemake.sh: --profile hpc → --profile hpc/config.
All per-experiment artifacts now live under experiments/<stage>/<exp>/: sweeps/ sweep scripts and summary.csv (tracked example) data/ per-job outputs (gitignored) results/ aggregated CSVs (gitignored) Updated Snakefile, rules, workflow_example.yaml, sweep scripts, docs, template snippets, and .gitignore. Old data/ and results/ root dirs remain gitignored for backward compat.
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified FilesNo covered modified files...
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
data/,results/, andsweeps/underexperiments/<stage>/<exp>/to keep all per-experiment artifacts co-located.hpc/config files intohpc/config/; replace hardcodedCLUSTER_ROOTSdict with aclusters.yamlfile that users copy from a template.workflow/plots/matplotlib style infrastructure (common utilities + 4 mplstyles: default, one-column, two-column, presentation).use_hashoption and duplicate-path detection toConfigSet.slugify_all.Changes
experiments/directory structuredata/,results/, andsweeps/are replaced by a unifiedexperiments/tree:Updated Snakefile, rules,
workflow_example.yaml, docs, and all template snippets. The olddata/andresults/root directories remain gitignored for backward compat.hpc/config/reorganizationhpc/config_example.yamlandhpc/submit_example.yamlmove tohpc/config/. A newhpc/config/clusters_example.yamlreplaces the hardcodedCLUSTER_ROOTSdict inlifecycle.py: users copy it tohpc/config/clusters.yaml(gitignored) and fill in their SSH host aliases and scratch paths. The--profile hpcflag inrun_snakemake.shis updated to--profile hpc/config.workflow/plots/matplotlib style infrastructureNew
workflow/plots/package withcommon.py(use_style,read_results_csv, style constants) and four mplstyles:default,one_column,two_column, andpresentation.ConfigSet.slugify_allimprovementsuse_hash=Trueuses each config's content hash as the directory name instead of a human-readable slug — guarantees uniqueness when slugs would collide.ValueErrorlisting all collisions.generate_and_save.Version
Bumped to 0.5.0.