feat: add the sectioned config schema - #69
Open
Hendrik-code wants to merge 1 commit into
Open
Conversation
smauglab/config.py, with no consumers yet: nothing reads it until the builder lands, and the shipped configs are untouched by this commit. A config becomes a sectioned document -- GPU, CPU, pipeline, and '_'-prefixed comments -- where a key is a class name exactly and a parameter is a constructor argument exactly, both checked against the registry. A flat, section-less document is rejected rather than migrated. It used to be read as "GPU or CPU, whichever the keys look like", and the two namespaces overlap enough that GaussianBlurTransform meant a different transform depending on which builder read it. Every problem in a file is reported at once, with "did you mean" suggestions. The old behaviour surfaced one per run, which for a 30-key config is 30 edit-run cycles. pipeline.order is the opt-in discussed on the registry commit. The default, `registry`, is registry.PIPELINE_ORDER: fixed, and the same for every config. `config` takes the order the keys appear in the file instead. It is not the default because it makes a pipeline sensitive to something people reasonably treat as cosmetic -- reordering or reformatting a config would silently change what it does -- so opting in makes that intent explicit and greppable. It is independent of pipeline.mode. PipelineMode, OrderSource and validate_section live here rather than in the builder that pr56 put them in. They describe a config document, so config.py depends only on the registry and the builder will import them from here. That is also what lets this land on its own. pipeline.mode replaces encoding the arrangement in the trainer *class*, which baked the choice into the name of every run directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #68.
smauglab/config.py, with no consumers yet — nothing reads it until the builder lands, and the shipped configs are untouched by this PR.A config becomes a sectioned document —
GPU,CPU,pipeline, and_-prefixed comments — where a key is a class name exactly and a parameter is a constructor argument exactly, both checked against the registry.{ "_comment": "ignored", "pipeline": { "mode": "sequential", "order": "registry" }, "GPU": { "RandomFlipTransformGPU": { "flip_axis": [0], "p": 0.5 } }, "CPU": { ... } }A flat config is rejected, not migrated
It used to be read as "GPU or CPU, whichever the keys look like". The two namespaces overlap enough that
GaussianBlurTransformmeant a different transform depending on which builder read it. That guess is gone.Every problem at once
The old behaviour surfaced one problem per run — for a 30-key config, 30 edit-run cycles. Now:
That last one matters:
probability→pis the commonest migration mistake and difflib cannot bridge it (the two strings score ~0.17), so it comes from an explicit hint table that a test keeps diagnostic-only.pipeline.order— the opt-in you asked forregistryisregistry.PIPELINE_ORDER: fixed, and the same for every config.configtakes the order the keys appear in the file instead.It is not the default because it makes a pipeline sensitive to something people reasonably treat as cosmetic — reordering or reformatting a config would silently change what it does. Opting in makes that intent explicit and greppable. It's independent of
pipeline.mode.A layering change from the original PR
PipelineMode,OrderSourceandvalidate_sectionlive here rather than in the builder, where #56 put them. They describe a config document, soconfig.pydepends only on the registry and the builder will import them from here. That's also what lets this PR land on its own rather than being welded to the builder.pipeline.modereplaces encoding the arrangement in the trainer class — which baked the choice into the name of every run directory and could not be varied without a new class.Testing
unit_tests/test_config_schema.py— 22 tests: sections, the flat-config rejection, cumulative problem reporting, the suggestion machinery, andpipeline.orderin both directions.Full suite: 163 passed, 5 skipped, 479 subtests.
ruffandmypyclean.