refactor: the registry builds the pipelines; delete the four if ladders - #70
Open
Hendrik-code wants to merge 1 commit into
Open
refactor: the registry builds the pipelines; delete the four if ladders#70Hendrik-code wants to merge 1 commit into
Hendrik-code wants to merge 1 commit into
Conversation
smauglab/transforms/build.py does the dispatch once, for all three GPU pipeline
modes and for the CPU one. The ladders it replaces go:
gpu/transforms.py 714 -> 31 lines
gpu/transforms_list.py 974 -> 117 lines (two ladders in one file)
cpu/transforms.py 412 -> 80 lines
Which class a config key maps to, which parameters it accepts, and where it sits
in the pipeline all come from the registry now, so the four copies cannot drift
again. The temporary aliases the previous commits left in contrast.py go with
them.
Configs move to the sectioned, class-name-keyed format in the same commit,
because the ladders and the old key names are the same thing. All 29 shipped
configs were rewritten by migration/migrate.py rather than by hand; the originals
are kept verbatim as unit_tests/fixtures/legacy_configs/ so the migrator has
something to be tested against.
That this changed nothing is checked, not asserted.
unit_tests/fixtures/legacy_effective_kwargs.json records the class and effective
constructor kwargs of every transform the ladders built, for every config and
both GPU modes, captured from the commit before they were deleted by
instrumenting each __init__. unit_tests/test_builder.py rebuilds each of those
pipelines through the registry and compares. All 27 match.
Three differences are normalised away, and the test says why: the ladders built
one class parameterised by kernel_type/func/invert_image where the builder builds
the leaf; that shared class therefore bound kernel-specific parameters its leaves
do not declare (a Scharr transform carried `sigma`, which only the blur reads);
and p_batch is new and defaults to kornia's own 1.0 either way.
The remaining 21 of the 48 pipelines could not be compared because the ladder
crashed on them: AugTransformsGPURandomOrder did
config.get("RandomChooseXTransforms").get(...), which is an AttributeError for
every config that omits that block -- 21 of 29. They build now. That is a fix,
not a regression, but it does mean those configs have no "before" to check
against.
pipeline.order is wired through build_transforms: registry order by default,
config key order when a config asks for it.
.gitignore's repo-wide *.json would have swallowed the fixtures and the migration
table, so each gets an explicit un-ignore.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #69. The largest PR in the stack, and the one that breaks configs.
The four
ifladders gosmauglab/transforms/build.pydoes the dispatch once, for all three GPU pipeline modes and the CPU one:gpu/transforms.pygpu/transforms_list.py(two ladders)cpu/transforms.pyWhich class a config key maps to, which parameters it accepts, and where it sits in the pipeline all come from the registry now, so the four copies cannot drift again. The temporary aliases #66 left in
contrast.pygo with them.Configs move to the new format in the same commit
The ladders and the old key names are the same thing, so they can't be separated. All 29 shipped configs were rewritten by
migration/migrate.py, not by hand, and the originals are kept verbatim inunit_tests/fixtures/legacy_configs/so the migrator has something to be tested against.migration/is how you bring one forward, and the error messages point at it.How I know it changed nothing
Not asserted — checked.
unit_tests/fixtures/legacy_effective_kwargs.jsonrecords the class and effective constructor kwargs of every transform the ladders built, for every config and both GPU modes. It was captured from the commit before they were deleted, by instrumenting each__init__.unit_tests/test_builder.pyrebuilds each of those pipelines through the registry and compares.All 27 match.
Three differences are normalised away, and the test file says why:
kernel_type/func/invert_image; the builder builds the leaf;sigma, which only the blur reads;p_batchis new and defaults to kornia's own1.0either way.The 21 pipelines with no "before"
Of 48 config×mode combinations, only 27 could be compared. The other 21 crashed in the old ladder:
That's 21 of 29 configs —
AugTransformsGPURandomOrderwas unusable for all of them. They build now, becauseconfig.pipeline_options()always returns a dict. That's a fix, not a regression, but it does mean those configs have no baseline to check against.Also
pipeline.orderis wired throughbuild_transforms: registry order by default, config key order when a config asks for it. Tested both ways..gitignore's repo-wide*.jsonwould have swallowed the fixtures and the migration table, so each gets an explicit un-ignore.Testing
unit_tests/test_builder.py(9 tests / 27 subtests) plus the existing suite: 172 passed, 5 skipped, 503 subtests.ruffandmypyclean.