Repository navigation
Fit optional native Score cuts and Choice weights on calibration - #101
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flat compilation can now opt in to native Score cuts and Choice selection weights with --fit-decision-knobs. Fitting uses cached calibration predictions, five group-preserving held-out folds, bounded positive weights and ordered scale-bounded cuts; conservative ties/failures keep an explicit unchanged result. The default compilation path is unchanged. Raw native probabilities, Noul P(true), vendor confidence and selected-outcome gate provenance remain distinct. Hierarchy rejects this unsupported opt-in before backend construction.
Closes #92. Native fitting decision was approved and recorded on #95/#92; #95 is closed. Existing serialized artifacts retain their payloads and checksums. Schemas, inspect output and docs are updated; no measured gain is claimed.
Validation: independent review passed, 22 focused policy tests; full installed-DSPy-3.3.1 suite 640 passed, zero skipped; whole-tree Ruff passed. Wheel/sdist built, 49 package/starter files and four entrypoints verified; fresh installed-wheel flat/hierarchy demos and synthetic study register/select/test/report passed outside checkout with zero provider calls and no DSPy/GEPA runtime imports. Synthetic fitted artifact roundtrip/raw-probability checks passed.
Retained earlier verification failures: an interrupted mutable-root run; a nested replay identity failure while concurrent build created package metadata. Stable post-build causal test and full suite passed. Initial wheel study attempts mixed incompatible fixtures and correctly rejected; final complete fixture passed. Hosted exact-head checks remain required before merge.