Repository navigation
Keep ordinal fit labels consistent with their probabilities - #44
Merged
Merged
Conversation
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Live perspective ranking returned an 88/100 fit score with a weak label, incorrectly making the entire response low-confidence. The managed adapter trusted a generated label index that contradicted the returned probabilities. Derive the label from the maximum validated probability, retain the first label on exact ties (the lower ordinal fit), and preserve score normalization, malformed-index rejection and existing retry limits.
This corrects both falsely low and falsely high labels. It adds no model calls, changes no confidence threshold, and keeps genuinely weak public results low-confidence. Compact and earlier object-shaped ordinal answers follow the same rule. The separate serious-input categorical decision and generic perspective contract are preserved.
Validation: 62 focused tests pass, including the exact 88/100 contradiction, conservative ties, opposite-direction false confidence and the real public Worker response. The preceding full 222-test suite and quality/build gates passed; final exact-source CI is required before release, followed by unchanged strict production qualification. Refs #30 and sass-maker/free-ai#95.