Repository navigation
A literal value reaches the adjudicator and the review card instead of a question mark - #1220
Merged
Merged
Conversation
…f a question mark A fact whose object is a value (a title, an employee number, an amount) has no entity on the other end, so its line read "phrase → ?". The adjudicator and the review card both lost it; on the identity bench about half of all live facts are values. The line now shows the value, cut to 120 characters. The two adjudicator rows of the scripted namesake cases (#1202) that measured this leave the known-failure list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Wayland Yang <wayland0916@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Wayland Yang <wayland0916@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A fact whose object is a value (a title, an employee number, an amount) has no entity on the other end, so
entity_fact_lines_manyrendered it asphrase → ?. The adjudicator's request and the review card both read these lines. On the identity bench about 52% of live facts are values, so half of what the adjudicator was shown about each record was a question mark. The scripted namesake cases from #1202 named this as a known failure (adjudicator_literal_clues_reach_request).What changes
adjudicator_literal_clues_reach_requestrows leaveKNOWN_FAILURES; the resolver row stays.Measured
Identity bench, both builds in one session, eight bases a side (deepseek-v4-flash through OpenRouter, reasoning off, bge-m3):
The F1 gap comes from two bases and sits inside the spread. No namesake mixing happened that day on either side, so the bench could not show the benefit either.
Controlled re-adjudication: four bases built once with batch adjudication held back (71 queued pairs), restored four times and adjudicated by each build twice. On the 54 pairs every run decided (one baseline batch failed three times on a truncated model reply):
The change moves verdicts no more than rerunning the same build does. The adjudicator itself flips about 15% of these verdicts between runs. The two distinct 张伟 were never merged in any run.
Tests
namesake_scripted_casespasses with the shorter list;cargo test -p utopia-storeand the adjudication and governance tests pass; clippy is clean.🤖 Generated with Claude Code