Repository navigation
Prefer grounded subject evidence before namesake profile scores - #1251
Conversation
Signed-off-by: Virtue039 <virtue039@qq.com>
…azy fact counts Signed-off-by: Virtue039 <virtue039@qq.com>
…mesake-evidence Signed-off-by: Virtue039 <virtue039@qq.com>
WaylandYang
left a comment
There was a problem hiding this comment.
The namesake tier itself is small and fallback-safe: single-candidate and non-namesake mentions take today's path, no model or embedding call is added, and the two scripted known failures now pass with sixteen guard cases. Three things before it lands:
- An undeclared behaviour change.
extraction_open.rsused to pass the whole chunk text for every named mention; it now passesmention_evidence(), the subject's own located single-sentence quotes, orNone. A mention that appears only as an object, or whose quotes fail the filters, now getsNone, so the existing #331 tie-margin and grey-zone corroboration silently stop running for it. Either keep the chunk text as the fallback for those paths, or state the change and add a case for an object-only mention. - The bench numbers do not support the claim, and your own strict replay says so: all 96 named-entity phases are identical on both builds, so the F1 gain is adjudicator variance. Cut the description to what the head measured (the scripted cases), and drop the live-run deltas.
- The +313 lines of tests mostly re-prove unchanged code (Single / Repeated / Excluded / MixedWithoutClue). Keep the cases the change adds.
Worth landing before the 0044 d6 redesign once those are done; mention_evidence is the piece d6 would keep.
…mesake-evidence Signed-off-by: Virtue039 <virtue039@qq.com>
…ressions Signed-off-by: Virtue039 <virtue039@qq.com>
WaylandYang
left a comment
There was a problem hiding this comment.
The #331 change is now declared and pinned by the object_only case, the description claims only what the scripted cases measure, and the tests that re-proved unchanged code are gone. Object-only ties going to two human reviews instead of an automatic attach is the trade I offered; noted. Approving; this supersedes the change request.
Sync upstream dev into re-dev (2026-10-22, third): deeplethe#1354, deeplethe#1251, deeplethe#1327, deeplethe#1248, deeplethe#1353, deeplethe#1265
An exact-name mention could attach by profile score even when a statement's grounded quote uniquely identifies another existing namesake. Use quotes owned by the current subject before profile tiers, including when the profile scores are far apart.
Noneto the existing tie-margin and grey-zone corroboration paths as well. The whole chunk is no longer a fallback clue. Profile tiers and their normal review behavior then decide; an unresolved profile tie can create an entity and two human reviews. A DB-backed object-only case pins that outcome in both creation orders.This is the resolution-side fix for #1136. The adjudication discussion remains in #1193.
Validation
34 fixed checks pass in both creation orders, retaining the original eight inputs/gold and #1220's real request-content checks. Added cases cover quote ownership/filtering, ambiguous clues, typed/untyped namesakes, incompatible types, stored Markdown spelling and the object-only consequence above. The two passage-over-profile entries are removed from the known-failure list.
Restoring whole-chunk fallback in a temporary negative control makes the object-only test fail because the two expected human reviews disappear. That control was reverted before final validation.
On macOS arm64 with DB/PDF require guards enabled: workspace tests 1,653 passed / 0 failed / 6 existing ignored,
cargo fmt --all --check, locked workspace/all-target clippy (-D warnings), frozen frontend install and frontend build pass. Synced with dev8a9979fon 2026-10-22. CI 39776644634 passes all six jobs at signed commit23561dd761408695fc6a8b9d21a46d82c1ce0d0d.The retained live-run and strict-replay evidence did not demonstrate a change in named-entity choices. This PR claims the deterministic bug fix established by the scripted cases; it makes no live-model accuracy claim.
Refs #1136, #1202, #1220.