Skip to content

Prefer grounded subject evidence before namesake profile scores - #1251

Merged
WaylandYang merged 6 commits into
deeplethe:devfrom
virtue039:codex/mention-local-namesake-evidence
Oct 22, 2026
Merged

WaylandYang merged 6 commits into
deeplethe:devfrom
virtue039:codex/mention-local-namesake-evidence

Conversation

@virtue039

@virtue039 virtue039 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

An exact-name mention could attach by profile score even when a statement's grounded quote uniquely identifies another existing namesake. Use quotes owned by the current subject before profile tiers, including when the profile scores are far apart.

  • Accept only source-located, single-sentence quotes owned by that subject. Missing, fabricated, shared, overlapping and cross-sentence quotes cannot supply identity clues. Shared or conflicting clues cannot become unique by retrying a smaller candidate set.
  • For this comparison, include typed/untyped exact namesakes while respecting incompatible types and caller exclusions. Choosing an unclassified entity does not classify it.
  • Intentional change to The facts that would settle a namesake take no part in settling it #331's text input: an object-only mention, or one whose subject quotes fail the filters, now supplies None to the existing tie-margin and grey-zone corroboration paths as well. The whole chunk is no longer a fallback clue. Profile tiers and their normal review behavior then decide; an unresolved profile tie can create an entity and two human reviews. A DB-backed object-only case pins that outcome in both creation orders.

This is the resolution-side fix for #1136. The adjudication discussion remains in #1193.

Validation

34 fixed checks pass in both creation orders, retaining the original eight inputs/gold and #1220's real request-content checks. Added cases cover quote ownership/filtering, ambiguous clues, typed/untyped namesakes, incompatible types, stored Markdown spelling and the object-only consequence above. The two passage-over-profile entries are removed from the known-failure list.

Restoring whole-chunk fallback in a temporary negative control makes the object-only test fail because the two expected human reviews disappear. That control was reverted before final validation.

On macOS arm64 with DB/PDF require guards enabled: workspace tests 1,653 passed / 0 failed / 6 existing ignored, cargo fmt --all --check, locked workspace/all-target clippy (-D warnings), frozen frontend install and frontend build pass. Synced with dev 8a9979f on 2026-10-22. CI 39776644634 passes all six jobs at signed commit 23561dd761408695fc6a8b9d21a46d82c1ce0d0d.

The retained live-run and strict-replay evidence did not demonstrate a change in named-entity choices. This PR claims the deterministic bug fix established by the scripted cases; it makes no live-model accuracy claim.

Refs #1136, #1202, #1220.

…azy fact counts

Signed-off-by: Virtue039 <virtue039@qq.com>
…mesake-evidence

Signed-off-by: Virtue039 <virtue039@qq.com>
@virtue039
virtue039 marked this pull request as ready for review October 22, 2026 08:31

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The namesake tier itself is small and fallback-safe: single-candidate and non-namesake mentions take today's path, no model or embedding call is added, and the two scripted known failures now pass with sixteen guard cases. Three things before it lands:

  1. An undeclared behaviour change. extraction_open.rs used to pass the whole chunk text for every named mention; it now passes mention_evidence(), the subject's own located single-sentence quotes, or None. A mention that appears only as an object, or whose quotes fail the filters, now gets None, so the existing #331 tie-margin and grey-zone corroboration silently stop running for it. Either keep the chunk text as the fallback for those paths, or state the change and add a case for an object-only mention.
  2. The bench numbers do not support the claim, and your own strict replay says so: all 96 named-entity phases are identical on both builds, so the F1 gain is adjudicator variance. Cut the description to what the head measured (the scripted cases), and drop the live-run deltas.
  3. The +313 lines of tests mostly re-prove unchanged code (Single / Repeated / Excluded / MixedWithoutClue). Keep the cases the change adds.

Worth landing before the 0044 d6 redesign once those are done; mention_evidence is the piece d6 would keep.

…mesake-evidence

Signed-off-by: Virtue039 <virtue039@qq.com>
…ressions

Signed-off-by: Virtue039 <virtue039@qq.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The #331 change is now declared and pinned by the object_only case, the description claims only what the scripted cases measure, and the tests that re-proved unchanged code are gone. Object-only ties going to two human reviews instead of an automatic attach is the trade I offered; noted. Approving; this supersedes the change request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants