Skip to content

Remember approved property examples so definition edits expose alignment regressions. - #1178

Merged
WaylandYang merged 4 commits into
deeplethe:devfrom
virtue039:feat/ontology-regression-cases
Oct 10, 2026
Merged

WaylandYang merged 4 commits into
deeplethe:devfrom
virtue039:feat/ontology-regression-cases

Conversation

@virtue039

@virtue039 virtue039 commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Why

An approved property has no saved example expectations, so later alignment changes cannot be compared with what the person accepted. This implements the first slice of ADR 0061 cut 2, with the scope agreed in #1103.

What changes

  • Keep knowledge-base-scoped statement/property/direction cases, with actor and adoption / person origin.
  • Return example statement IDs from the existing grouped signatures query, in the same order as its examples and quotes. A proposal keeps its displayed example's ID; adoption commits the cases together with the proposal's adopted status. Older proposals without IDs write no cases, and removed examples are skipped.
  • Add one Editor-only API to save an expected interpretation from a statement.
  • Record the comparison when the existing binding decision commits. Property and direction must match; a person's bound signature explicitly passes with human_bound: true. Store both check time and actual decision time.
  • Use migration 0109 as requested in review, extend the existing reference checks, and record the implementation boundary in the dated note in ADR 0061. The workbench follows in a second PR.

Refs #1103.

How it was checked

  • cargo fmt --all --check and cargo clippy --workspace --all-targets -- -D warnings pass.
  • cargo test --workspace with UTOPIA_TEST_REQUIRE_DB=1 and UTOPIA_TEST_REQUIRE_PDFTOTEXT=1, against an initialized isolated PostgreSQL 16.15 / pgvector 0.8.7 database and Poppler 26.10.0.
  • The workspace run reports 1,481 passed, 0 failed and 6 existing opt-in tests ignored.
  • Two focused new database-backed tests cover the Editor API with wrong-role and foreign-statement rejection, and committed pass/mismatch/human-bound results. The existing adoption test verifies source IDs and saved cases with adopted status, including a legacy proposal without IDs. The existing signatures test verifies IDs follow the same ordered examples.
  • Frozen dependency installation, web build, and all 262 existing web tests pass using Node 20.19.5 / pnpm 10.2.1. This PR has no UI changes.

The comparisons observe existing alignment; they are not a live-model semantic accuracy benchmark.

Before review

  • Every commit is signed off (git commit -s)
  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass
  • SQL under crates/utopia-store/ was tested with UTOPIA_DATABASE_URL set
  • Migration 0109 follows the maintainer's reservation of 0107 and 0108; CURRENT_SCHEMA_VERSION is 98, matching the migration file count on this branch
  • The data model and API follow ADR 0061 and its maintainer-approved first-slice boundary

…ent regressions.

Signed-off-by: Virtue039 <virtue039@qq.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this. It follows the scope we agreed on #1103, and the core is right: the table, cases written with adoption in one commit, one Editor API, and a result recorded when a binding decision commits, with human_bound stated in it.

Four changes before it can land, all toward making it smaller:

  1. Take the example ids from the pass that already reads the examples. propose_locked now calls example_statements once per signature, up to 120 queries a round, each scanning open statements through the phrase normalisation, which no index covers. phrase_bindings::signatures already collects examples and quotes in one pass; please return the statement ids from there. #1108 removed the same per-group scans for a base with 36,000 open statements.
  2. Drop the quote fallback for older proposals. A proposal made before this change writes no cases when adopted. That removes the second half of example_statements and its tests.
  3. Keep the tests to what the change introduces. About 1,400 of the 1,900 lines are tests. One per behaviour is enough: adoption writes its cases in the same commit; the API adds a case and refuses the wrong role or another base's statement; a decision records pass, mismatch and human_bound. The concurrency, rollback, model-failure and cross-base cases go beyond this change.
  4. Housekeeping. Migration 0107 is promised to #1097 and 0108 is in use on #1131, so please take 0109. The design note fits in the dated note you already added to 0061, without a separate file under docs/design/, which holds one file per domain.

I will look again as soon as it is updated. Thanks again for the careful work.

Signed-off-by: Virtue039 <virtue039@qq.com>
…expensive.

Signed-off-by: Virtue039 <virtue039@qq.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for turning this around so quickly. All four points are addressed: the example ids come from the signatures pass, the quote fallback is gone, the tests are down to one per behaviour, and the migration is 0109 with the note folded into 0061. I ran the store, API and agent tests against a fresh database here and they pass, as does CI. Merging now; the workbench view is welcome as the second PR. Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants