Repository navigation
A source closes the facts it stopped stating, in batches, with its write token - #36
Merged
Merged
Conversation
…ite token
A statements source re-pushes its items, and a fact an item stops stating
keeps only superseded evidence: it sits in Review as unconfirmed forever,
and the only way out was one session-only close per fact.
POST /kbs/{id}/sources/{source_id}/close-unconfirmed closes up to `limit`
of them (1..=5000, 1000 by default) through close_superseded, ending each
at the moment its last evidence was superseded, so the history stays. It
takes the facts whose evidence all comes from that source, open statements
first, leaves names and typed rows that follow a statement to the
materialize it queues, and answers closed, skipped (a start after that
moment, left for a person) and remaining. A write token for the base is
enough, as for an ontology import (RC-004).
rewrite_end_tx did not copy layer and phrase, so closing an open
statement, from Review too, wrote a typed row with no predicate that
materialize never read again; the closed row now stays a statement.
gedaiu
added a commit
to re-cinq/lore
that referenced
this pull request
Oct 24, 2026
Bumps `utopia_image_tag` to `0.1.0-re.22` and `utopia_image_digest` to
`sha256:4d82c086...`, the multi-arch index digest checked against ghcr.
re.22 adds `POST /kbs/{id}/sources/{source_id}/close-unconfirmed`
(re-cinq/utopia#36, RC-004): a statements source closes the facts it
stopped stating, in batches, with its write token, each ended when its
last evidence was superseded. It also fixes closing an open statement,
which wrote a typed row with no predicate that materialize never read
again. No new migration since re.21.
After merge: `terraform apply` from main. Then the one-off Job in
`~/.lore/utopia-close/` closes the unconfirmed facts of every
`re-cinq/*` statements source (about 1.65M, `known_as` names left
alone).
Related: #3391, #3397.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A statements source re-pushes its items. When an item stops stating a fact (a value that changed, a relation that went away), the old fact keeps only superseded evidence and stays in Review as unconfirmed for good. Across our bases that is about 1.65 million facts, and the only way out was
POST /facts/{id}/close, one fact at a time and session-only.POST /kbs/{id}/sources/{source_id}/close-unconfirmedwith{"limit": n}(1 to 5000, 1000 by default) closes up tonof them throughclose_superseded, the same invalidate-and-rewrite Review's close uses, so each stays in history with an end. The end is the moment its last evidence chunk was superseded, cut to the second. It picks facts whose evidence all comes from that source's documents and every chunk of it superseded (UNCONFIRMED_FACT), open statements first;known_asnames are left alone, and so are typed rows built from a statement, which materialize rebuilds closed once their statement is closed (the route queues that job). A fact whose start is after that moment is not closed and is counted asskipped, for a person to look at. The answer is{closed, skipped, remaining}, so a pusher loops untilremainingis 0. Auth is the ontology import's: a write token for the base, or an Editor session. One audit row per batch,source.close_unconfirmed. The decision is RC-004;web/src/docs/ingest.mddocuments the route.Building it found a bug in
rewrite_end_tx: it did not copylayerandphrase, so closing an open statement, from Review as well, wrote a typed row with no predicate, which materialize never read again and could not rebuild from. The closed row now stays a statement.rehome_txand the interval-correction insert have the same gap and are left as they are; RC-004 notes them.Tests:
a_source_closes_only_the_facts_it_alone_stopped_stating(store) anda_write_token_closes_through_its_own_bases_source_only(server);cargo test -p utopia-storeandcargo test --bin utopia-servergreen on a fresh migrated database, fmt and clippy-D warningsclean.🤖 Generated with Claude Code