feat(dice-storage): Drivine drift-report store and observed-schema source - #87
Conversation
7ca29f9 to
39b3afd
Compare
39b3afd to
fc4b567
Compare
fc4b567 to
da61530
Compare
83d1230 to
9c18815
Compare
9c18815 to
4968354
Compare
4968354 to
55dce72
Compare
55dce72 to
33d243a
Compare
33d243a to
4bf7e86
Compare
4bf7e86 to
5aa6772
Compare
5aa6772 to
ac087aa
Compare
799d5a2 to
c17d51a
Compare
97abe3e to
2482222
Compare
2482222 to
6f1db2d
Compare
|
@jimador - appears rebase is now required |
|
@igordayen both threads are answered above and the annotations version is fixed in |
@jimador yes, i wanted to approve, but it appears rebase is still required. |
6f1db2d to
6177b7a
Compare
|
@igordayen rebased. The whole metamodel stack (#83 through #88 and #106) is now on top of main after the user-guide merge, replayed commit for commit with no conflicts; the only difference between the old and new tops is main's own change. Full |
|
Heads up on the red checks across this stack: every run dies in the first seconds with |
6177b7a to
eaef7b9
Compare
eaef7b9 to
38e07f2
Compare
…urce Three scoped drift-report reads that scope before limiting, ordered by capture instant with a persisted per-schema sequence tie-break so pages are repeatable; injective scope keys (global vs ctx:<id>). Observed schema identifies dice bookkeeping by property shape rather than name, so same-named domain types stay visible and the metamodel never observes its own nodes as drift. End-to-end drift-check IT: declare, ingest an undeclared mention, live run, report persisted, proposition quarantined. Refs #45; stacks on feat/metamodel-drift. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Comment and doc text only; no code change. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
…labels The whole observation now comes from a single read transaction, so a concurrent write landing between two schema queries can no longer produce a combined observation the graph never held. Neo4j reads committed data per row, so the KDoc states what that leaves open rather than claiming snapshot isolation. ObservedSchema gains an entityTypeBasis. A mention's type is domain data an extractor wrote, living in its own namespace apart from graph labels, so a MENTION_TYPES observation compares against declared type names and aliases with no widening to inherited labels. Before this, a mention typed Agent conformed whenever Agent was a parent label of a governed Person, and an undeclared type rode into the graph on that. GRAPH_LABELS keeps the previous semantics and stays the default, and @jvmoverloads preserves the old three-argument constructor for Java callers. DrivineMetamodelVersionStore now overrides sweptVersion and markSwept, tracking sweptContentHash on the schema's counter node, so the durable store follows the baseline independently of write order. Enabling transaction management is the larger change here. dice-storage's test harness never enabled it, and neither did production: Spring Boot 4.1 moved TransactionAutoConfiguration into spring-boot-transaction, which nothing on the classpath brought in. Every @transactional in the module was decorative. Adding the module dependency makes them real, and Boot's configuration backs off for a consumer who already enables it. That activation exposed a latent assumption. DrivinePropositionRepository documents holding its stripe lock across the commit, which only held while its TransactionTemplate ran its own transaction; an ambient transaction made it join instead, moving the commit outside the lock. REQUIRES_NEW restores it, pinned by a test that pauses the writer inside beforeCommit and admits a sibling only there. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The unscoped observation read graph labels only, so an undeclared mention type with no same-named projected label never reached the default drift check, and a mention typed as a governed type's inherited label passed by riding it. ObservedSchema now carries mention types as their own set: the differ judges labels under the observation's basis and mention types under the mention rule, unioning the drift, and the Drivine source fills both from one snapshot. The bookkeeping exclusion derives from the storage schema definitions, so a host domain node shaped like DICE bookkeeping stays observed. DrivineMetamodelVersionStore declares SweptBaselineStore, binding its existing swept tracking to the contract that moved there. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Drivine writes _DrivineSchema for its own schema bookkeeping. DICE's schema definitions cannot derive a label the library owns, so every whole-graph check reported it as drift. Infrastructure labels get their own enumerated exclusion beside the derived catalog, matched by exact name, with a test proving an unknown label still drifts. Also retires the word seam from the module docs and CHANGELOG. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The bookkeeping exclusion was a hand list of four schema objects, and the whole-graph observation read db.labels(), which Neo4j populates for a label the moment a constraint names it, nodes or none. Any store a later slice adds arrived as permanent whole-graph drift: its labels were in the graph and the list had never heard of them. Ownership now derives from the registered DiceStorageSchema beans, so DDL and exclusion come off one list and a schema that registers is a schema the observation excludes, and the observation counts only labels carrying at least one node, because constraint DDL is schema machinery while observation reports data. The guard tests compare the classpath's schema objects against the live context registration, so an unregistered schema fails the build here in place of drifting in production. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Registering a DiceStorageSchema bean now says on the interface what it does: the schema's labels and relationship types become storage bookkeeping the drift observation stops reporting, which is right for a store's own nodes and wrong for domain data, whose home is the DataDictionary. The guard fixture supplies a transaction manager so a store bean that manages its own transactions can wire in the scanned context, and a stale KDoc link follows the nodeShapes rename. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The Drivine property bag is merge-only, so a release that dropped the two quarantine keys from the proposition's metadata left previousStatus and reason on the node. Saving a proposition now removes every metadata.* property the saved metadata no longer names, with one dynamic REMOVE after the node write. The drift integration test checks the keys are gone. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The version now comes from the root pom's dependencyManagement. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
38e07f2 to
8004df5
Compare
PR 5 of the metamodel train, stacked on #86. The Drivine half of the drift story.
DrivineDriftReportStoreruns three distinct Cypher reads (all / global / one context), each scoping before limiting, newest-first with a persisted per-schema sequence as the deterministic tie-break, and injective scope keys so a context id equal to the old sentinel cannot overwrite a global report.DrivineObservedSchemaSourceobserves the whole graph via label and relationship-type queries plus a mention-type query, and observes a context via the proposition walk.DrivineMetamodelVersionStoredeclaresSweptBaselineStore, binding itsmarkSwept/sweptVersionsemantics. A full-loop integration test runs declare → ingest an undeclared mention → check → report persisted and hash-resolvable → sweep → proposition quarantined, against a real Neo4j container.Changed in this review round:
ObservedSchema.EntityTypeBasissplits the declared set: the mention-type arm compares against declared type names plus aliases, so a mention typed with a governed type's inherited parent label no longer passes governance silently. Global mention-type observation is new too — a recorded mention type that projected no node was previously invisible to unscoped checks.DiceStorageSchemais the contract each Cypher-backed store's schema object implements;DiceOwnedSchema.of(registered)derives the bookkeeping exclusion from that bean list, and the DDL comes off the same list viadiceStorageCatalog, so a store cannot get its constraints while sitting outside the exclusion. The hand-written label map is gone; a store registered nowhere stays visible to observation, which is what a drift check should report._DrivineSchemainventory label is excluded by name.@Transactionalexposed that Boot 4.1 moved transaction autoconfiguration intospring-boot-transaction; the module now depends on it, and the dedup boundary runsREQUIRES_NEW.Breaking changes: none. New stores and one new constraint set for hosts adopting drift persistence. Behavioral notes:
DrivineObservedSchemaSourcegains a required constructor argument (branch-new type), and the transaction activation is a genuine runtime change on upgrade, stated in the CHANGELOG.Status: the derived-ownership surface (
DiceStorageSchema,diceStorageCatalog,DiceOwnedSchema) is EXPERIMENTAL; a store takes part by being registered.sequenceDiagram participant R as DriftCheckRunner participant S as DrivineObservedSchemaSource participant N as Neo4j R->>S: observe(contextId) activate S Note over S,N: one transaction, one read snapshot S->>N: labels with nodes, relationship types S->>N: mention types N-->>S: combined observation deactivate S S-->>R: ObservedSchema with basis Note over R: MENTION_TYPES compares against declared names plus aliases Note over R: GRAPH_LABELS compares against the full label closure Note over R: registered DiceStorageSchema beans define the exclusionNext in stack: #88 — opt-in Spring Boot wiring and the operator surface.
previousStatusandreasonfrom the proposition's metadata left both on the node.DrivinePropositionRepositorynow removes everymetadata.*property the saved metadata no longer names, with one dynamicREMOVEafter the node write (Neo4j 5.24+). The drift integration test checks both keys are gone after a release.