Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -547,6 +547,75 @@ and the consumer PRs that deliver it).
committed independently and stayed committed whatever happened next. Which exceptions trigger
that rollback follows Spring's defaults: a `RuntimeException` or an `Error` rolls back and a
checked exception commits, and custom rollback rules can override either behaviour.
- Spring Boot auto-configuration for schema governance: `MetamodelAutoConfiguration` in
`dice-storage-autoconfigure`. It registers only when the application supplies a
Comment thread
jimador marked this conversation as resolved.
`DeclaredSchemaSource` bean, and then wires the loop: version store, drift-report store,
observed-schema source, the two differ roles, quarantine policy, a `DriftSweepCapable`, the drift
runner, and a `SchemaCatalog` carrying the metamodel uniqueness constraints. Every wired
collaborator is `@ConditionalOnMissingBean`, so an application that defines its own keeps it.
Settings live under `embabel.dice.metamodel`: `enabled=false` removes the beans in one
environment while the declared-schema bean stays in place, and `drift.mode` is `off` or `observe`
(the default), which picks whether a `DriftCheckRunner` bean is registered.
Backend selection follows `embabel.dice.store.type`, the same switch the proposition store reads.
Under `graph` the Drivine/Neo4j version store, drift log and observed-schema source are wired.
Under the default in-memory backend an application that declares a schema still starts with no
`PersistenceManager` anywhere: it gets `InMemoryMetamodelVersionStore`, the differ, the policy and
the sweep, and it gets no drift log, no observed-schema source and no runner, because there is no
live graph to observe.
Nothing in the wiring runs a check or moves a proposition. A check happens when the application
calls `DriftCheckRunner.run()`, and it reads, compares and writes a `DriftReport` and touches no
proposition. Quarantine happens when the application calls `DriftSweepCapable.sweep` on a diff it
decided to act on; there is no scheduler and no property that makes DICE sweep by itself. The
wired sweep announces each status transition to every `DiceEventListener` bean on the context
through a `CompositeDiceEventListener`, so a registered `ProjectionLineageStaleCascade` marks the
projection records derived from a quarantined proposition stale.
**Compatibility: additive.** No symbol that exists on the previous release changes shape or
behavior. An application with no `DeclaredSchemaSource` bean sees no change at all. One that
declares a schema and selects the graph backend needs the metamodel constraints, which the
module's `SchemaCatalog` bean supplies, and a `PersistenceManager` on the context; a
`PropositionStore` brings the sweep with it, and its absence leaves the rest of the loop working.

- An operator surface for schema governance. **EXPERIMENTAL** (shape may change before 1.0) — every new public type this change adds is marked `@ApiStatus.Experimental`, the way the extraction stack marks its own unsettled types. Opt-in twice over: the service is wired when a `DeclaredSchemaSource` bean is present, and the HTTP routes appear only when the host also imports `DiceRestConfiguration`.
Until now the loop produced two kinds of inspectable state — drift reports and quarantined propositions — and offered no way to reach either outside a
debugger. `GovernanceOperationsService` in `dice` is the one way in: `latestReports` and
`reportsInContext` read the drift log, `currentDeclaredVersion` reports the declaration in force
along with whether it has been stamped and which version the last completed sweep reconciled
against, `runCheck` runs a check, and `releaseProposition` lifts one quarantine hold.
`GovernanceController` puts it on HTTP under `/api/v1/metamodel` and `GovernanceTools` exposes the
same five operations as `@LlmTool` agent tools; both call the one service, so the two front ends
cannot answer differently.
Reads are bounded: `limit` defaults to 20, must fall between 1 and 200, and a value outside that
answers `400` naming the bound. `since` takes an ISO-8601 instant. A check reports and moves no
proposition, so its response carries the full impact a sweep would evaluate — both drift sets, the
declaration's own movement in `declaredDiff`, and the two merged into `sweepImpact`. A release is
scoped by the context in its path before it writes, so a proposition in another context answers
`404` untouched; a successful release restores the status the proposition carried before quarantine
and answers the state it is in afterwards.
Wiring: `MetamodelAutoConfiguration` registers the service, under the governance conditions plus a
`DriftReportStore`, `DriftCheckRunner`, `DriftSweepCapable` and `PropositionStore` on the context.
It is `@ConditionalOnMissingBean`, so an application that defines its own keeps it and both front
ends run through that one. Building the context stamps nothing, writes no report and moves no
proposition.
`GovernanceController` has no auto-configuration. It joins `DiceRestConfiguration`, the single
`@Import` a host uses to open any DICE REST surface, and switches itself on when a
`GovernanceOperationsService` is there to answer the routes — so the governance endpoints follow
the one REST activation idiom the proposition-pipeline, memory and discovery controllers already
follow. A host that imports DICE REST and declared no schema starts clean and resolves zero
`/api/v1/metamodel` URLs; a host that wants the loop with no endpoint open leaves the import out.
A consumer that declares its own `GovernanceController` bean keeps it, and the shipped one backs
off. Note that `GovernanceController` still ships in the `dice` jar, so a consumer contract test
that scans the classpath for `@RestController` classes sees its six routes whether or not any
context registers them.
`GovernanceTools` is constructed by the host — `GovernanceTools.asTools(service)` — the way every
other DICE tool object is. No DICE auto-configuration registers a tool bean.
Release works one proposition at a time, by design. Nothing in the model ties a quarantined
proposition back to the report whose application held it: a `DriftReport` carries no identity a
reason could name, and the reason a sweep writes names the two schemas and nothing about the check.
Quarantine itself is applied by exactly one thing, a host calling `DriftSweepCapable.sweep`; no
schedule and no property does it. See `docs/design/metamodel-wiring.md`.
**Compatibility: additive.** New types only; no existing symbol changes shape or behavior, and an
application with no `DeclaredSchemaSource` bean sees no change. Nothing appears on an application's
HTTP surface until it imports `DiceRestConfiguration`.

- Optional source revisions in the `dice` core provenance model, the first slice of DICE #64.
`ProvenanceEntry` gains a sixth field, `sourceRevision`: an opaque, provider-defined string,
Expand Down Expand Up @@ -628,6 +697,15 @@ and the consumer PRs that deliver it).

### Fixed

- `MetamodelAutoConfiguration` and the metamodel wiring tests referenced the drift quarantine types
at their former home in `com.embabel.dice.metamodel(.support)`. They moved to
`com.embabel.dice.spi` in the `dice` module when quarantine was given its own
`PropositionStatus.QUARANTINED`, and the wiring was left pointing at the old package, so
`dice-storage-autoconfigure` did not compile from clean. The affected tests also still asserted
`PropositionStatus.STALE` after a drift sweep. Imports corrected and the post-sweep assertions
moved to `QUARANTINED`. **Compatibility: additive.** No shipped symbol changes; the module now
builds from a clean tree.

- `:Source.display` in the graph projection is write-once. A `:Source` node is global — one locator
key is one node across every context that cites it — and `display` used to be refreshed on every
write, so whichever writer ran last owned the label every other context read. It is now set on
Expand Down
10 changes: 9 additions & 1 deletion dice-storage-autoconfigure/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,22 @@ Drivine/Neo4j implementations). The *why* behind these decisions is in

## What's here

Two files, both under `com.embabel.dice.storage.autoconfigure`:
Under `com.embabel.dice.storage.autoconfigure`:

- **`DiceStorageAutoConfiguration`** — declares the store beans for both backends: `PropositionRepository`,
`ChunkHistoryStore`, `DecayManager`, `ProjectionRecordStore`, `CollectorRecordStore`, and the
`SchemaCatalog` beans (constraints, range indexes, vector index). Also `DiceDecaySchedulingConfiguration`,
the separate auto-config that schedules the decay tick.
- **`DiceStoreProperties`** — `@ConfigurationProperties(prefix = "embabel.dice.store")`: the `type`
switch plus nested `decay` and `vector-index` blocks.
- **`MetamodelAutoConfiguration`** — the opt-in wiring for schema governance. It registers only when
the host declares a `DeclaredSchemaSource` bean, and then supplies the version store, drift log,
observed-schema source, differ, quarantine policy, drift runner, and the metamodel `SchemaCatalog`.
Every wired collaborator is `@ConditionalOnMissingBean`, so a host that defines its own keeps it,
except the metamodel `SchemaCatalog` bean, which carries no `@ConditionalOnMissingBean`.
See [`docs/design/metamodel-wiring.md`](../docs/design/metamodel-wiring.md).
- **`MetamodelProperties`** — `@ConfigurationProperties(prefix = "embabel.dice.metamodel")`: the
`enabled` kill switch and the `drift.mode` property (`off` | `observe`, defaulting to `observe`).

## How backend selection works

Expand Down
60 changes: 59 additions & 1 deletion dice-storage-autoconfigure/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@
<artifactId>dice-storage</artifactId>
</dependency>

<!--
Governance contracts (DeclaredSchemaSource, DriftCheckRunner, the differ and policy
defaults) that MetamodelAutoConfiguration wires. Reachable through dice-storage, but
declared here because this module imports the types directly.
-->
<dependency>
<groupId>com.embabel.dice</groupId>
<artifactId>dice-metamodel</artifactId>
</dependency>

<!-- Ai / EmbeddingService for backend + vector-index sizing -->
<dependency>
<groupId>com.embabel.agent</groupId>
Expand Down Expand Up @@ -71,6 +81,16 @@
<version>${kotlin.version}</version>
</dependency>

<!--
JetBrains annotations for @ApiStatus; the version comes from the root pom.
It has to be declared here: another path reaches the same artifact at provided scope,
which wins the resolution and stops dice's own compile-scope copy travelling downstream.
-->
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
Expand Down Expand Up @@ -103,7 +123,45 @@
<dependency>
<groupId>org.mockito.kotlin</groupId>
<artifactId>mockito-kotlin</artifactId>
<version>5.4.0</version>
<scope>test</scope>
</dependency>

<!--
Test: WebApplicationContextRunner needs a servlet API to build its mock context, and that
is how the governance REST bean's web condition is checked. Runtime needs nothing new: the
controller lives in dice, which already declares the servlet API itself.
-->
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<scope>test</scope>
</dependency>

<!--
Test: the governance wiring tests read the URLs a context really exposes off a live
RequestMappingHandlerMapping, which lives in spring-webmvc. `dice` declares spring-webmvc
optional, so it stops there and never reaches this module's classpath. Asking whether a
GovernanceController bean exists is a weaker question than asking which routes resolve,
and the second one is what a host's endpoint snapshot is about.
-->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<scope>test</scope>
</dependency>

<!--
Test: the metamodel autoconfiguration IT drives the whole governance loop against a real
Neo4j. Drivine's test support wires transactions; Neo4jTestContainer supplies the database.
-->
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>neo4j</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
/*
* Copyright 2024-2026 Embabel Pty Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.embabel.dice.storage.autoconfigure

/**
* Configuration prefixes the dice modules read, kept in one place so an annotation cannot misspell one.
*/
internal object DicePropertyPrefixes {
const val DICE = "embabel.dice"
const val STORE = "$DICE.store"
const val METAMODEL = "$DICE.metamodel"
const val METAMODEL_DRIFT = "$METAMODEL.drift"
}
Loading
Loading