From 61cf188923a3e6c130af7eea2442eae7846f8d62 Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:45:25 -0400 Subject: [PATCH 01/10] feat(dice-storage-autoconfigure): opt-in metamodel wiring MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MetamodelAutoConfiguration activates only when the host declares a DeclaredSchemaSource bean — declaring your schema is the opt-in, the way providing a DataSource turns on JPA. Real @AutoConfiguration with every default @ConditionalOnMissingBean so consumer beans win in either registration order; drift runner wired against the base PropositionStore port. embabel.dice.metamodel properties: enabled kill switch and drift.mode off|observe|quarantine — observe is the default, quarantine never is. Also excludes Drivine's _DrivineSchema inventory label from observed schema by verified shape. Refs #45; stacks on feat/metamodel-drift-store. Completes the metamodel train. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- CHANGELOG.md | 17 + dice-storage-autoconfigure/AGENTS.md | 9 +- dice-storage-autoconfigure/pom.xml | 25 + .../MetamodelAutoConfiguration.kt | 269 +++++++++++ .../autoconfigure/MetamodelProperties.kt | 83 ++++ .../ObserveOnlyDriftCheckRunner.kt | 57 +++ ...ot.autoconfigure.AutoConfiguration.imports | 3 +- ...tamodelAutoConfigurationIntegrationTest.kt | 179 +++++++ .../MetamodelAutoConfigurationTest.kt | 443 ++++++++++++++++++ .../autoconfigure/MetamodelTestFixtures.kt | 169 +++++++ .../autoconfigure/Neo4jTestContainer.kt | 61 +++ .../src/test/resources/application.yml | 21 + docs/design/INDEX.md | 7 +- docs/design/architecture.md | 3 +- docs/design/metamodel-diff.md | 4 +- docs/design/metamodel-drift.md | 4 +- docs/design/metamodel-versioning.md | 6 +- docs/design/metamodel-wiring.md | 139 ++++++ 18 files changed, 1489 insertions(+), 10 deletions(-) create mode 100644 dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt create mode 100644 dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt create mode 100644 dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt create mode 100644 dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt create mode 100644 dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt create mode 100644 dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt create mode 100644 dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt create mode 100644 dice-storage-autoconfigure/src/test/resources/application.yml create mode 100644 docs/design/metamodel-wiring.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b6e61b3..bbb0c0a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -547,6 +547,23 @@ and the consumer PRs that deliver it). committed independently and stayed committed whatever happened next. Which exceptions trigger that rollback follows Spring's defaults: a `RuntimeException` or an `Error` rolls back and a checked exception commits, and custom rollback rules can override either behaviour. +- Spring Boot auto-configuration for schema governance: `MetamodelAutoConfiguration` in + `dice-storage-autoconfigure`. It registers only when the application supplies a + `DeclaredSchemaSource` bean, and then wires the whole loop against the existing Drivine + `PersistenceManager`: version store, drift-report store, observed-schema source, differ, + quarantine policy, drift runner, and a `SchemaCatalog` carrying the six metamodel uniqueness + constraints. Every wired collaborator is `@ConditionalOnMissingBean`, so an application that + defines its own keeps it. Settings live under `embabel.dice.metamodel`: `enabled=false` removes + the beans in one environment without deleting the declared-schema bean, and `drift.mode` is + `off`, `observe` or `quarantine`, defaulting to `observe`. Under `observe` the registered runner + is an `ObserveOnlyDriftCheckRunner`, which downgrades `run(dryRun = false)` to a dry run, logs + the downgrade, and returns a report with `dryRun = true`. Under `quarantine` the runner is + `DefaultDriftCheckRunner` and a live run moves stranded propositions to `STALE`. Nothing runs on + a schedule; a caller decides when a check happens. `DrivineObservedSchemaSource` also excludes + Drivine's own `_DrivineSchema` inventory label by shape. + **Compatibility: additive.** An application with no `DeclaredSchemaSource` bean sees no behavior + change. One with one needs the six metamodel constraints, which the module's `SchemaCatalog` + bean supplies, and a `PersistenceManager` and `PropositionStore` on the context. - Optional source revisions in the `dice` core provenance model, the first slice of DICE #64. `ProvenanceEntry` gains a sixth field, `sourceRevision`: an opaque, provider-defined string, diff --git a/dice-storage-autoconfigure/AGENTS.md b/dice-storage-autoconfigure/AGENTS.md index 2015cc49..bad29761 100644 --- a/dice-storage-autoconfigure/AGENTS.md +++ b/dice-storage-autoconfigure/AGENTS.md @@ -8,7 +8,7 @@ Drivine/Neo4j implementations). The *why* behind these decisions is in ## What's here -Two files, both under `com.embabel.dice.storage.autoconfigure`: +Under `com.embabel.dice.storage.autoconfigure`: - **`DiceStorageAutoConfiguration`** — declares the store beans for both backends: `PropositionRepository`, `ChunkHistoryStore`, `DecayManager`, `ProjectionRecordStore`, `CollectorRecordStore`, and the @@ -16,6 +16,13 @@ Two files, both under `com.embabel.dice.storage.autoconfigure`: the separate auto-config that schedules the decay tick. - **`DiceStoreProperties`** — `@ConfigurationProperties(prefix = "embabel.dice.store")`: the `type` switch plus nested `decay` and `vector-index` blocks. +- **`MetamodelAutoConfiguration`** — the opt-in wiring for schema governance. Activates only when the + host declares a `DeclaredSchemaSource` bean (no declared schema, no metamodel beans at all); every + default is `@ConditionalOnMissingBean`, so consumer-supplied implementations always win. See + [`docs/design/metamodel-wiring.md`](../docs/design/metamodel-wiring.md). +- **`MetamodelProperties`** — `@ConfigurationProperties(prefix = "embabel.dice.metamodel")`: the + `enabled` kill switch and `drift.mode` escalation tier (`off` | `observe` | `quarantine`; + quarantine is never a default). ## How backend selection works diff --git a/dice-storage-autoconfigure/pom.xml b/dice-storage-autoconfigure/pom.xml index 6ad1e6c7..2acf3ee5 100644 --- a/dice-storage-autoconfigure/pom.xml +++ b/dice-storage-autoconfigure/pom.xml @@ -23,6 +23,16 @@ dice-storage + + + com.embabel.dice + dice-metamodel + + com.embabel.agent @@ -106,6 +116,21 @@ 5.4.0 test + + + + org.testcontainers + neo4j + test + + + org.testcontainers + junit-jupiter + test + diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt new file mode 100644 index 00000000..d85c2585 --- /dev/null +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt @@ -0,0 +1,269 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import com.embabel.dice.metamodel.DeclaredObservedDiffer +import com.embabel.dice.metamodel.DeclaredSchemaSource +import com.embabel.dice.metamodel.DriftCheckRunner +import com.embabel.dice.metamodel.DriftQuarantinePolicy +import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.MetamodelVersionStore +import com.embabel.dice.metamodel.ObservedSchemaSource +import com.embabel.dice.metamodel.support.DefaultDriftCheckRunner +import com.embabel.dice.metamodel.support.MentionTypeDriftQuarantinePolicy +import com.embabel.dice.metamodel.support.StructuralMetamodelDiffer +import com.embabel.dice.proposition.PropositionStore +import com.embabel.dice.storage.DrivineDriftReportStore +import com.embabel.dice.storage.DrivineMetamodelVersionStore +import com.embabel.dice.storage.DrivineObservedSchemaSource +import com.embabel.dice.storage.MetamodelSchema +import org.drivine.manager.PersistenceManager +import org.drivine.schema.SchemaCatalog +import org.slf4j.LoggerFactory +import org.springframework.boot.autoconfigure.AutoConfiguration +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty +import org.springframework.boot.context.properties.EnableConfigurationProperties +import org.springframework.context.annotation.Bean + +/** + * Wires the schema-governance loop: version stamps, a drift log, a snapshot of what the live graph + * holds, the comparison between the two, and the runner that sequences them. + * + * ## It only switches on if you declared a schema + * + * Governance activates when — and only when — the application supplies a `DeclaredSchemaSource` + * bean. No declared schema, no metamodel beans at all: not the stores, not the differ, not the + * runner, nothing on the classpath doing work in the background. It is the same bargain Spring Boot + * makes with JPA and a `DataSource`. Bring the one thing only you can decide, and the rest of the + * plumbing appears; bring nothing and you don't pay for anything. + * + * That seam is deliberate rather than convenient. There is no sensible default declared schema — + * a schema is a statement about what an application governs, and guessing one would either govern + * everything (turning every exploratory type an LLM extracted into reported drift) or govern nothing + * (making the whole loop a no-op that still logs as if it were working). Requiring the bean makes + * the opt-in explicit and legible in the consumer's own code. + * + * `embabel.dice.metamodel.enabled=false` is the kill switch on top of that — the way to turn + * governance off in one environment without deleting the bean. It is only consulted once the bean + * exists, so it changes nothing for an application that never declared a schema. + * + * ## Tiers, and why quarantine is never the default + * + * `embabel.dice.metamodel.drift.mode` picks how far a check may go: + * + * - `off` — stores and stamps only. Nothing compares a declaration against a graph. + * - `observe` (**default**) — checks run and reports are written; no proposition is ever touched. + * The runner is an [ObserveOnlyDriftCheckRunner], so the guarantee holds even against a caller who + * passes `dryRun = false`. + * - `quarantine` — the real [DefaultDriftCheckRunner]. `run(dryRun = false)` now moves stranded + * propositions to `STALE` with a reason. + * + * Reporting is safe to leave running forever; changing proposition state is a decision somebody has + * to make on purpose. A mistyped type name in a declared schema is a cheap mistake under `observe` + * and an expensive one under `quarantine`, so the escalation is opt-in in exactly the direction that + * mistakes are recoverable. + * + * Nothing here schedules anything. This wires the *capability* to run a check; when a check runs is + * the application's call — a cron, an admin endpoint, a startup hook. + * + * ## Defaults, and replacing them + * + * Every default is `@ConditionalOnMissingBean`, and this is a real `@AutoConfiguration` registered + * through `META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`. That + * pairing is what makes "your bean wins" a guarantee rather than a coincidence: Spring Boot + * registers every application bean definition before it processes a single auto-configuration, so a + * consumer's bean is already there when these conditions are evaluated — no matter which order the + * two were declared in. + * + * The defaults are Drivine/Neo4j-backed, so this needs the same `PersistenceManager` the rest of + * `dice-storage` uses. An application that declared a schema without a graph connection fails at + * startup with the missing bean named, which is the right kind of loud: silently wiring nothing + * would leave someone believing governance was running when it wasn't. + * + * To replace the differ, supply a `DeclaredObservedDiffer` bean — that's the one the runner needs. + * The shipped [StructuralMetamodelDiffer] answers both questions (declaration vs declaration, and + * declaration vs live graph), so it is registered under its concrete type and resolves as either + * interface. + * + * The metamodel constraints ride along as a [SchemaCatalog] bean, the same way the proposition and + * lineage constraints do; Drivine's `SchemaManager` applies them idempotently on startup. They are + * not optional — the stores' MERGEs are only race-free under them. + */ +@AutoConfiguration(after = [DiceStorageAutoConfiguration::class]) +@ConditionalOnBean(DeclaredSchemaSource::class) +@ConditionalOnProperty( + prefix = "embabel.dice.metamodel", + name = ["enabled"], + havingValue = "true", + matchIfMissing = true, +) +@EnableConfigurationProperties(MetamodelProperties::class) +class MetamodelAutoConfiguration { + + private val logger = LoggerFactory.getLogger(MetamodelAutoConfiguration::class.java) + + /** + * The uniqueness constraints the governance stores need. Declared as data in `dice-storage` so + * the constraint list and the label list the observed-schema source excludes stay one edit + * apart. + * + * No `@ConditionalOnMissingBean`, matching the other schema beans in this module: catalogs + * accumulate rather than compete, and a consumer adding their own doesn't mean these stop being + * required. + */ + @Bean + fun metamodelSchema(): SchemaCatalog = SchemaCatalog.of(MetamodelSchema.specs()) + + @Bean + @ConditionalOnMissingBean(MetamodelVersionStore::class) + fun metamodelVersionStore(persistenceManager: PersistenceManager): MetamodelVersionStore { + logger.debug("Wiring default MetamodelVersionStore: DrivineMetamodelVersionStore") + return DrivineMetamodelVersionStore(persistenceManager) + } + + @Bean + @ConditionalOnMissingBean(DriftReportStore::class) + fun driftReportStore(persistenceManager: PersistenceManager): DriftReportStore { + logger.debug("Wiring default DriftReportStore: DrivineDriftReportStore") + return DrivineDriftReportStore(persistenceManager) + } + + @Bean + @ConditionalOnMissingBean(ObservedSchemaSource::class) + fun observedSchemaSource(persistenceManager: PersistenceManager): ObservedSchemaSource { + logger.debug("Wiring default ObservedSchemaSource: DrivineObservedSchemaSource") + return DrivineObservedSchemaSource(persistenceManager) + } + + /** + * The shipped differ, registered under its concrete type so it resolves as both a + * `MetamodelDiffer` (two declarations compared) and a [DeclaredObservedDiffer] (a declaration + * compared against a live graph). One object, two questions. + * + * Backing off keys on [DeclaredObservedDiffer] alone, because that is the collaborator the + * runner needs. A consumer who supplies their own drift differ takes this one's place entirely; + * a consumer who supplies only a `MetamodelDiffer` — a different question, and a legitimate + * thing to want on its own — still gets a working drift check rather than a context that + * refuses to start. + */ + @Bean + @ConditionalOnMissingBean(DeclaredObservedDiffer::class) + fun structuralMetamodelDiffer(): StructuralMetamodelDiffer { + logger.debug("Wiring default differ: StructuralMetamodelDiffer") + return StructuralMetamodelDiffer() + } + + @Bean + @ConditionalOnMissingBean(DriftQuarantinePolicy::class) + fun driftQuarantinePolicy(): DriftQuarantinePolicy { + logger.debug("Wiring default DriftQuarantinePolicy: MentionTypeDriftQuarantinePolicy") + return MentionTypeDriftQuarantinePolicy() + } + + /** + * The `quarantine` tier: the real runner, which honours `run(dryRun = false)`. + * + * Mutually exclusive with [observeOnlyDriftCheckRunner] by property value rather than by + * declaration order, so which one you get never depends on how the two happened to be listed. + */ + @Bean + @ConditionalOnMissingBean(DriftCheckRunner::class) + @ConditionalOnProperty( + prefix = "embabel.dice.metamodel.drift", + name = ["mode"], + havingValue = "quarantine", + ) + fun quarantiningDriftCheckRunner( + declaredSchemaSource: DeclaredSchemaSource, + versionStore: MetamodelVersionStore, + observedSchemaSource: ObservedSchemaSource, + differ: DeclaredObservedDiffer, + driftReportStore: DriftReportStore, + quarantinePolicy: DriftQuarantinePolicy, + propositionStore: PropositionStore, + ): DriftCheckRunner { + logger.info( + "Metamodel drift checking wired in QUARANTINE mode: run(dryRun = false) will move " + + "stranded propositions to STALE. Nothing runs on a schedule; a caller decides when.", + ) + return defaultRunner( + declaredSchemaSource, versionStore, observedSchemaSource, differ, + driftReportStore, quarantinePolicy, propositionStore, + ) + } + + /** + * The default `observe` tier: the real runner behind an [ObserveOnlyDriftCheckRunner], so no + * caller can turn this context's checks live without changing the property. + */ + @Bean + @ConditionalOnMissingBean(DriftCheckRunner::class) + @ConditionalOnProperty( + prefix = "embabel.dice.metamodel.drift", + name = ["mode"], + havingValue = "observe", + matchIfMissing = true, + ) + fun observeOnlyDriftCheckRunner( + declaredSchemaSource: DeclaredSchemaSource, + versionStore: MetamodelVersionStore, + observedSchemaSource: ObservedSchemaSource, + differ: DeclaredObservedDiffer, + driftReportStore: DriftReportStore, + quarantinePolicy: DriftQuarantinePolicy, + propositionStore: PropositionStore, + ): DriftCheckRunner { + logger.info("Metamodel drift checking wired in OBSERVE mode: checks report, nothing is quarantined") + return ObserveOnlyDriftCheckRunner( + defaultRunner( + declaredSchemaSource, versionStore, observedSchemaSource, differ, + driftReportStore, quarantinePolicy, propositionStore, + ), + ) + } + + /** + * Builds the shipped runner. A plain function rather than a shared `@Bean`, because + * `@AutoConfiguration` runs with `proxyBeanMethods = false` — a `@Bean` method called from + * another one would build a second, unmanaged instance. Only one of the two runner beans is + * ever registered, so there is nothing to share anyway. + * + * [propositionStore] is the base `PropositionStore` port, not `PropositionRepository`. A drift + * check reads propositions by context or in bulk and saves them back; it never does vector + * search, graph traversal, or temporal query. Asking for the wider interface here would shut a + * plain store-and-retrieve backend out of governance over capabilities it is never asked to + * use — and a `PropositionRepository` satisfies this parameter anyway, so nothing is lost. + */ + private fun defaultRunner( + declaredSchemaSource: DeclaredSchemaSource, + versionStore: MetamodelVersionStore, + observedSchemaSource: ObservedSchemaSource, + differ: DeclaredObservedDiffer, + driftReportStore: DriftReportStore, + quarantinePolicy: DriftQuarantinePolicy, + propositionStore: PropositionStore, + ): DriftCheckRunner = DefaultDriftCheckRunner( + declaredSchemaSource = declaredSchemaSource, + versionStore = versionStore, + observedSchemaSource = observedSchemaSource, + differ = differ, + driftReportStore = driftReportStore, + quarantinePolicy = quarantinePolicy, + propositionStore = propositionStore, + ) +} diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt new file mode 100644 index 00000000..58099f79 --- /dev/null +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt @@ -0,0 +1,83 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import org.springframework.boot.context.properties.ConfigurationProperties + +/** + * How far the schema-governance loop is allowed to go. + * + * Governance escalates in tiers, and the tier is a decision an operator makes rather than something + * the framework guesses. Each one is only safe on top of the one below it, and each is worth having + * on its own — you can report drift for a year without ever quarantining anything. + */ +enum class DriftMode { + + /** + * No drift checking at all. Stamps and stores are still wired, so an application can record and + * read schema versions, but nothing compares the declaration against a live graph. + */ + OFF, + + /** + * Check and report, never touch a proposition. The default. The runner you get here refuses to + * do a live run: ask it for one and it downgrades to a dry run, says so in the log, and reports + * back `dryRun = true`. + */ + OBSERVE, + + /** + * Check, report, and let a caller quarantine. The runner honours `run(dryRun = false)`, which + * moves stranded propositions to `STALE` with a reason. Still nothing runs on its own — a + * caller has to ask. + */ + QUARANTINE, +} + +/** + * Settings for the metamodel governance loop. + * + * None of this switches governance on by itself. The loop is wired only when the application + * supplies a `DeclaredSchemaSource` bean — no declared schema, no governance, whatever these + * properties say. What they control is what happens once one exists. + */ +@ConfigurationProperties(prefix = "embabel.dice.metamodel") +data class MetamodelProperties( + + /** + * Kill switch. `false` removes every metamodel bean even when a `DeclaredSchemaSource` is + * present — the way to turn governance off for one environment without deleting the bean. + */ + val enabled: Boolean = true, + + /** Drift checking: how far a check is allowed to go. */ + val drift: DriftProperties = DriftProperties(), +) { + + /** Drift-check settings. */ + data class DriftProperties( + + /** + * The escalation tier: `off`, `observe` (the default), or `quarantine`. See [DriftMode]. + * + * Quarantine is never the default. Reporting can't hurt anything, so it's safe to leave on + * forever; changing proposition state is a decision somebody has to make on purpose, and + * making it the default would mean a schema someone mistyped could strand real knowledge on + * the next scheduled check. + */ + val mode: DriftMode = DriftMode.OBSERVE, + ) +} diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt new file mode 100644 index 00000000..7e6cdb21 --- /dev/null +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt @@ -0,0 +1,57 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import com.embabel.agent.core.ContextId +import com.embabel.dice.metamodel.DriftCheckResult +import com.embabel.dice.metamodel.DriftCheckRunner +import org.slf4j.LoggerFactory + +/** + * A [DriftCheckRunner] that will only ever observe. Checks run, reports are written, and no + * proposition is ever touched — asking for a live run gets you a dry one instead. + * + * This is what `embabel.dice.metamodel.drift.mode=observe` wires, and it exists so the observe tier + * is a property of the wiring rather than a habit callers have to keep. A dry-run *default* only + * protects the caller who never passes an argument; a scheduler, an admin endpoint, or a stray + * `run(dryRun = false)` in a script all reach past it. Wrapping the real runner means the guarantee + * holds no matter who calls or what they pass, and switching the tier is a config change rather + * than a code change. + * + * The downgrade is loud but not fatal. Throwing would take down a scheduled job for asking a + * reasonable question in the wrong environment; instead the call succeeds, the report is written, + * and the answer says plainly what happened — [DriftCheckResult.dryRun] comes back `true` and + * [DriftCheckResult.quarantinedCount] is `0`, so a caller that cares can tell it was downgraded + * without reading the log. + * + * @param delegate The real runner, which decides everything except whether the run is live. + */ +internal class ObserveOnlyDriftCheckRunner( + private val delegate: DriftCheckRunner, +) : DriftCheckRunner { + + private val logger = LoggerFactory.getLogger(ObserveOnlyDriftCheckRunner::class.java) + + override fun run(dryRun: Boolean, contextId: ContextId?): DriftCheckResult { + if (!dryRun) { + logger.warn( + "Live drift check requested but the drift mode is 'observe'; running dry instead. " + + "Set embabel.dice.metamodel.drift.mode=quarantine to allow quarantining.", + ) + } + return delegate.run(dryRun = true, contextId = contextId) + } +} diff --git a/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports index 2d717e9b..46249c08 100644 --- a/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ b/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports @@ -1,3 +1,4 @@ com.embabel.dice.storage.autoconfigure.DiceStorageAutoConfiguration com.embabel.dice.storage.autoconfigure.DiceDecaySchedulingConfiguration -com.embabel.dice.storage.autoconfigure.CollectorAutoConfiguration \ No newline at end of file +com.embabel.dice.storage.autoconfigure.CollectorAutoConfiguration +com.embabel.dice.storage.autoconfigure.MetamodelAutoConfiguration \ No newline at end of file diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt new file mode 100644 index 00000000..10a6aa08 --- /dev/null +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt @@ -0,0 +1,179 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import com.embabel.dice.metamodel.DeclaredSchemaSource +import com.embabel.dice.metamodel.DriftCheckRunner +import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.MetamodelVersionStore +import com.embabel.dice.proposition.PropositionStatus +import com.embabel.dice.storage.DrivineDriftReportStore +import com.embabel.dice.storage.DrivineMetamodelVersionStore +import com.embabel.dice.storage.MetamodelSchema +import org.assertj.core.api.Assertions.assertThat +import org.drivine.autoconfigure.EnableDrivine +import org.drivine.autoconfigure.EnableDrivineTestConfig +import org.drivine.manager.PersistenceManager +import org.drivine.manager.PersistenceManagerFactory +import org.drivine.query.QuerySpecification +import org.junit.jupiter.api.BeforeEach +import org.junit.jupiter.api.Test +import org.springframework.beans.factory.annotation.Autowired +import org.springframework.boot.autoconfigure.ImportAutoConfiguration +import org.springframework.boot.test.context.SpringBootTest +import org.springframework.context.annotation.Bean +import org.springframework.context.annotation.Configuration +import org.springframework.context.annotation.EnableAspectJAutoProxy +import org.springframework.test.context.DynamicPropertyRegistry +import org.springframework.test.context.DynamicPropertySource + +/** + * The auto-configured governance loop against a real Neo4j. + * + * The wiring tests prove which beans appear; this proves they add up to something that works. An + * application supplies one `DeclaredSchemaSource` bean and a graph connection, and gets a runner + * that stamps a version, asks the live database what it holds, and leaves a drift report behind — + * all of which is read back out of the database here, not off the objects that wrote it. + * + * The drift is real and comes from the graph: a `(:Ghost)` node nobody declared. The proposition + * side is deliberately kept in memory ([MapPropositionStore]) — this test is about the + * auto-configured Drivine stores and the runner that sequences them, and `dice-storage`'s own + * integration tests already cover quarantining through a graph-backed repository. It also makes + * the point twice over that the runner asks for the base `PropositionStore` port. + */ +@SpringBootTest( + classes = [MetamodelIntegrationTestApplication::class], + properties = ["embabel.dice.metamodel.drift.mode=quarantine"], +) +class MetamodelAutoConfigurationIntegrationTest { + + companion object { + @JvmStatic + @DynamicPropertySource + fun neo4jProperties(registry: DynamicPropertyRegistry) = Neo4jTestContainer.registerProperties(registry) + } + + @Autowired + private lateinit var runner: DriftCheckRunner + + @Autowired + private lateinit var versionStore: MetamodelVersionStore + + @Autowired + private lateinit var driftReportStore: DriftReportStore + + @Autowired + private lateinit var propositionStore: MapPropositionStore + + @Autowired + private lateinit var persistenceManager: PersistenceManager + + @BeforeEach + fun freshGraph() { + // The Spring context is cached across methods, so both the graph and the in-memory + // proposition store carry the previous test's writes unless they are reset. + propositionStore.reset() + (MetamodelSchema.LABELS + "Ghost").forEach { label -> + persistenceManager.execute(QuerySpecification.withStatement("MATCH (n:$label) DETACH DELETE n")) + } + // A domain node nobody declared. This is the drift the check has to find, and it has to + // come out of the database rather than a canned snapshot for the test to mean anything. + persistenceManager.execute(QuerySpecification.withStatement("CREATE (:Ghost {name: 'undeclared'})")) + } + + @Test + fun `the auto-configured beans are the Drivine ones`() { + assertThat(versionStore).isInstanceOf(DrivineMetamodelVersionStore::class.java) + assertThat(driftReportStore).isInstanceOf(DrivineDriftReportStore::class.java) + assertThat(runner).isNotInstanceOf(ObserveOnlyDriftCheckRunner::class.java) + } + + @Test + fun `a live run stamps the schema, finds the undeclared type, and persists a report`() { + val result = runner.run(dryRun = false, contextId = null) + + assertThat(result.dryRun).isFalse() + assertThat(result.driftedEntityTypes).contains("Ghost") + + // The stamp resolves out of the database by the hash the report carries -- the guarantee + // "stamp before you report" exists to buy. + val stamped = versionStore.findVersion(MetamodelTestFixtures.SCHEMA_NAME, result.report.versionHash) + assertThat(stamped).isNotNull + assertThat(stamped!!.entityTypeNames).containsExactly("Person") + + // The report is durable: read back through the store, not off the result object. + val reports = driftReportStore.driftReports(MetamodelTestFixtures.SCHEMA_NAME, limit = 10) + assertThat(reports).hasSize(1) + assertThat(reports.single().driftedEntityTypes).contains("Ghost") + assertThat(reports.single().versionHash).isEqualTo(result.report.versionHash) + + // And the tier did what the tier says: the proposition mentioning Ghost is quarantined, + // the one mentioning Person is left alone. + assertThat(result.quarantinedCount).isEqualTo(1) + assertThat(propositionStore.findByStatus(PropositionStatus.STALE)).hasSize(1) + assertThat(propositionStore.findByStatus(PropositionStatus.ACTIVE)).hasSize(1) + } + + /** + * The self-reporting trap: stamping a version and writing a report both add nodes to the very + * graph the next check observes. If those labels came back as drift, every check would fire + * because the last one ran. + * + * The same goes for `_DrivineSchema`, the inventory node Drivine's `SchemaManager` writes + * when it applies a `SchemaCatalog`: no application can silence it by declaring it, so + * `DrivineObservedSchemaSource` excludes it by shape along with dice's own bookkeeping. + */ + @Test + fun `governance never reports its own bookkeeping as drift`() { + runner.run() + val second = runner.run() + + assertThat(second.driftedEntityTypes).contains("Ghost") + assertThat(second.driftedEntityTypes).doesNotContainAnyElementsOf(MetamodelSchema.LABELS) + assertThat(second.driftedEntityTypes).doesNotContain("_DrivineSchema") + } +} + +/** + * A minimal host: Drivine's test support for the connection and transactions, one + * `DeclaredSchemaSource` to open the gate, one `PropositionStore`, and nothing else. Everything the + * governance loop needs comes from [MetamodelAutoConfiguration]. + * + * `@ImportAutoConfiguration` rather than `@EnableAutoConfiguration` so this stays a focused test of + * one auto-configuration instead of dragging in every one on the classpath. The `.imports` + * registration itself is checked in `MetamodelAutoConfigurationTest`. + */ +@Configuration(proxyBeanMethods = false) +@EnableDrivine +@EnableDrivineTestConfig +@EnableAspectJAutoProxy(proxyTargetClass = true) +@ImportAutoConfiguration(MetamodelAutoConfiguration::class) +internal open class MetamodelIntegrationTestApplication { + + @Bean + open fun persistenceManager(factory: PersistenceManagerFactory): PersistenceManager = factory.get("neo") + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun propositionStore(): MapPropositionStore = MapPropositionStore( + listOf( + MetamodelTestFixtures.proposition("Ada haunts the archive", mentionType = "Ghost"), + MetamodelTestFixtures.proposition("Ada wrote the notes", mentionType = "Person"), + ), + ) +} diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt new file mode 100644 index 00000000..5384303a --- /dev/null +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt @@ -0,0 +1,443 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import com.embabel.agent.core.ContextId +import com.embabel.dice.metamodel.DeclaredObservedDiff +import com.embabel.dice.metamodel.DeclaredObservedDiffer +import com.embabel.dice.metamodel.DeclaredSchema +import com.embabel.dice.metamodel.DeclaredSchemaSource +import com.embabel.dice.metamodel.DriftCheckResult +import com.embabel.dice.metamodel.DriftCheckRunner +import com.embabel.dice.metamodel.DriftQuarantinePolicy +import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.MetamodelDiff +import com.embabel.dice.metamodel.MetamodelDiffer +import com.embabel.dice.metamodel.MetamodelVersionStore +import com.embabel.dice.metamodel.ObservedSchema +import com.embabel.dice.metamodel.ObservedSchemaSource +import com.embabel.dice.metamodel.QuarantineResult +import com.embabel.dice.metamodel.support.DefaultDriftCheckRunner +import com.embabel.dice.metamodel.support.MentionTypeDriftQuarantinePolicy +import com.embabel.dice.metamodel.support.StructuralMetamodelDiffer +import com.embabel.dice.proposition.Proposition +import com.embabel.dice.proposition.PropositionRepository +import com.embabel.dice.proposition.PropositionStatus +import com.embabel.dice.storage.DrivineDriftReportStore +import com.embabel.dice.storage.DrivineMetamodelVersionStore +import com.embabel.dice.storage.DrivineObservedSchemaSource +import org.assertj.core.api.Assertions.assertThat +import org.drivine.manager.PersistenceManager +import org.drivine.schema.SchemaCatalog +import org.junit.jupiter.api.Test +import org.mockito.kotlin.mock +import org.springframework.beans.factory.getBean +import org.springframework.boot.autoconfigure.AutoConfigurations +import org.springframework.boot.test.context.runner.ApplicationContextRunner +import org.springframework.context.annotation.Bean +import org.springframework.context.annotation.Configuration +import org.springframework.core.io.support.PathMatchingResourcePatternResolver + +/** + * Wiring tests for [MetamodelAutoConfiguration]. No Spring Boot app and no database — just the + * auto-configuration, a stub `PersistenceManager`, and whichever governance beans a given test + * wants the application to have supplied. + * + * The questions here are the ones the wiring can get wrong on its own: does governance stay + * completely absent until somebody declares a schema, does a consumer's bean win regardless of + * declaration order, and does each drift tier produce a runner that actually behaves like that + * tier. + */ +class MetamodelAutoConfigurationTest { + + private val autoConfiguration = AutoConfigurations.of(MetamodelAutoConfiguration::class.java) + + /** The application declared a schema and has a graph connection: the normal case. */ + private val runner = ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(DeclaredSchemaConfig::class.java) + + // ---- The opt-in gate ---- + + @Test + fun `no DeclaredSchemaSource bean means no metamodel beans at all`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(NoDeclaredSchemaConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).doesNotHaveBean(MetamodelAutoConfiguration::class.java) + assertThat(ctx).doesNotHaveBean(MetamodelProperties::class.java) + assertThat(ctx).doesNotHaveBean(MetamodelVersionStore::class.java) + assertThat(ctx).doesNotHaveBean(DriftReportStore::class.java) + assertThat(ctx).doesNotHaveBean(ObservedSchemaSource::class.java) + assertThat(ctx).doesNotHaveBean(DeclaredObservedDiffer::class.java) + assertThat(ctx).doesNotHaveBean(DriftQuarantinePolicy::class.java) + assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) + assertThat(ctx).doesNotHaveBean(SchemaCatalog::class.java) + } + } + + @Test + fun `a DeclaredSchemaSource bean wires the whole loop with Drivine-backed defaults`() { + runner.run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).hasSingleBean(MetamodelAutoConfiguration::class.java) + assertThat(ctx.getBean()).isInstanceOf(DrivineMetamodelVersionStore::class.java) + assertThat(ctx.getBean()).isInstanceOf(DrivineDriftReportStore::class.java) + assertThat(ctx.getBean()).isInstanceOf(DrivineObservedSchemaSource::class.java) + assertThat(ctx.getBean()) + .isInstanceOf(MentionTypeDriftQuarantinePolicy::class.java) + assertThat(ctx).hasSingleBean(DriftCheckRunner::class.java) + assertThat(ctx).hasSingleBean(SchemaCatalog::class.java) + } + } + + @Test + fun `the metamodel SchemaCatalog carries the uniqueness constraints the stores need`() { + runner.run { ctx -> + val labels = ctx.getBean().items.map { it.label } + assertThat(labels).contains( + "MetamodelVersion", + "MetamodelSchemaCounter", + "MetamodelDriftReport", + "MetamodelDriftReportCounter", + ) + } + } + + @Test + fun `enabled=false kills every metamodel bean even with a declared schema`() { + runner + .withPropertyValues("embabel.dice.metamodel.enabled=false") + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).doesNotHaveBean(MetamodelAutoConfiguration::class.java) + assertThat(ctx).doesNotHaveBean(MetamodelVersionStore::class.java) + assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) + // The DeclaredSchemaSource the application supplied is untouched — only ours go. + assertThat(ctx).hasSingleBean(DeclaredSchemaSource::class.java) + } + } + + // ---- The differ, resolvable as both of the questions it answers ---- + + @Test + fun `the default differ resolves as both MetamodelDiffer and DeclaredObservedDiffer`() { + runner.run { ctx -> + val asStructural = ctx.getBean() + assertThat(ctx.getBean()).isSameAs(asStructural) + assertThat(ctx.getBean()).isSameAs(asStructural) + } + } + + // ---- Consumer beans win, in either declaration order ---- + + @Test + fun `a consumer differ wins whether it is registered before or after the auto-configuration`() { + bothOrders(CustomDifferConfig::class.java) { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx.getBean()).isInstanceOf(CustomDiffer::class.java) + assertThat(ctx).doesNotHaveBean(StructuralMetamodelDiffer::class.java) + } + } + + @Test + fun `a consumer quarantine policy wins whether it is registered before or after`() { + bothOrders(CustomPolicyConfig::class.java) { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).hasSingleBean(DriftQuarantinePolicy::class.java) + assertThat(ctx.getBean()).isInstanceOf(CustomPolicy::class.java) + } + } + + @Test + fun `a consumer drift-check runner wins whether it is registered before or after`() { + bothOrders(CustomRunnerConfig::class.java) { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).hasSingleBean(DriftCheckRunner::class.java) + assertThat(ctx.getBean()).isInstanceOf(CustomRunner::class.java) + } + } + + @Test + fun `consumer stores win, and then no Drivine connection is needed at all`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).doesNotHaveBean(PersistenceManager::class.java) + assertThat(ctx.getBean()) + .isInstanceOf(RecordingMetamodelVersionStore::class.java) + assertThat(ctx.getBean()).isInstanceOf(RecordingDriftReportStore::class.java) + assertThat(ctx.getBean()).isInstanceOf(FixedObservedSchemaSource::class.java) + } + } + + // ---- The narrow proposition port ---- + + @Test + fun `the runner wires against a bare PropositionStore, with no PropositionRepository in sight`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).doesNotHaveBean(PropositionRepository::class.java) + assertThat(ctx).hasSingleBean(DriftCheckRunner::class.java) + } + } + + // ---- The drift tiers ---- + + @Test + fun `mode off leaves the stores wired and no runner`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .withPropertyValues("embabel.dice.metamodel.drift.mode=off") + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.OFF) + assertThat(ctx).hasSingleBean(MetamodelVersionStore::class.java) + assertThat(ctx).hasSingleBean(DriftReportStore::class.java) + assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) + } + } + + @Test + fun `observe is the default tier and downgrades a live run to a dry one`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .run { ctx -> + assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.OBSERVE) + assertThat(ctx.getBean()).isInstanceOf(ObserveOnlyDriftCheckRunner::class.java) + + // Ask for a live run. Drift is real -- 'Ghost' is observed and never declared -- so + // the quarantine tier would act here. Observe must not. + val result = ctx.getBean().run(dryRun = false, contextId = null) + + assertThat(result.dryRun).isTrue() + assertThat(result.hasDrift).isTrue() + assertThat(result.driftedEntityTypes).containsExactly("Ghost") + assertThat(result.quarantinedCount).isZero() + + val store = ctx.getBean() + assertThat(store.saved).hasSize(1) + assertThat(store.saved.single().driftedEntityTypes).containsExactly("Ghost") + + val propositions = ctx.getBean().findAll() + assertThat(propositions).allMatch { it.status == PropositionStatus.ACTIVE } + } + } + + @Test + fun `explicit observe mode wires the same observe-only runner`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .withPropertyValues("embabel.dice.metamodel.drift.mode=observe") + .run { ctx -> + assertThat(ctx.getBean()).isInstanceOf(ObserveOnlyDriftCheckRunner::class.java) + } + } + + @Test + fun `quarantine mode wires the real runner and a live run really quarantines`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .withPropertyValues("embabel.dice.metamodel.drift.mode=quarantine") + .run { ctx -> + assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.QUARANTINE) + assertThat(ctx.getBean()).isInstanceOf(DefaultDriftCheckRunner::class.java) + + val result = ctx.getBean().run(dryRun = false, contextId = null) + + assertThat(result.dryRun).isFalse() + assertThat(result.driftedEntityTypes).containsExactly("Ghost") + assertThat(result.quarantinedCount).isEqualTo(1) + + val store = ctx.getBean() + assertThat(store.findByStatus(PropositionStatus.STALE)).hasSize(1) + assertThat(store.findByStatus(PropositionStatus.ACTIVE)).hasSize(1) + } + } + + @Test + fun `quarantine mode still leaves a dry run harmless`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .withPropertyValues("embabel.dice.metamodel.drift.mode=quarantine") + .run { ctx -> + val result = ctx.getBean().run() + + assertThat(result.dryRun).isTrue() + assertThat(result.quarantinedCount).isZero() + assertThat(ctx.getBean().findByStatus(PropositionStatus.STALE)).isEmpty() + } + } + + @Test + fun `every run stamps the declared version before the report is written`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryGovernanceConfig::class.java) + .run { ctx -> + ctx.getBean().run() + + val versions = ctx.getBean().saved + val reports = ctx.getBean().saved + assertThat(versions).hasSize(1) + assertThat(reports.single().versionHash).isEqualTo(versions.single().contentHash) + } + } + + // ---- Registration ---- + + @Test + fun `the class is registered as a real auto-configuration, not a scanned Configuration`() { + val declared = PathMatchingResourcePatternResolver() + .getResources("classpath*:META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports") + .flatMap { it.inputStream.bufferedReader().readLines() } + .map { it.trim() } + + assertThat(declared).contains(MetamodelAutoConfiguration::class.java.name) + } + + /** + * Runs [assertions] twice: once with [userConfiguration] registered before the + * auto-configuration and once after. Order-independence is the whole point of shipping this as + * a real `@AutoConfiguration` — a plain `@Configuration` with the same + * `@ConditionalOnMissingBean` annotations would win or lose depending on which got processed + * first. + */ + private fun bothOrders( + userConfiguration: Class<*>, + assertions: (org.springframework.boot.test.context.assertj.AssertableApplicationContext) -> Unit, + ) { + ApplicationContextRunner() + .withUserConfiguration(DeclaredSchemaConfig::class.java, userConfiguration) + .withConfiguration(autoConfiguration) + .run(assertions) + + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(DeclaredSchemaConfig::class.java, userConfiguration) + .run(assertions) + } +} + +/** A graph-connected application that has declared a schema: the Drivine defaults apply. */ +@Configuration(proxyBeanMethods = false) +internal open class DeclaredSchemaConfig { + + @Bean + open fun persistenceManager(): PersistenceManager = mock() + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun propositionStore(): MapPropositionStore = MapPropositionStore() +} + +/** A graph-connected application that has declared nothing. Governance must stay away. */ +@Configuration(proxyBeanMethods = false) +internal open class NoDeclaredSchemaConfig { + + @Bean + open fun persistenceManager(): PersistenceManager = mock() + + @Bean + open fun propositionStore(): MapPropositionStore = MapPropositionStore() +} + +/** + * An application that brought its own governance stores, so no Drivine connection is needed and the + * whole loop can be driven in memory. The observed schema holds a `Ghost` type the declaration never + * mentions, and one of the two propositions mentions it — so a live run has exactly one thing to + * quarantine and one thing to leave alone. + */ +@Configuration(proxyBeanMethods = false) +internal open class InMemoryGovernanceConfig { + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun versionStore(): RecordingMetamodelVersionStore = RecordingMetamodelVersionStore() + + @Bean + open fun driftReportStore(): RecordingDriftReportStore = RecordingDriftReportStore() + + @Bean + open fun observedSchemaSource(): FixedObservedSchemaSource = FixedObservedSchemaSource() + + @Bean + open fun propositionStore(): MapPropositionStore = MapPropositionStore( + listOf( + MetamodelTestFixtures.proposition("Ada haunts the archive", mentionType = "Ghost"), + MetamodelTestFixtures.proposition("Ada wrote the notes", mentionType = "Person"), + ), + ) +} + +internal class CustomDiffer : DeclaredObservedDiffer { + override fun diffAgainstObserved(declared: DeclaredSchema, observed: ObservedSchema): DeclaredObservedDiff = + DeclaredObservedDiff( + declared = declared, + observedSchema = observed, + driftedEntityTypes = emptySet(), + driftedRelationshipTypes = emptySet(), + unobservedEntityTypes = emptySet(), + unobservedRelationshipTypes = emptySet(), + ) +} + +@Configuration(proxyBeanMethods = false) +internal open class CustomDifferConfig { + + @Bean + open fun customDiffer(): DeclaredObservedDiffer = CustomDiffer() +} + +internal class CustomPolicy : DriftQuarantinePolicy { + override fun evaluate(diff: MetamodelDiff, propositions: Iterable): QuarantineResult = + QuarantineResult(conforming = emptyList(), quarantined = emptyList()) +} + +@Configuration(proxyBeanMethods = false) +internal open class CustomPolicyConfig { + + @Bean + open fun customPolicy(): DriftQuarantinePolicy = CustomPolicy() +} + +internal class CustomRunner : DriftCheckRunner { + override fun run(dryRun: Boolean, contextId: ContextId?): DriftCheckResult = + throw UnsupportedOperationException("never called; this test only asks which bean won") +} + +@Configuration(proxyBeanMethods = false) +internal open class CustomRunnerConfig { + + @Bean + open fun customRunner(): DriftCheckRunner = CustomRunner() +} diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt new file mode 100644 index 00000000..c0500c12 --- /dev/null +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt @@ -0,0 +1,169 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import com.embabel.agent.core.ContextId +import com.embabel.agent.rag.service.RetrievableIdentifier +import com.embabel.dice.metamodel.DeclaredSchema +import com.embabel.dice.metamodel.DeclaredSchemaSource +import com.embabel.dice.metamodel.DriftReport +import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.MetamodelVersion +import com.embabel.dice.metamodel.MetamodelVersionStore +import com.embabel.dice.metamodel.ObservedSchema +import com.embabel.dice.metamodel.ObservedSchemaSource +import com.embabel.dice.proposition.EntityMention +import com.embabel.dice.proposition.Proposition +import com.embabel.dice.proposition.PropositionStatus +import com.embabel.dice.proposition.PropositionStore +import java.time.Instant + +/** + * Hand-written stand-ins for the governance collaborators, used by the wiring tests. + * + * Real objects rather than mocks on purpose. Half of what these tests check is *behaviour* — did + * the observe tier really refuse to quarantine, did a report really get written — and a mock that + * records calls can't answer that without restating the runner's logic in the assertions. These + * keep their state in a list you can read afterwards. + */ +internal object MetamodelTestFixtures { + + const val SCHEMA_NAME = "test-schema" + + val CONTEXT_ID = ContextId("metamodel-test-context") + + /** A declared schema that governs one entity type, `Person`, and no relationships. */ + fun declaredSchema(vararg entityTypeNames: String = arrayOf("Person")): DeclaredSchema = + DeclaredSchema( + version = MetamodelVersion( + schemaName = SCHEMA_NAME, + entityTypeNames = entityTypeNames.toList(), + entityTypeLabels = emptyMap(), + entityTypeProperties = emptyMap(), + relationshipNames = emptyList(), + ), + relationshipTypeNames = emptySet(), + ) + + /** A proposition mentioning [mentionType], so the quarantine policy has something to catch. */ + fun proposition(text: String, mentionType: String): Proposition = + Proposition( + contextId = CONTEXT_ID, + text = text, + mentions = listOf(EntityMention(span = text, type = mentionType)), + confidence = 0.9, + ) +} + +/** A [DeclaredSchemaSource] handing back one fixed declaration. */ +internal class FixedDeclaredSchemaSource( + private val declared: DeclaredSchema = MetamodelTestFixtures.declaredSchema(), +) : DeclaredSchemaSource { + override fun declare(): DeclaredSchema = declared +} + +/** An [ObservedSchemaSource] handing back one fixed snapshot, whatever the scope. */ +internal class FixedObservedSchemaSource( + private val observed: ObservedSchema = ObservedSchema( + entityTypeNames = setOf("Person", "Ghost"), + relationshipTypeNames = emptySet(), + capturedAt = Instant.parse("2026-01-01T00:00:00Z"), + ), +) : ObservedSchemaSource { + override fun observe(contextId: ContextId?): ObservedSchema = observed +} + +/** A [MetamodelVersionStore] that keeps stamps in a list, newest last. */ +internal class RecordingMetamodelVersionStore : MetamodelVersionStore { + + val saved = mutableListOf() + + override fun saveVersion(version: MetamodelVersion) { + saved += version + } + + override fun latestVersion(schemaName: String): MetamodelVersion? = + saved.lastOrNull { it.schemaName == schemaName } + + override fun versionHistory(schemaName: String): List = + saved.filter { it.schemaName == schemaName }.reversed() +} + +/** A [DriftReportStore] that keeps reports in a list, newest last. */ +internal class RecordingDriftReportStore : DriftReportStore { + + val saved = mutableListOf() + + override fun saveDriftReport(report: DriftReport) { + saved += report + } + + override fun driftReports(schemaName: String, limit: Int, since: Instant?): List = + saved.filter { it.schemaName == schemaName }.reversed().take(limit) + + override fun globalDriftReports(schemaName: String, limit: Int, since: Instant?): List = + driftReports(schemaName, limit, since).filter { it.contextId == null } + + override fun driftReportsInContext( + schemaName: String, + contextId: ContextId, + limit: Int, + since: Instant?, + ): List = driftReports(schemaName, limit, since).filter { it.contextId == contextId } +} + +/** + * A [PropositionStore] over an in-memory map. Deliberately the *base* port and nothing more: a + * context holding one of these has no `PropositionRepository`, which is what proves the drift + * runner asks for the narrow port it actually uses. + */ +internal class MapPropositionStore(private val initial: List = emptyList()) : PropositionStore { + + private val byId = linkedMapOf() + + init { + reset() + } + + /** Back to the propositions this store was built with, all `ACTIVE` again. */ + fun reset() { + byId.clear() + initial.forEach { byId[it.id] = it } + } + + override fun save(proposition: Proposition): Proposition { + byId[proposition.id] = proposition + return proposition + } + + override fun findById(id: String): Proposition? = byId[id] + + override fun findByEntity(entityIdentifier: RetrievableIdentifier): List = emptyList() + + override fun findByStatus(status: PropositionStatus): List = + byId.values.filter { it.status == status } + + override fun findByGrounding(chunkId: String): List = + byId.values.filter { chunkId in it.grounding } + + override fun findByMinLevel(minLevel: Int): List = byId.values.filter { it.level >= minLevel } + + override fun findAll(): List = byId.values.toList() + + override fun delete(id: String): Boolean = byId.remove(id) != null + + override fun count(): Int = byId.size +} diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt new file mode 100644 index 00000000..3fc394b6 --- /dev/null +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt @@ -0,0 +1,61 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +import org.springframework.test.context.DynamicPropertyRegistry +import org.testcontainers.containers.Neo4jContainer +import org.testcontainers.utility.DockerImageName + +/** + * One self-managed Neo4j testcontainer, shared by every IT in this module's test JVM, pinned to a + * version we choose instead of the `neo4j:5.26.1-community` `@EnableDrivineTestConfig` hardcodes + * (no override hook of its own, and 0.0.58 is drivine4j's newest release). `5.26.1-community` has + * a confirmed upstream bug (https://github.com/neo4j/neo4j/issues/13597): a dynamic + * relationship-type parameter gets baked into the query-plan cache on first execution and + * silently reused for every later execution of the same query text with a different value. + * Confirmed fixed on `neo4j:2026.05-community`, which is what this starts. + * + * Rather than let Drivine start its own container, each test class wires this one in via + * `test.neo4j.use-local=true` -- the officially-supported switch that tells + * `DrivineTestConfiguration` to use whatever's in the Spring `Environment` for the `neo` datasource + * as-is, instead of overriding host/port/password with its own testcontainer. A `@DynamicPropertySource` + * method in each test class (see [registerProperties]) supplies those values from *this* container, + * started on first use and reused (via Kotlin `object`/`by lazy`) for the rest of the test JVM. + * + * Duplicated (not shared) with `dice-storage`'s copy -- each module's tests run in their own forked + * JVM/classpath. + */ +object Neo4jTestContainer { + + const val PASSWORD = "test-password" + + val instance: Neo4jContainer<*> by lazy { + Neo4jContainer(DockerImageName.parse("neo4j:2026.05-community").asCompatibleSubstituteFor(DockerImageName.parse("neo4j"))) + .withAdminPassword(PASSWORD) + .withPlugins("apoc") + .withNeo4jConfig("dbms.security.procedures.unrestricted", "apoc.*") + .withNeo4jConfig("dbms.security.procedures.allowlist", "apoc.*") + .also { it.start() } + } + + /** Points Drivine's `neo` datasource at [instance] instead of its own testcontainer. */ + fun registerProperties(registry: DynamicPropertyRegistry) { + registry.add("test.neo4j.use-local") { "true" } + registry.add("database.datasources.neo.host") { instance.host } + registry.add("database.datasources.neo.port") { instance.getMappedPort(7687) } + registry.add("database.datasources.neo.password") { PASSWORD } + } +} diff --git a/dice-storage-autoconfigure/src/test/resources/application.yml b/dice-storage-autoconfigure/src/test/resources/application.yml new file mode 100644 index 00000000..1c827455 --- /dev/null +++ b/dice-storage-autoconfigure/src/test/resources/application.yml @@ -0,0 +1,21 @@ +spring: + main: + allow-bean-definition-overriding: true + +# Drivine datasource for the metamodel integration test. Host, port and password are replaced at +# runtime by Neo4jTestContainer via @DynamicPropertySource; the rest has to be here because +# Drivine's ConnectionProperties refuses to bind with any of its fields missing. +database: + datasources: + neo: + host: localhost + port: 7687 + username: neo4j + password: test + type: NEO4J + database-name: neo4j + +logging: + level: + org.drivine: INFO + com.embabel.dice.storage.autoconfigure: DEBUG diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index 75541b33..c8f62f55 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -84,6 +84,9 @@ you need. - [metamodel-drift.md](metamodel-drift.md) — checking a live graph against a declared schema: the runner's declare, stamp, observe, diff, report sequence, the bounds on every read of the drift log, and quarantine, which marks affected propositions stale rather than deleting them. +- [metamodel-wiring.md](metamodel-wiring.md) — turning governance on in a Spring Boot host: the + opt-in is supplying a `DeclaredSchemaSource` bean, the escalation tier is one property, and the + default tier reports drift without touching any proposition. ## Modules @@ -94,9 +97,9 @@ dependency map. Quick pointer to where each is documented: | --- | --- | | `dice` (core) | most notes above — propositions, pipeline, projections, hygiene, retrieval | | `dice-storage` | [durable-storage.md](durable-storage.md), [graph-projection.md](graph-projection.md), [prolog-projection.md](prolog-projection.md) | -| `dice-storage-autoconfigure` | [durable-storage.md](durable-storage.md) | +| `dice-storage-autoconfigure` | [durable-storage.md](durable-storage.md), [metamodel-wiring.md](metamodel-wiring.md) | | `dice-ingestion` | [ingestion.md](ingestion.md) | | `dice-report` | [report.md](report.md) | -| `dice-metamodel` | [metamodel-versioning.md](metamodel-versioning.md), [metamodel-diff.md](metamodel-diff.md), [metamodel-drift.md](metamodel-drift.md) | +| `dice-metamodel` | [metamodel-versioning.md](metamodel-versioning.md), [metamodel-diff.md](metamodel-diff.md), [metamodel-drift.md](metamodel-drift.md), [metamodel-wiring.md](metamodel-wiring.md) | | `dice-integration-tests` | not separately documented — exercises the above end-to-end | diff --git a/docs/design/architecture.md b/docs/design/architecture.md index 43cbd944..308c705e 100644 --- a/docs/design/architecture.md +++ b/docs/design/architecture.md @@ -13,7 +13,7 @@ DICE is a multi-module Maven build. Each module's intent, and what it's allowed |---|---| | `dice` | The core: proposition model, pipeline, gates, projection interfaces, query facades, agent tools, REST controllers. In-memory implementations only — no database driver. | | `dice-storage` | The durable Neo4j backend: `Drivine`-based repository, graph/Prolog/lineage projectors, schema and index bootstrap, and the governance persistence side: `MetamodelVersionStore`, the `DriftReportStore` drift log, and the `ObservedSchemaSource` that asks the live graph what it holds, excluding dice's own bookkeeping labels and edges so governance doesn't observe itself. Depends on `dice` and `dice-metamodel`. | -| `dice-storage-autoconfigure` | Spring Boot autoconfiguration that wires `dice-storage`'s beans (repository, projectors, trust scorer) into a host application. Depends on `dice-storage`. | +| `dice-storage-autoconfigure` | Spring Boot autoconfiguration that wires `dice-storage`'s beans (repository, projectors, trust scorer) into a host application, plus the schema-governance loop: version store, drift log, observed-schema source, differ, quarantine policy and drift runner. That loop is wired only when the application supplies a `DeclaredSchemaSource` bean, and its escalation tier is one property, `off` / `observe` / `quarantine`, defaulting to `observe`. See [metamodel-wiring.md](metamodel-wiring.md). Depends on `dice-storage`. | | `dice-ingestion` | Content-hash dedup ledger and source adapters that sit in front of `PropositionPipeline`, so the same artifact is never extracted twice concurrently. Depends on `dice`. | | `dice-report` | Rationale and structured report generation over propositions and their lineage. Depends on `dice`. | | `dice-metamodel` | Schema governance: content-hash stamps over the governed part of a `DataDictionary`, the declared-schema contract, the version and drift-report store contracts, diffing, drift checking, and non-destructive quarantine. A leaf over `embabel-agent-api`, with no dependency on `dice`; `dice-storage` implements its store contracts. | @@ -300,3 +300,4 @@ and `CollectorRecord` MERGE on their natural keys so replayed writes are idempot | REST surface | `dice/web/rest/DiscoveryController.kt` | | Events | `dice/common/` (event types), `EventEmittingPropositionRepository` | | Spring Boot wiring | `dice-storage-autoconfigure/DiceStorageAutoConfiguration.kt` | +| Schema-governance wiring | `dice-storage-autoconfigure/MetamodelAutoConfiguration.kt` | diff --git a/docs/design/metamodel-diff.md b/docs/design/metamodel-diff.md index 5650c5de..a72f0780 100644 --- a/docs/design/metamodel-diff.md +++ b/docs/design/metamodel-diff.md @@ -338,7 +338,9 @@ schema changes, so stamp with the `GovernedTypeSelector` first (or take both sta `DeclaredSchema.from(...)`) and use the `MetamodelVersion` overload. `StructuralMetamodelDiffer` is stateless and thread-safe; one shared instance is fine. This module -has no Spring wiring, so it's an ordinary constructor call until the autoconfigure slice arrives. +has no Spring wiring, so it's an ordinary constructor call. In a Boot app, +`dice-storage-autoconfigure` registers one, resolvable as both a `MetamodelDiffer` and a +`DeclaredObservedDiffer`; see [metamodel-wiring.md](metamodel-wiring.md). ## What comes next diff --git a/docs/design/metamodel-drift.md b/docs/design/metamodel-drift.md index 13f69359..6abbd74f 100644 --- a/docs/design/metamodel-drift.md +++ b/docs/design/metamodel-drift.md @@ -94,7 +94,9 @@ graph, and record what you find. The drift check sits here, and it is all `Drift **Quarantine.** Act on a lossy change by marking the affected propositions stale, which is a different thing from deleting them. A host does this through `DriftSweepCapable`, at a moment it chooses. This slice adds the contracts and a reference implementation; a durable store's own -implementation, and any wiring, come later. +implementation comes later. In a Boot app the wiring is described in +[metamodel-wiring.md](metamodel-wiring.md). Nothing schedules a check, and nothing sweeps unless a +host calls it. Each tier builds on the one below it, and each is useful on its own. You can stamp for a year without detecting, and detect for a year without quarantining. Rejecting undeclared types at write diff --git a/docs/design/metamodel-versioning.md b/docs/design/metamodel-versioning.md index 6b7e9d77..6afd0dc7 100644 --- a/docs/design/metamodel-versioning.md +++ b/docs/design/metamodel-versioning.md @@ -172,9 +172,9 @@ class MyAppDeclaredSchemaSource( } ``` -Versioning starts here: with no declared schema, nothing is stamped. The Spring wiring that lands -in a later slice activates only when a `DeclaredSchemaSource` bean is present, so an application -that hasn't decided what it governs is left alone. +Versioning starts here: with no declared schema, nothing is stamped. The Spring wiring activates +only when a `DeclaredSchemaSource` bean is present, so an application that hasn't decided what it +governs is left alone. See [metamodel-wiring.md](metamodel-wiring.md). ## Declared renames diff --git a/docs/design/metamodel-wiring.md b/docs/design/metamodel-wiring.md new file mode 100644 index 00000000..74ea5912 --- /dev/null +++ b/docs/design/metamodel-wiring.md @@ -0,0 +1,139 @@ +# Metamodel wiring: turning governance on + +The governance pieces — stamps, diffs, drift checks, quarantine — are ordinary objects with +constructors. This note is about the last mile: how a Spring Boot application gets them without +assembling them by hand, and why turning them on is a thing you have to *do* rather than a thing +that happens to you. + +## The bargain: bring a declared schema, get the plumbing + +`MetamodelAutoConfiguration` activates when — and only when — the application supplies a +`DeclaredSchemaSource` bean. + +That is the same deal Spring Boot makes with JPA. Put a `DataSource` on the context and an +`EntityManagerFactory`, a transaction manager, and a repository infrastructure appear behind it. +Leave it out and none of that machinery exists; it isn't disabled, it was never created. Declaring a +schema is the DICE equivalent: the one decision only the application can make, and the signal that +it wants everything downstream of it. + +```mermaid +flowchart TD + A{"DeclaredSchemaSource
bean present?"} + A -- no --> Z["Nothing. No stores, no differ,
no runner, no cost."] + A -- yes --> B{"embabel.dice.metamodel
.enabled"} + B -- false --> Z + B -- true --> C["Version store, drift log,
observed-schema source,
differ, quarantine policy,
schema constraints"] + C --> D{"embabel.dice.metamodel
.drift.mode"} + D -- off --> E["No runner.
Stamps and history only."] + D -- "observe (default)" --> F["ObserveOnlyDriftCheckRunner
reports; never touches a proposition"] + D -- quarantine --> G["DefaultDriftCheckRunner
run(dryRun = false) marks stranded
propositions STALE"] +``` + +Requiring the bean is not ceremony. There is no sensible default declared schema, and both ways of +guessing one are bad. Govern everything in the `DataDictionary` and every exploratory type an +extractor invented becomes reported drift, which trains everyone to ignore the reports. Govern +nothing and the loop is a no-op that still writes reassuring log lines. Making the application say +what it governs puts that decision in the consumer's own code, where a reader can see it. + +`embabel.dice.metamodel.enabled=false` sits on top as a kill switch — the way to switch governance +off in one environment without deleting the bean. It is only consulted once the bean exists, so it +changes nothing for an application that never declared a schema. + +## Tiers, and why quarantine is never the default + +`embabel.dice.metamodel.drift.mode` picks how far a check may go. It mirrors the three tiers in +[metamodel-drift.md](metamodel-drift.md), one property value each: + +| Mode | What you get | What it can change | +| --- | --- | --- | +| `off` | Stores and stamps. Schema history is recorded and readable. | Nothing | +| `observe` (default) | A runner that checks and writes reports | Nothing | +| `quarantine` | The real runner | Marks stranded propositions `STALE`, with a reason | + +Reporting is safe to leave running forever, so it is the default. Changing proposition state is a +decision somebody has to make on purpose, so it is not. + +The asymmetry is about which mistakes are recoverable. A typo in a declared type name under +`observe` produces a report naming a type you expected to see — annoying, five minutes to fix. +The same typo under `quarantine` marks every proposition mentioning that type stale on the next +scheduled check. Quarantine is non-destructive, so that is recoverable too, but only if somebody +notices. Defaults should fail in the direction where nobody has to notice. + +`observe` is enforced by wiring rather than by convention. The bean you get is an +`ObserveOnlyDriftCheckRunner` wrapping the real one, and it downgrades `run(dryRun = false)` to a dry +run, logs a warning, and returns a result whose `dryRun` is `true`. A dry-run *default* only protects +the caller who passes no arguments; a scheduler, an admin endpoint, or one line in a script all reach +straight past it. Making the tier a property of the bean means the guarantee holds no matter who +calls, and moving between tiers is a config change rather than a code change. + +Nothing is scheduled here. This wires the *capability* to run a check; when one runs is the +application's call. + +## Your bean always wins, whatever the order + +Every default is `@ConditionalOnMissingBean`, and `MetamodelAutoConfiguration` is a real +`@AutoConfiguration` listed in +`META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`. + +Both halves matter, and the second is the one that is easy to get wrong. `@ConditionalOnMissingBean` +asks "does a bean of this type exist *yet*?" — a question whose answer depends on when it is asked. +A plain `@Configuration` picked up by component scanning carries the same annotations but is +processed in scan order, so whether the consumer's bean has been registered by then is luck. Spring +Boot registers every application bean definition *before* it processes a single auto-configuration, +so registering through the imports file turns "your bean wins" from a coincidence into a guarantee. +`MetamodelAutoConfigurationTest` checks it from both directions, registering the consumer's +configuration before and after the auto-configuration and asserting the same result. + +The replaceable pieces: + +| Type | Default | Backed by | +| --- | --- | --- | +| `MetamodelVersionStore` | `DrivineMetamodelVersionStore` | Neo4j | +| `DriftReportStore` | `DrivineDriftReportStore` | Neo4j | +| `ObservedSchemaSource` | `DrivineObservedSchemaSource` | Neo4j | +| `DeclaredObservedDiffer` | `StructuralMetamodelDiffer` | pure JVM | +| `DriftQuarantinePolicy` | `MentionTypeDriftQuarantinePolicy` | pure JVM | +| `DriftCheckRunner` | per `drift.mode`, above | — | + +The differ is registered under its concrete type, because `StructuralMetamodelDiffer` answers two +different questions — declaration against declaration (`MetamodelDiffer`) and declaration against a +live graph (`DeclaredObservedDiffer`) — and both need to resolve. Backing off keys on +`DeclaredObservedDiffer` alone, since that is the collaborator the runner needs. Supplying only a +`MetamodelDiffer`, which is a legitimate thing to want on its own, leaves drift checking working +rather than refusing to start. + +The metamodel uniqueness constraints ride along as a `SchemaCatalog` bean, the same way the +proposition and lineage constraints do; Drivine's `SchemaManager` applies them idempotently on +startup. They are not optional — the stores' MERGEs are only race-free under them — so unlike the +store beans they carry no `@ConditionalOnMissingBean`. Catalogs accumulate rather than compete. + +## The runner takes the narrow port + +`DefaultDriftCheckRunner` is wired against `PropositionStore`, the base persistence port — not +`PropositionRepository`, which adds vector search, graph traversal, temporal query, and core search +operations on top. + +A drift check reads propositions by context or in bulk and saves the flagged copies back. That is +all of it. Asking for the wider interface at the wiring layer would shut a plain store-and-retrieve +backend out of governance over capabilities it is never asked to use, and it would do so invisibly: +the context would simply fail to find a bean. A `PropositionRepository` satisfies the narrow +parameter anyway, so nothing is given up by asking for less. + +## Failing loudly + +The defaults are Drivine/Neo4j-backed, and they are not gated on a `PersistenceManager` being +present. An application that declared a schema without a graph connection fails at startup with the +missing bean named. + +That is deliberate. The alternative — quietly wiring nothing when the connection is missing — leaves +somebody believing governance is running when it isn't, which is the one outcome a governance +feature must never produce. An application that genuinely wants the loop without Drivine supplies its +own three stores; they are `@ConditionalOnMissingBean` like everything else, and then no +`PersistenceManager` is asked for at all. + +## Property reference + +| Property | Default | Meaning | +| --- | --- | --- | +| `embabel.dice.metamodel.enabled` | `true` | Kill switch. Only consulted when a `DeclaredSchemaSource` bean exists | +| `embabel.dice.metamodel.drift.mode` | `observe` | `off`, `observe`, or `quarantine` — see above | From 7e4de11a5f1f7feacb9025699be22de885af8537 Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Sun, 30 Aug 2026 21:29:21 -0400 Subject: [PATCH 02/10] docs(dice-storage-autoconfigure): voice pass on wiring docs and KDoc Comment and doc text only; no code change. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- dice-storage-autoconfigure/AGENTS.md | 13 +- .../MetamodelAutoConfiguration.kt | 98 +++++++------ .../autoconfigure/MetamodelProperties.kt | 27 ++-- .../ObserveOnlyDriftCheckRunner.kt | 24 ++-- ...tamodelAutoConfigurationIntegrationTest.kt | 49 +++---- .../MetamodelAutoConfigurationTest.kt | 32 ++--- .../autoconfigure/MetamodelTestFixtures.kt | 14 +- .../autoconfigure/Neo4jTestContainer.kt | 31 ++--- docs/design/metamodel-wiring.md | 130 ++++++++---------- 9 files changed, 199 insertions(+), 219 deletions(-) diff --git a/dice-storage-autoconfigure/AGENTS.md b/dice-storage-autoconfigure/AGENTS.md index bad29761..054d0d24 100644 --- a/dice-storage-autoconfigure/AGENTS.md +++ b/dice-storage-autoconfigure/AGENTS.md @@ -16,13 +16,14 @@ Under `com.embabel.dice.storage.autoconfigure`: the separate auto-config that schedules the decay tick. - **`DiceStoreProperties`** — `@ConfigurationProperties(prefix = "embabel.dice.store")`: the `type` switch plus nested `decay` and `vector-index` blocks. -- **`MetamodelAutoConfiguration`** — the opt-in wiring for schema governance. Activates only when the - host declares a `DeclaredSchemaSource` bean (no declared schema, no metamodel beans at all); every - default is `@ConditionalOnMissingBean`, so consumer-supplied implementations always win. See - [`docs/design/metamodel-wiring.md`](../docs/design/metamodel-wiring.md). +- **`MetamodelAutoConfiguration`** — the opt-in wiring for schema governance. It registers only when + the host declares a `DeclaredSchemaSource` bean, and then supplies the version store, drift log, + observed-schema source, differ, quarantine policy, drift runner, and the metamodel `SchemaCatalog`. + Every wired collaborator is `@ConditionalOnMissingBean`, so a host that defines its own keeps it. + See [`docs/design/metamodel-wiring.md`](../docs/design/metamodel-wiring.md). - **`MetamodelProperties`** — `@ConfigurationProperties(prefix = "embabel.dice.metamodel")`: the - `enabled` kill switch and `drift.mode` escalation tier (`off` | `observe` | `quarantine`; - quarantine is never a default). + `enabled` kill switch and the `drift.mode` escalation tier (`off` | `observe` | `quarantine`, + defaulting to `observe`). ## How backend selection works diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt index d85c2585..b421e9a4 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt @@ -44,65 +44,61 @@ import org.springframework.context.annotation.Bean * Wires the schema-governance loop: version stamps, a drift log, a snapshot of what the live graph * holds, the comparison between the two, and the runner that sequences them. * - * ## It only switches on if you declared a schema + * ## The opt-in is a declared schema * - * Governance activates when — and only when — the application supplies a `DeclaredSchemaSource` - * bean. No declared schema, no metamodel beans at all: not the stores, not the differ, not the - * runner, nothing on the classpath doing work in the background. It is the same bargain Spring Boot - * makes with JPA and a `DataSource`. Bring the one thing only you can decide, and the rest of the - * plumbing appears; bring nothing and you don't pay for anything. + * Governance activates when the application supplies a `DeclaredSchemaSource` bean, and only then. + * With no declared schema there are no metamodel beans at all: no stores, no differ, no runner. It + * is the same arrangement Spring Boot has with JPA and a `DataSource`. * - * That seam is deliberate rather than convenient. There is no sensible default declared schema — - * a schema is a statement about what an application governs, and guessing one would either govern - * everything (turning every exploratory type an LLM extracted into reported drift) or govern nothing - * (making the whole loop a no-op that still logs as if it were working). Requiring the bean makes - * the opt-in explicit and legible in the consumer's own code. + * There is no sensible default declared schema. A schema states what an application governs, and + * guessing one would either govern everything, turning every exploratory type an LLM extracted into + * reported drift, or govern nothing, leaving the loop a no-op that still logs as if it were working. + * Requiring the bean puts the opt-in in the consumer's own code, where a reader can see it. * - * `embabel.dice.metamodel.enabled=false` is the kill switch on top of that — the way to turn - * governance off in one environment without deleting the bean. It is only consulted once the bean - * exists, so it changes nothing for an application that never declared a schema. + * `embabel.dice.metamodel.enabled=false` is a kill switch on top of that, for switching governance + * off in one environment while the bean stays in place. It is consulted only once the bean exists, + * so it changes nothing for an application that never declared a schema. * - * ## Tiers, and why quarantine is never the default + * ## Drift tiers * - * `embabel.dice.metamodel.drift.mode` picks how far a check may go: + * `embabel.dice.metamodel.drift.mode` sets how far a check may go: * * - `off` — stores and stamps only. Nothing compares a declaration against a graph. - * - `observe` (**default**) — checks run and reports are written; no proposition is ever touched. - * The runner is an [ObserveOnlyDriftCheckRunner], so the guarantee holds even against a caller who - * passes `dryRun = false`. - * - `quarantine` — the real [DefaultDriftCheckRunner]. `run(dryRun = false)` now moves stranded + * - `observe` (**default**) — checks run and reports are written; no proposition is touched. The + * runner is an [ObserveOnlyDriftCheckRunner], so the guarantee holds against a caller who passes + * `dryRun = false`. + * - `quarantine` — the real [DefaultDriftCheckRunner]. `run(dryRun = false)` moves stranded * propositions to `STALE` with a reason. * - * Reporting is safe to leave running forever; changing proposition state is a decision somebody has - * to make on purpose. A mistyped type name in a declared schema is a cheap mistake under `observe` - * and an expensive one under `quarantine`, so the escalation is opt-in in exactly the direction that - * mistakes are recoverable. + * The default is `observe`. Reporting is safe to leave running indefinitely; changing proposition + * state is a decision somebody makes on purpose. A mistyped type name in a declared schema costs + * five minutes under `observe` and marks every proposition mentioning that type stale under + * `quarantine`, so the escalation is opt-in in the direction where mistakes are recoverable. * - * Nothing here schedules anything. This wires the *capability* to run a check; when a check runs is - * the application's call — a cron, an admin endpoint, a startup hook. + * Nothing here schedules anything. This wires the capability to run a check; when a check runs is + * the application's call, from a cron, an admin endpoint, or a startup hook. * * ## Defaults, and replacing them * * Every default is `@ConditionalOnMissingBean`, and this is a real `@AutoConfiguration` registered * through `META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`. That - * pairing is what makes "your bean wins" a guarantee rather than a coincidence: Spring Boot - * registers every application bean definition before it processes a single auto-configuration, so a - * consumer's bean is already there when these conditions are evaluated — no matter which order the - * two were declared in. + * pairing is what makes "your bean wins" a guarantee: Spring Boot registers every application bean + * definition before it processes a single auto-configuration, so a consumer's bean is already there + * when these conditions are evaluated, whichever order the two were declared in. * * The defaults are Drivine/Neo4j-backed, so this needs the same `PersistenceManager` the rest of * `dice-storage` uses. An application that declared a schema without a graph connection fails at - * startup with the missing bean named, which is the right kind of loud: silently wiring nothing - * would leave someone believing governance was running when it wasn't. + * startup with the missing bean named. Wiring nothing instead would leave someone believing + * governance was running when it wasn't. * - * To replace the differ, supply a `DeclaredObservedDiffer` bean — that's the one the runner needs. + * To replace the differ, supply a `DeclaredObservedDiffer` bean, which is the one the runner needs. * The shipped [StructuralMetamodelDiffer] answers both questions (declaration vs declaration, and * declaration vs live graph), so it is registered under its concrete type and resolves as either * interface. * * The metamodel constraints ride along as a [SchemaCatalog] bean, the same way the proposition and - * lineage constraints do; Drivine's `SchemaManager` applies them idempotently on startup. They are - * not optional — the stores' MERGEs are only race-free under them. + * lineage constraints do; Drivine's `SchemaManager` applies them idempotently on startup. The + * stores' MERGEs are only race-free under them, so they are required. */ @AutoConfiguration(after = [DiceStorageAutoConfiguration::class]) @ConditionalOnBean(DeclaredSchemaSource::class) @@ -122,9 +118,8 @@ class MetamodelAutoConfiguration { * the constraint list and the label list the observed-schema source excludes stay one edit * apart. * - * No `@ConditionalOnMissingBean`, matching the other schema beans in this module: catalogs - * accumulate rather than compete, and a consumer adding their own doesn't mean these stop being - * required. + * No `@ConditionalOnMissingBean`, matching the other schema beans in this module. Catalogs + * accumulate, so a consumer adding their own gets both, and these are still required. */ @Bean fun metamodelSchema(): SchemaCatalog = SchemaCatalog.of(MetamodelSchema.specs()) @@ -153,13 +148,12 @@ class MetamodelAutoConfiguration { /** * The shipped differ, registered under its concrete type so it resolves as both a * `MetamodelDiffer` (two declarations compared) and a [DeclaredObservedDiffer] (a declaration - * compared against a live graph). One object, two questions. + * compared against a live graph). * * Backing off keys on [DeclaredObservedDiffer] alone, because that is the collaborator the - * runner needs. A consumer who supplies their own drift differ takes this one's place entirely; - * a consumer who supplies only a `MetamodelDiffer` — a different question, and a legitimate - * thing to want on its own — still gets a working drift check rather than a context that - * refuses to start. + * runner needs. A consumer who supplies their own drift differ takes this one's place entirely. + * A consumer who supplies only a `MetamodelDiffer`, which is a legitimate thing to want on its + * own, still gets a working drift check. */ @Bean @ConditionalOnMissingBean(DeclaredObservedDiffer::class) @@ -178,8 +172,8 @@ class MetamodelAutoConfiguration { /** * The `quarantine` tier: the real runner, which honours `run(dryRun = false)`. * - * Mutually exclusive with [observeOnlyDriftCheckRunner] by property value rather than by - * declaration order, so which one you get never depends on how the two happened to be listed. + * Mutually exclusive with [observeOnlyDriftCheckRunner] by property value, so which one you get + * doesn't depend on the order the two are listed in. */ @Bean @ConditionalOnMissingBean(DriftCheckRunner::class) @@ -208,8 +202,8 @@ class MetamodelAutoConfiguration { } /** - * The default `observe` tier: the real runner behind an [ObserveOnlyDriftCheckRunner], so no - * caller can turn this context's checks live without changing the property. + * The default `observe` tier: the real runner behind an [ObserveOnlyDriftCheckRunner], so + * turning this context's checks live takes a property change. */ @Bean @ConditionalOnMissingBean(DriftCheckRunner::class) @@ -243,11 +237,11 @@ class MetamodelAutoConfiguration { * another one would build a second, unmanaged instance. Only one of the two runner beans is * ever registered, so there is nothing to share anyway. * - * [propositionStore] is the base `PropositionStore` port, not `PropositionRepository`. A drift - * check reads propositions by context or in bulk and saves them back; it never does vector - * search, graph traversal, or temporal query. Asking for the wider interface here would shut a - * plain store-and-retrieve backend out of governance over capabilities it is never asked to - * use — and a `PropositionRepository` satisfies this parameter anyway, so nothing is lost. + * [propositionStore] is the base `PropositionStore` port. A drift check reads propositions by + * context or in bulk and saves them back, and uses none of the vector search, graph traversal + * or temporal query that `PropositionRepository` adds. Asking for the wider interface here + * would shut a plain store-and-retrieve backend out of governance over capabilities it is + * never asked to use, and a `PropositionRepository` satisfies this parameter anyway. */ private fun defaultRunner( declaredSchemaSource: DeclaredSchemaSource, diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt index 58099f79..4b9bf024 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt @@ -20,9 +20,8 @@ import org.springframework.boot.context.properties.ConfigurationProperties /** * How far the schema-governance loop is allowed to go. * - * Governance escalates in tiers, and the tier is a decision an operator makes rather than something - * the framework guesses. Each one is only safe on top of the one below it, and each is worth having - * on its own — you can report drift for a year without ever quarantining anything. + * Governance escalates in tiers, and an operator picks the tier. Each tier is useful on its own: an + * application can report drift for a year and quarantine nothing. */ enum class DriftMode { @@ -33,16 +32,15 @@ enum class DriftMode { OFF, /** - * Check and report, never touch a proposition. The default. The runner you get here refuses to - * do a live run: ask it for one and it downgrades to a dry run, says so in the log, and reports - * back `dryRun = true`. + * The default. Check and report, touching no proposition. Ask the runner wired here for a live + * run and it downgrades to a dry run, logs the downgrade, and reports back `dryRun = true`. */ OBSERVE, /** * Check, report, and let a caller quarantine. The runner honours `run(dryRun = false)`, which - * moves stranded propositions to `STALE` with a reason. Still nothing runs on its own — a - * caller has to ask. + * moves stranded propositions to `STALE` with a reason. Nothing runs on its own; a caller has + * to ask. */ QUARANTINE, } @@ -51,15 +49,15 @@ enum class DriftMode { * Settings for the metamodel governance loop. * * None of this switches governance on by itself. The loop is wired only when the application - * supplies a `DeclaredSchemaSource` bean — no declared schema, no governance, whatever these - * properties say. What they control is what happens once one exists. + * supplies a `DeclaredSchemaSource` bean, whatever these properties say. They control what happens + * once one exists. */ @ConfigurationProperties(prefix = "embabel.dice.metamodel") data class MetamodelProperties( /** * Kill switch. `false` removes every metamodel bean even when a `DeclaredSchemaSource` is - * present — the way to turn governance off for one environment without deleting the bean. + * present, so governance can be switched off for one environment while the bean stays in place. */ val enabled: Boolean = true, @@ -73,10 +71,9 @@ data class MetamodelProperties( /** * The escalation tier: `off`, `observe` (the default), or `quarantine`. See [DriftMode]. * - * Quarantine is never the default. Reporting can't hurt anything, so it's safe to leave on - * forever; changing proposition state is a decision somebody has to make on purpose, and - * making it the default would mean a schema someone mistyped could strand real knowledge on - * the next scheduled check. + * The default is `observe`. Reporting is safe to leave running indefinitely; changing + * proposition state is a decision somebody makes on purpose. Defaulting to `quarantine` + * would let a mistyped schema strand real knowledge on the next check. */ val mode: DriftMode = DriftMode.OBSERVE, ) diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt index 7e6cdb21..69da004e 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt @@ -21,21 +21,19 @@ import com.embabel.dice.metamodel.DriftCheckRunner import org.slf4j.LoggerFactory /** - * A [DriftCheckRunner] that will only ever observe. Checks run, reports are written, and no - * proposition is ever touched — asking for a live run gets you a dry one instead. + * A [DriftCheckRunner] that only observes. Checks run and reports are written; a request for a live + * run is downgraded to a dry one. * - * This is what `embabel.dice.metamodel.drift.mode=observe` wires, and it exists so the observe tier - * is a property of the wiring rather than a habit callers have to keep. A dry-run *default* only - * protects the caller who never passes an argument; a scheduler, an admin endpoint, or a stray - * `run(dryRun = false)` in a script all reach past it. Wrapping the real runner means the guarantee - * holds no matter who calls or what they pass, and switching the tier is a config change rather - * than a code change. + * This is what `embabel.dice.metamodel.drift.mode=observe` wires, so the observe tier is enforced by + * the wiring. A dry-run default on the method only protects the caller who passes no argument, and a + * scheduler, an admin endpoint, or a stray `run(dryRun = false)` in a script all reach past it. + * Wrapping the real runner holds the guarantee whoever calls and whatever they pass, and switching + * the tier is then a config change. * - * The downgrade is loud but not fatal. Throwing would take down a scheduled job for asking a - * reasonable question in the wrong environment; instead the call succeeds, the report is written, - * and the answer says plainly what happened — [DriftCheckResult.dryRun] comes back `true` and - * [DriftCheckResult.quarantinedCount] is `0`, so a caller that cares can tell it was downgraded - * without reading the log. + * The downgrade logs a warning and returns normally. Throwing would take down a scheduled job for + * asking a reasonable question in the wrong environment. The report is written, + * [DriftCheckResult.dryRun] comes back `true` and [DriftCheckResult.quarantinedCount] is `0`, so a + * caller can tell it was downgraded without reading the log. * * @param delegate The real runner, which decides everything except whether the run is live. */ diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt index 10a6aa08..109a5353 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt @@ -43,16 +43,16 @@ import org.springframework.test.context.DynamicPropertySource /** * The auto-configured governance loop against a real Neo4j. * - * The wiring tests prove which beans appear; this proves they add up to something that works. An + * The wiring tests cover which beans appear. This one covers whether they work together. An * application supplies one `DeclaredSchemaSource` bean and a graph connection, and gets a runner - * that stamps a version, asks the live database what it holds, and leaves a drift report behind — - * all of which is read back out of the database here, not off the objects that wrote it. + * that stamps a version, asks the live database what it holds, and leaves a drift report behind. + * Every assertion reads that back out of the database. * - * The drift is real and comes from the graph: a `(:Ghost)` node nobody declared. The proposition - * side is deliberately kept in memory ([MapPropositionStore]) — this test is about the - * auto-configured Drivine stores and the runner that sequences them, and `dice-storage`'s own - * integration tests already cover quarantining through a graph-backed repository. It also makes - * the point twice over that the runner asks for the base `PropositionStore` port. + * The drift comes from the graph: a `(:Ghost)` node nobody declared. The proposition side stays in + * memory ([MapPropositionStore]), since this test is about the auto-configured Drivine stores and + * the runner that sequences them, and `dice-storage`'s own integration tests already cover + * quarantining through a graph-backed repository. It also exercises the runner's use of the base + * `PropositionStore` port. */ @SpringBootTest( classes = [MetamodelIntegrationTestApplication::class], @@ -89,8 +89,8 @@ class MetamodelAutoConfigurationIntegrationTest { (MetamodelSchema.LABELS + "Ghost").forEach { label -> persistenceManager.execute(QuerySpecification.withStatement("MATCH (n:$label) DETACH DELETE n")) } - // A domain node nobody declared. This is the drift the check has to find, and it has to - // come out of the database rather than a canned snapshot for the test to mean anything. + // A domain node nobody declared. This is the drift the check has to find, and it comes + // out of the database rather than a canned snapshot. persistenceManager.execute(QuerySpecification.withStatement("CREATE (:Ghost {name: 'undeclared'})")) } @@ -108,33 +108,34 @@ class MetamodelAutoConfigurationIntegrationTest { assertThat(result.dryRun).isFalse() assertThat(result.driftedEntityTypes).contains("Ghost") - // The stamp resolves out of the database by the hash the report carries -- the guarantee - // "stamp before you report" exists to buy. + // The stamp resolves out of the database by the hash the report carries. That is what + // stamping before reporting buys. val stamped = versionStore.findVersion(MetamodelTestFixtures.SCHEMA_NAME, result.report.versionHash) assertThat(stamped).isNotNull assertThat(stamped!!.entityTypeNames).containsExactly("Person") - // The report is durable: read back through the store, not off the result object. + // The report is durable: read back through the store rather than off the result object. val reports = driftReportStore.driftReports(MetamodelTestFixtures.SCHEMA_NAME, limit = 10) assertThat(reports).hasSize(1) assertThat(reports.single().driftedEntityTypes).contains("Ghost") assertThat(reports.single().versionHash).isEqualTo(result.report.versionHash) - // And the tier did what the tier says: the proposition mentioning Ghost is quarantined, - // the one mentioning Person is left alone. + // The tier did what it says: the proposition mentioning Ghost is quarantined, the one + // mentioning Person is left alone. assertThat(result.quarantinedCount).isEqualTo(1) assertThat(propositionStore.findByStatus(PropositionStatus.STALE)).hasSize(1) assertThat(propositionStore.findByStatus(PropositionStatus.ACTIVE)).hasSize(1) } /** - * The self-reporting trap: stamping a version and writing a report both add nodes to the very - * graph the next check observes. If those labels came back as drift, every check would fire - * because the last one ran. + * Stamping a version and writing a report both add nodes to the graph the next check observes. + * If those labels came back as drift, every check would report drift caused by the previous + * check. * - * The same goes for `_DrivineSchema`, the inventory node Drivine's `SchemaManager` writes - * when it applies a `SchemaCatalog`: no application can silence it by declaring it, so - * `DrivineObservedSchemaSource` excludes it by shape along with dice's own bookkeeping. + * `_DrivineSchema` is the same problem from outside dice: Drivine's `SchemaManager` writes that + * inventory node when it applies a `SchemaCatalog`, and no application can silence it by + * declaring it. `DrivineObservedSchemaSource` excludes it by shape along with dice's own + * bookkeeping labels. */ @Test fun `governance never reports its own bookkeeping as drift`() { @@ -152,9 +153,9 @@ class MetamodelAutoConfigurationIntegrationTest { * `DeclaredSchemaSource` to open the gate, one `PropositionStore`, and nothing else. Everything the * governance loop needs comes from [MetamodelAutoConfiguration]. * - * `@ImportAutoConfiguration` rather than `@EnableAutoConfiguration` so this stays a focused test of - * one auto-configuration instead of dragging in every one on the classpath. The `.imports` - * registration itself is checked in `MetamodelAutoConfigurationTest`. + * `@ImportAutoConfiguration` keeps this a test of one auto-configuration; + * `@EnableAutoConfiguration` would drag in every one on the classpath. The `.imports` registration + * itself is checked in `MetamodelAutoConfigurationTest`. */ @Configuration(proxyBeanMethods = false) @EnableDrivine diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt index 5384303a..ec4d6d71 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt @@ -52,14 +52,13 @@ import org.springframework.context.annotation.Configuration import org.springframework.core.io.support.PathMatchingResourcePatternResolver /** - * Wiring tests for [MetamodelAutoConfiguration]. No Spring Boot app and no database — just the + * Wiring tests for [MetamodelAutoConfiguration]. No Spring Boot app and no database: the * auto-configuration, a stub `PersistenceManager`, and whichever governance beans a given test * wants the application to have supplied. * - * The questions here are the ones the wiring can get wrong on its own: does governance stay - * completely absent until somebody declares a schema, does a consumer's bean win regardless of - * declaration order, and does each drift tier produce a runner that actually behaves like that - * tier. + * These cover what the wiring alone can get wrong: whether governance stays absent until somebody + * declares a schema, whether a consumer's bean wins whichever declaration order it arrives in, and + * whether each drift tier produces a runner that behaves like that tier. */ class MetamodelAutoConfigurationTest { @@ -128,12 +127,13 @@ class MetamodelAutoConfigurationTest { assertThat(ctx).doesNotHaveBean(MetamodelAutoConfiguration::class.java) assertThat(ctx).doesNotHaveBean(MetamodelVersionStore::class.java) assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) - // The DeclaredSchemaSource the application supplied is untouched — only ours go. + // The DeclaredSchemaSource the application supplied stays; only the metamodel + // beans are removed. assertThat(ctx).hasSingleBean(DeclaredSchemaSource::class.java) } } - // ---- The differ, resolvable as both of the questions it answers ---- + // ---- The differ, resolvable under both interfaces ---- @Test fun `the default differ resolves as both MetamodelDiffer and DeclaredObservedDiffer`() { @@ -228,8 +228,8 @@ class MetamodelAutoConfigurationTest { assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.OBSERVE) assertThat(ctx.getBean()).isInstanceOf(ObserveOnlyDriftCheckRunner::class.java) - // Ask for a live run. Drift is real -- 'Ghost' is observed and never declared -- so - // the quarantine tier would act here. Observe must not. + // Ask for a live run. 'Ghost' is observed and undeclared, so the quarantine tier + // would act here. val result = ctx.getBean().run(dryRun = false, contextId = null) assertThat(result.dryRun).isTrue() @@ -323,9 +323,9 @@ class MetamodelAutoConfigurationTest { /** * Runs [assertions] twice: once with [userConfiguration] registered before the - * auto-configuration and once after. Order-independence is the whole point of shipping this as - * a real `@AutoConfiguration` — a plain `@Configuration` with the same - * `@ConditionalOnMissingBean` annotations would win or lose depending on which got processed + * auto-configuration and once after. Shipping this as a real `@AutoConfiguration` is what makes + * the result order-independent. A plain `@Configuration` with the same + * `@ConditionalOnMissingBean` annotations would win or lose depending on which was processed * first. */ private fun bothOrders( @@ -358,7 +358,7 @@ internal open class DeclaredSchemaConfig { open fun propositionStore(): MapPropositionStore = MapPropositionStore() } -/** A graph-connected application that has declared nothing. Governance must stay away. */ +/** A graph-connected application that has declared nothing, so no governance beans appear. */ @Configuration(proxyBeanMethods = false) internal open class NoDeclaredSchemaConfig { @@ -371,9 +371,9 @@ internal open class NoDeclaredSchemaConfig { /** * An application that brought its own governance stores, so no Drivine connection is needed and the - * whole loop can be driven in memory. The observed schema holds a `Ghost` type the declaration never - * mentions, and one of the two propositions mentions it — so a live run has exactly one thing to - * quarantine and one thing to leave alone. + * whole loop runs in memory. The observed schema holds a `Ghost` type the declaration never + * mentions, and one of the two propositions mentions it, so a live run has one thing to quarantine + * and one to leave alone. */ @Configuration(proxyBeanMethods = false) internal open class InMemoryGovernanceConfig { diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt index c0500c12..9e254802 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt @@ -34,10 +34,10 @@ import java.time.Instant /** * Hand-written stand-ins for the governance collaborators, used by the wiring tests. * - * Real objects rather than mocks on purpose. Half of what these tests check is *behaviour* — did - * the observe tier really refuse to quarantine, did a report really get written — and a mock that - * records calls can't answer that without restating the runner's logic in the assertions. These - * keep their state in a list you can read afterwards. + * These are real objects, because half of what the tests check is behaviour: whether the observe + * tier refused to quarantine, whether a report was written. A mock that records calls can only + * answer that by restating the runner's logic in the assertions. These keep their state in a list + * the test reads afterwards. */ internal object MetamodelTestFixtures { @@ -126,9 +126,9 @@ internal class RecordingDriftReportStore : DriftReportStore { } /** - * A [PropositionStore] over an in-memory map. Deliberately the *base* port and nothing more: a - * context holding one of these has no `PropositionRepository`, which is what proves the drift - * runner asks for the narrow port it actually uses. + * A [PropositionStore] over an in-memory map. It implements the base port and nothing more, so a + * context holding one has no `PropositionRepository`. That is what shows the drift runner asks only + * for the narrow port. */ internal class MapPropositionStore(private val initial: List = emptyList()) : PropositionStore { diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt index 3fc394b6..dd0de65e 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/Neo4jTestContainer.kt @@ -20,23 +20,22 @@ import org.testcontainers.containers.Neo4jContainer import org.testcontainers.utility.DockerImageName /** - * One self-managed Neo4j testcontainer, shared by every IT in this module's test JVM, pinned to a - * version we choose instead of the `neo4j:5.26.1-community` `@EnableDrivineTestConfig` hardcodes - * (no override hook of its own, and 0.0.58 is drivine4j's newest release). `5.26.1-community` has - * a confirmed upstream bug (https://github.com/neo4j/neo4j/issues/13597): a dynamic - * relationship-type parameter gets baked into the query-plan cache on first execution and - * silently reused for every later execution of the same query text with a different value. - * Confirmed fixed on `neo4j:2026.05-community`, which is what this starts. + * One self-managed Neo4j testcontainer, shared by every IT in this module's test JVM, on a version + * we pick. `@EnableDrivineTestConfig` hardcodes `neo4j:5.26.1-community` with no override hook, and + * 0.0.58 is drivine4j's newest release. `5.26.1-community` has a confirmed upstream bug + * (https://github.com/neo4j/neo4j/issues/13597): a dynamic relationship-type parameter gets baked + * into the query-plan cache on first execution and silently reused for every later execution of the + * same query text with a different value. Confirmed fixed on `neo4j:2026.05-community`, which is + * what this starts. * - * Rather than let Drivine start its own container, each test class wires this one in via - * `test.neo4j.use-local=true` -- the officially-supported switch that tells - * `DrivineTestConfiguration` to use whatever's in the Spring `Environment` for the `neo` datasource - * as-is, instead of overriding host/port/password with its own testcontainer. A `@DynamicPropertySource` - * method in each test class (see [registerProperties]) supplies those values from *this* container, - * started on first use and reused (via Kotlin `object`/`by lazy`) for the rest of the test JVM. + * Each test class wires this container in with `test.neo4j.use-local=true`, the supported switch + * that tells `DrivineTestConfiguration` to take the `neo` datasource's host, port and password from + * the Spring `Environment` as they are. A `@DynamicPropertySource` method in each test class (see + * [registerProperties]) supplies those values from this container, started on first use and reused + * (via Kotlin `object`/`by lazy`) for the rest of the test JVM. * - * Duplicated (not shared) with `dice-storage`'s copy -- each module's tests run in their own forked - * JVM/classpath. + * `dice-storage` keeps its own copy; each module's tests run in a forked JVM with its own + * classpath. */ object Neo4jTestContainer { @@ -51,7 +50,7 @@ object Neo4jTestContainer { .also { it.start() } } - /** Points Drivine's `neo` datasource at [instance] instead of its own testcontainer. */ + /** Points Drivine's `neo` datasource at [instance]. */ fun registerProperties(registry: DynamicPropertyRegistry) { registry.add("test.neo4j.use-local") { "true" } registry.add("database.datasources.neo.host") { instance.host } diff --git a/docs/design/metamodel-wiring.md b/docs/design/metamodel-wiring.md index 74ea5912..bdb5592c 100644 --- a/docs/design/metamodel-wiring.md +++ b/docs/design/metamodel-wiring.md @@ -1,20 +1,15 @@ # Metamodel wiring: turning governance on -The governance pieces — stamps, diffs, drift checks, quarantine — are ordinary objects with -constructors. This note is about the last mile: how a Spring Boot application gets them without -assembling them by hand, and why turning them on is a thing you have to *do* rather than a thing -that happens to you. +The governance pieces (stamps, diffs, drift checks, quarantine) are ordinary objects with +constructors. This note covers how a Spring Boot application gets them without assembling them by +hand, and what it has to do to switch them on. -## The bargain: bring a declared schema, get the plumbing +## The opt-in is a declared schema -`MetamodelAutoConfiguration` activates when — and only when — the application supplies a -`DeclaredSchemaSource` bean. - -That is the same deal Spring Boot makes with JPA. Put a `DataSource` on the context and an -`EntityManagerFactory`, a transaction manager, and a repository infrastructure appear behind it. -Leave it out and none of that machinery exists; it isn't disabled, it was never created. Declaring a -schema is the DICE equivalent: the one decision only the application can make, and the signal that -it wants everything downstream of it. +`MetamodelAutoConfiguration` activates when the application supplies a `DeclaredSchemaSource` bean, +and only then. It is the same arrangement Spring Boot has with JPA: a `DataSource` on the context +brings an `EntityManagerFactory` and the repository infrastructure with it, and leaving the +`DataSource` out means that machinery was never created. ```mermaid flowchart TD @@ -25,24 +20,24 @@ flowchart TD B -- true --> C["Version store, drift log,
observed-schema source,
differ, quarantine policy,
schema constraints"] C --> D{"embabel.dice.metamodel
.drift.mode"} D -- off --> E["No runner.
Stamps and history only."] - D -- "observe (default)" --> F["ObserveOnlyDriftCheckRunner
reports; never touches a proposition"] + D -- "observe (default)" --> F["ObserveOnlyDriftCheckRunner
reports; touches no proposition"] D -- quarantine --> G["DefaultDriftCheckRunner
run(dryRun = false) marks stranded
propositions STALE"] ``` -Requiring the bean is not ceremony. There is no sensible default declared schema, and both ways of -guessing one are bad. Govern everything in the `DataDictionary` and every exploratory type an -extractor invented becomes reported drift, which trains everyone to ignore the reports. Govern -nothing and the loop is a no-op that still writes reassuring log lines. Making the application say -what it governs puts that decision in the consumer's own code, where a reader can see it. +There is no sensible default declared schema, and both ways of guessing one are bad. Govern +everything in the `DataDictionary` and every exploratory type an extractor invented becomes reported +drift, which trains everyone to ignore the reports. Govern nothing and the loop is a no-op that +still writes log lines saying it ran. Making the application say what it governs puts that decision +in the consumer's own code, where a reader can see it. -`embabel.dice.metamodel.enabled=false` sits on top as a kill switch — the way to switch governance -off in one environment without deleting the bean. It is only consulted once the bean exists, so it +`embabel.dice.metamodel.enabled=false` is a kill switch on top of that, for switching governance off +in one environment while the bean stays in place. It is consulted only once the bean exists, so it changes nothing for an application that never declared a schema. -## Tiers, and why quarantine is never the default +## Drift tiers -`embabel.dice.metamodel.drift.mode` picks how far a check may go. It mirrors the three tiers in -[metamodel-drift.md](metamodel-drift.md), one property value each: +`embabel.dice.metamodel.drift.mode` sets how far a check may go. It maps the three tiers in +[metamodel-drift.md](metamodel-drift.md) onto one property value each: | Mode | What you get | What it can change | | --- | --- | --- | @@ -50,37 +45,36 @@ changes nothing for an application that never declared a schema. | `observe` (default) | A runner that checks and writes reports | Nothing | | `quarantine` | The real runner | Marks stranded propositions `STALE`, with a reason | -Reporting is safe to leave running forever, so it is the default. Changing proposition state is a -decision somebody has to make on purpose, so it is not. +The default is `observe`. Reporting is safe to leave running indefinitely; changing proposition +state is a decision somebody makes on purpose. -The asymmetry is about which mistakes are recoverable. A typo in a declared type name under -`observe` produces a report naming a type you expected to see — annoying, five minutes to fix. -The same typo under `quarantine` marks every proposition mentioning that type stale on the next -scheduled check. Quarantine is non-destructive, so that is recoverable too, but only if somebody -notices. Defaults should fail in the direction where nobody has to notice. +Which mistakes are recoverable drives that choice. A typo in a declared type name under `observe` +produces a report naming a type you expected to see, which takes five minutes to fix. The same typo +under `quarantine` marks every proposition mentioning that type stale on the next check. Quarantine +is non-destructive, so that is recoverable too, but only if somebody notices. -`observe` is enforced by wiring rather than by convention. The bean you get is an -`ObserveOnlyDriftCheckRunner` wrapping the real one, and it downgrades `run(dryRun = false)` to a dry -run, logs a warning, and returns a result whose `dryRun` is `true`. A dry-run *default* only protects -the caller who passes no arguments; a scheduler, an admin endpoint, or one line in a script all reach -straight past it. Making the tier a property of the bean means the guarantee holds no matter who -calls, and moving between tiers is a config change rather than a code change. +The `observe` guarantee lives in the wiring. The bean you get is an `ObserveOnlyDriftCheckRunner` +wrapping the real one; it downgrades `run(dryRun = false)` to a dry run, logs a warning, and returns +a result whose `dryRun` is `true`. A dry-run default on the method only protects the caller who +passes no arguments, and a scheduler, an admin endpoint, or one line in a script all reach straight +past it. Making the tier a property of the bean holds the guarantee whoever calls, and moving +between tiers is then a config change. -Nothing is scheduled here. This wires the *capability* to run a check; when one runs is the +Nothing is scheduled here. This wires the capability to run a check; when one runs is the application's call. -## Your bean always wins, whatever the order +## Replacing a default Every default is `@ConditionalOnMissingBean`, and `MetamodelAutoConfiguration` is a real `@AutoConfiguration` listed in `META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`. Both halves matter, and the second is the one that is easy to get wrong. `@ConditionalOnMissingBean` -asks "does a bean of this type exist *yet*?" — a question whose answer depends on when it is asked. -A plain `@Configuration` picked up by component scanning carries the same annotations but is -processed in scan order, so whether the consumer's bean has been registered by then is luck. Spring -Boot registers every application bean definition *before* it processes a single auto-configuration, -so registering through the imports file turns "your bean wins" from a coincidence into a guarantee. +asks whether a bean of this type exists *yet*, so the answer depends on when it is asked. A plain +`@Configuration` picked up by component scanning carries the same annotations but is processed in +scan order, which leaves it to luck whether the consumer's bean has been registered by then. Spring +Boot registers every application bean definition before it processes a single auto-configuration, so +registering through the imports file makes "your bean wins" a guarantee. `MetamodelAutoConfigurationTest` checks it from both directions, registering the consumer's configuration before and after the auto-configuration and asserting the same result. @@ -96,44 +90,40 @@ The replaceable pieces: | `DriftCheckRunner` | per `drift.mode`, above | — | The differ is registered under its concrete type, because `StructuralMetamodelDiffer` answers two -different questions — declaration against declaration (`MetamodelDiffer`) and declaration against a -live graph (`DeclaredObservedDiffer`) — and both need to resolve. Backing off keys on -`DeclaredObservedDiffer` alone, since that is the collaborator the runner needs. Supplying only a -`MetamodelDiffer`, which is a legitimate thing to want on its own, leaves drift checking working -rather than refusing to start. +questions — declaration against declaration (`MetamodelDiffer`) and declaration against a live graph +(`DeclaredObservedDiffer`) — and both need to resolve. Backing off keys on `DeclaredObservedDiffer` +alone, since that is the collaborator the runner needs. An application supplying only a +`MetamodelDiffer`, which is a legitimate thing to want on its own, still gets a working drift check. The metamodel uniqueness constraints ride along as a `SchemaCatalog` bean, the same way the proposition and lineage constraints do; Drivine's `SchemaManager` applies them idempotently on -startup. They are not optional — the stores' MERGEs are only race-free under them — so unlike the -store beans they carry no `@ConditionalOnMissingBean`. Catalogs accumulate rather than compete. +startup. The stores' MERGEs are only race-free under them, so that bean carries no +`@ConditionalOnMissingBean`. Catalogs accumulate, so an application adding its own gets both. ## The runner takes the narrow port -`DefaultDriftCheckRunner` is wired against `PropositionStore`, the base persistence port — not -`PropositionRepository`, which adds vector search, graph traversal, temporal query, and core search -operations on top. - -A drift check reads propositions by context or in bulk and saves the flagged copies back. That is -all of it. Asking for the wider interface at the wiring layer would shut a plain store-and-retrieve -backend out of governance over capabilities it is never asked to use, and it would do so invisibly: -the context would simply fail to find a bean. A `PropositionRepository` satisfies the narrow -parameter anyway, so nothing is given up by asking for less. +`DefaultDriftCheckRunner` is wired against `PropositionStore`, the base persistence port. +`PropositionRepository` adds vector search, graph traversal, temporal query, and core search +operations on top, and a drift check uses none of them: it reads propositions by context or in bulk +and saves the flagged copies back. -## Failing loudly +Asking for the wider interface at the wiring layer would shut a plain store-and-retrieve backend out +of governance over capabilities it is never asked to use, and would do so as a missing +`PropositionRepository` bean at startup. A `PropositionRepository` satisfies the narrow parameter, +so asking for less costs nothing. -The defaults are Drivine/Neo4j-backed, and they are not gated on a `PersistenceManager` being -present. An application that declared a schema without a graph connection fails at startup with the -missing bean named. +## A missing PersistenceManager fails startup -That is deliberate. The alternative — quietly wiring nothing when the connection is missing — leaves -somebody believing governance is running when it isn't, which is the one outcome a governance -feature must never produce. An application that genuinely wants the loop without Drivine supplies its -own three stores; they are `@ConditionalOnMissingBean` like everything else, and then no -`PersistenceManager` is asked for at all. +The defaults are Drivine/Neo4j-backed and are not gated on a `PersistenceManager` being present. An +application that declared a schema without a graph connection fails at startup with the missing bean +named. Wiring nothing when the connection is missing would leave somebody believing governance is +running when it isn't. An application that wants the loop without Drivine supplies its own three +stores; they are `@ConditionalOnMissingBean` like everything else, and then no `PersistenceManager` +is asked for at all. ## Property reference | Property | Default | Meaning | | --- | --- | --- | -| `embabel.dice.metamodel.enabled` | `true` | Kill switch. Only consulted when a `DeclaredSchemaSource` bean exists | +| `embabel.dice.metamodel.enabled` | `true` | Kill switch. Consulted only when a `DeclaredSchemaSource` bean exists | | `embabel.dice.metamodel.drift.mode` | `observe` | `off`, `observe`, or `quarantine` — see above | From 6e6582c65bc83278bd8be5905ad844e51a24ce4f Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Tue, 1 Sep 2026 15:22:11 -0400 Subject: [PATCH 03/10] Fix the autoconfiguration's composition and drop live-mode remnants The runner is evaluate-and-report only, so ObserveOnlyDriftCheckRunner had nothing left to refuse and is gone, and DriftMode reduces to OFF and OBSERVE: off suppresses the runner bean, and no property DICE reads can move a proposition. The wiring fixes follow the reviewer's findings: the application's event listener reaches every governance collaborator that can emit a status transition, so a deliberate quarantine drives ProjectionLineageStaleCascade; Drivine-backed governance beans register only under the graph backend, so a DeclaredSchemaSource under the default in-memory backend starts cleanly; and MetamodelDiffer and DeclaredObservedDiffer resolve independently, so distinct consumer beans are both honored. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- CHANGELOG.md | 40 +- .../MetamodelAutoConfiguration.kt | 284 +++++++----- .../autoconfigure/MetamodelProperties.kt | 34 +- .../ObserveOnlyDriftCheckRunner.kt | 55 --- ...tamodelAutoConfigurationIntegrationTest.kt | 59 ++- .../MetamodelAutoConfigurationTest.kt | 424 +++++++++++++++--- .../autoconfigure/MetamodelTestFixtures.kt | 22 +- docs/design/metamodel-wiring.md | 172 ++++--- 8 files changed, 751 insertions(+), 339 deletions(-) delete mode 100644 dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt diff --git a/CHANGELOG.md b/CHANGELOG.md index bbb0c0a0..ef9c2bde 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -549,21 +549,31 @@ and the consumer PRs that deliver it). checked exception commits, and custom rollback rules can override either behaviour. - Spring Boot auto-configuration for schema governance: `MetamodelAutoConfiguration` in `dice-storage-autoconfigure`. It registers only when the application supplies a - `DeclaredSchemaSource` bean, and then wires the whole loop against the existing Drivine - `PersistenceManager`: version store, drift-report store, observed-schema source, differ, - quarantine policy, drift runner, and a `SchemaCatalog` carrying the six metamodel uniqueness - constraints. Every wired collaborator is `@ConditionalOnMissingBean`, so an application that - defines its own keeps it. Settings live under `embabel.dice.metamodel`: `enabled=false` removes - the beans in one environment without deleting the declared-schema bean, and `drift.mode` is - `off`, `observe` or `quarantine`, defaulting to `observe`. Under `observe` the registered runner - is an `ObserveOnlyDriftCheckRunner`, which downgrades `run(dryRun = false)` to a dry run, logs - the downgrade, and returns a report with `dryRun = true`. Under `quarantine` the runner is - `DefaultDriftCheckRunner` and a live run moves stranded propositions to `STALE`. Nothing runs on - a schedule; a caller decides when a check happens. `DrivineObservedSchemaSource` also excludes - Drivine's own `_DrivineSchema` inventory label by shape. - **Compatibility: additive.** An application with no `DeclaredSchemaSource` bean sees no behavior - change. One with one needs the six metamodel constraints, which the module's `SchemaCatalog` - bean supplies, and a `PersistenceManager` and `PropositionStore` on the context. + `DeclaredSchemaSource` bean, and then wires the loop: version store, drift-report store, + observed-schema source, the two differ roles, quarantine policy, a `DriftSweepCapable`, the drift + runner, and a `SchemaCatalog` carrying the metamodel uniqueness constraints. Every wired + collaborator is `@ConditionalOnMissingBean`, so an application that defines its own keeps it. + Settings live under `embabel.dice.metamodel`: `enabled=false` removes the beans in one + environment while the declared-schema bean stays in place, and `drift.mode` is `off` or `observe` + (the default), which picks whether a `DriftCheckRunner` bean is registered. + Backend selection follows `embabel.dice.store.type`, the same switch the proposition store reads. + Under `graph` the Drivine/Neo4j version store, drift log and observed-schema source are wired. + Under the default in-memory backend an application that declares a schema still starts with no + `PersistenceManager` anywhere: it gets `InMemoryMetamodelVersionStore`, the differ, the policy and + the sweep, and it gets no drift log, no observed-schema source and no runner, because there is no + live graph to observe. + Nothing in the wiring runs a check or moves a proposition. A check happens when the application + calls `DriftCheckRunner.run()`, and it reads, compares and writes a `DriftReport` and touches no + proposition. Quarantine happens when the application calls `DriftSweepCapable.sweep` on a diff it + decided to act on; there is no scheduler and no property that makes DICE sweep by itself. The + wired sweep announces each status transition to every `DiceEventListener` bean on the context + through a `CompositeDiceEventListener`, so a registered `ProjectionLineageStaleCascade` marks the + projection records derived from a quarantined proposition stale. + **Compatibility: additive.** No symbol that exists on the previous release changes shape or + behavior. An application with no `DeclaredSchemaSource` bean sees no change at all. One that + declares a schema and selects the graph backend needs the metamodel constraints, which the + module's `SchemaCatalog` bean supplies, and a `PersistenceManager` on the context; a + `PropositionStore` brings the sweep with it, and its absence leaves the rest of the loop working. - Optional source revisions in the `dice` core provenance model, the first slice of DICE #64. `ProvenanceEntry` gains a sixth field, `sourceRevision`: an opaque, provider-defined string, diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt index b421e9a4..0e77684b 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt @@ -15,15 +15,22 @@ */ package com.embabel.dice.storage.autoconfigure +import com.embabel.dice.common.CompositeDiceEventListener +import com.embabel.dice.common.DiceEventListener import com.embabel.dice.metamodel.DeclaredObservedDiffer import com.embabel.dice.metamodel.DeclaredSchemaSource import com.embabel.dice.metamodel.DriftCheckRunner import com.embabel.dice.metamodel.DriftQuarantinePolicy import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.DriftSweepCapable +import com.embabel.dice.metamodel.InMemoryMetamodelVersionStore +import com.embabel.dice.metamodel.MetamodelDiffer import com.embabel.dice.metamodel.MetamodelVersionStore import com.embabel.dice.metamodel.ObservedSchemaSource +import com.embabel.dice.metamodel.SweptBaselineStore import com.embabel.dice.metamodel.support.DefaultDriftCheckRunner import com.embabel.dice.metamodel.support.MentionTypeDriftQuarantinePolicy +import com.embabel.dice.metamodel.support.PropositionStoreDriftSweep import com.embabel.dice.metamodel.support.StructuralMetamodelDiffer import com.embabel.dice.proposition.PropositionStore import com.embabel.dice.storage.DrivineDriftReportStore @@ -33,6 +40,7 @@ import com.embabel.dice.storage.MetamodelSchema import org.drivine.manager.PersistenceManager import org.drivine.schema.SchemaCatalog import org.slf4j.LoggerFactory +import org.springframework.beans.factory.ObjectProvider import org.springframework.boot.autoconfigure.AutoConfiguration import org.springframework.boot.autoconfigure.condition.ConditionalOnBean import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean @@ -42,13 +50,14 @@ import org.springframework.context.annotation.Bean /** * Wires the schema-governance loop: version stamps, a drift log, a snapshot of what the live graph - * holds, the comparison between the two, and the runner that sequences them. + * holds, the comparisons between them, the runner that sequences a check, and the sweep a host calls + * when it decides to act on one. * * ## The opt-in is a declared schema * * Governance activates when the application supplies a `DeclaredSchemaSource` bean, and only then. - * With no declared schema there are no metamodel beans at all: no stores, no differ, no runner. It - * is the same arrangement Spring Boot has with JPA and a `DataSource`. + * With no declared schema there are no metamodel beans at all: no stores, no differ, no runner, no + * sweep. It is the same arrangement Spring Boot has with JPA and a `DataSource`. * * There is no sensible default declared schema. A schema states what an application governs, and * guessing one would either govern everything, turning every exploratory type an LLM extracted into @@ -59,24 +68,41 @@ import org.springframework.context.annotation.Bean * off in one environment while the bean stays in place. It is consulted only once the bean exists, * so it changes nothing for an application that never declared a schema. * - * ## Drift tiers + * ## Nothing here runs, and nothing here quarantines * - * `embabel.dice.metamodel.drift.mode` sets how far a check may go: + * Every bean below is a capability sitting still. There is no scheduler and no startup hook: a check + * happens when the application calls [DriftCheckRunner.run], from a cron, an admin endpoint or a + * migration script. A check reads, compares and writes a report, and no path through it moves a + * proposition. * - * - `off` — stores and stamps only. Nothing compares a declaration against a graph. - * - `observe` (**default**) — checks run and reports are written; no proposition is touched. The - * runner is an [ObserveOnlyDriftCheckRunner], so the guarantee holds against a caller who passes - * `dryRun = false`. - * - `quarantine` — the real [DefaultDriftCheckRunner]. `run(dryRun = false)` moves stranded - * propositions to `STALE` with a reason. + * Quarantine is a second, deliberate step. The [DriftSweepCapable] bean below is what performs it, + * and it performs it when a host calls [DriftSweepCapable.sweep] with a diff it decided to act on. + * Nothing in this class calls that method. * - * The default is `observe`. Reporting is safe to leave running indefinitely; changing proposition - * state is a decision somebody makes on purpose. A mistyped type name in a declared schema costs - * five minutes under `observe` and marks every proposition mentioning that type stale under - * `quarantine`, so the escalation is opt-in in the direction where mistakes are recoverable. + * `embabel.dice.metamodel.drift.mode` picks whether the runner bean is registered at all: `off` for + * stamps and history alone, `observe` (the default) for the check as well. See [DriftMode]. * - * Nothing here schedules anything. This wires the capability to run a check; when a check runs is - * the application's call, from a cron, an admin endpoint, or a startup hook. + * ## Status transitions reach the application's listeners + * + * A sweep moves a proposition to `STALE`, and things downstream need to hear about it — + * `ProjectionLineageStaleCascade` marks the projection records derived from that proposition stale, + * and it can only do so if the transition is announced. So the sweep is built with every + * `DiceEventListener` bean the application registered, fanned out through a + * [CompositeDiceEventListener], which makes each delivery exception-safe. An application with no + * listener bean gets [DiceEventListener.DEV_NULL] and the same behaviour otherwise. + * + * ## Backend selection follows the store + * + * The Drivine/Neo4j governance beans register under `embabel.dice.store.type=graph`, the same switch + * [DiceStorageAutoConfiguration] uses for the proposition store, and they are declared before their + * in-memory counterparts so the fallback resolves by registration order. + * + * Under the default in-memory backend a host that declares a schema still starts. It gets the + * in-memory version store, the differ and the quarantine policy, and it gets the sweep when a + * `PropositionStore` is on the context. It gets no drift log and no observed-schema source, because + * both of those read a graph and there is no graph here, and therefore no drift-check runner: with + * nothing to observe, a check has no question to ask. Such a host can stamp schemas, compare + * declarations and sweep on a diff it built itself. * * ## Defaults, and replacing them * @@ -86,19 +112,9 @@ import org.springframework.context.annotation.Bean * definition before it processes a single auto-configuration, so a consumer's bean is already there * when these conditions are evaluated, whichever order the two were declared in. * - * The defaults are Drivine/Neo4j-backed, so this needs the same `PersistenceManager` the rest of - * `dice-storage` uses. An application that declared a schema without a graph connection fails at - * startup with the missing bean named. Wiring nothing instead would leave someone believing - * governance was running when it wasn't. - * - * To replace the differ, supply a `DeclaredObservedDiffer` bean, which is the one the runner needs. - * The shipped [StructuralMetamodelDiffer] answers both questions (declaration vs declaration, and - * declaration vs live graph), so it is registered under its concrete type and resolves as either - * interface. - * * The metamodel constraints ride along as a [SchemaCatalog] bean, the same way the proposition and * lineage constraints do; Drivine's `SchemaManager` applies them idempotently on startup. The - * stores' MERGEs are only race-free under them, so they are required. + * stores' MERGEs are only race-free under them, so they are required wherever the graph stores are. */ @AutoConfiguration(after = [DiceStorageAutoConfiguration::class]) @ConditionalOnBean(DeclaredSchemaSource::class) @@ -113,50 +129,88 @@ class MetamodelAutoConfiguration { private val logger = LoggerFactory.getLogger(MetamodelAutoConfiguration::class.java) + // ---- Graph backend (embabel.dice.store.type=graph) ---- + /** * The uniqueness constraints the governance stores need. Declared as data in `dice-storage` so * the constraint list and the label list the observed-schema source excludes stay one edit * apart. * - * No `@ConditionalOnMissingBean`, matching the other schema beans in this module. Catalogs - * accumulate, so a consumer adding their own gets both, and these are still required. + * Graph-only, since it is DDL for the Drivine stores below. No `@ConditionalOnMissingBean`, + * matching the other schema beans in this module. Catalogs accumulate, so a consumer adding + * their own gets both, and these are still required. */ @Bean + @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") fun metamodelSchema(): SchemaCatalog = SchemaCatalog.of(MetamodelSchema.specs()) + /** + * The version store, returned as a [SweptBaselineStore] on purpose. + * + * Both shipped stores track the reconciled baseline, and a host needs that capability by name: + * after it has swept every context it meant to reconcile it calls + * [SweptBaselineStore.markSwept] itself. Spring predicts a bean's type from the factory method's + * return type, so declaring the narrower [MetamodelVersionStore] here would hide the baseline + * capability at injection time and leave a `SweptBaselineStore` injection point unresolvable. + */ @Bean + @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(MetamodelVersionStore::class) - fun metamodelVersionStore(persistenceManager: PersistenceManager): MetamodelVersionStore { - logger.debug("Wiring default MetamodelVersionStore: DrivineMetamodelVersionStore") + fun drivineMetamodelVersionStore(persistenceManager: PersistenceManager): SweptBaselineStore { + logger.debug("Wiring graph MetamodelVersionStore: DrivineMetamodelVersionStore") return DrivineMetamodelVersionStore(persistenceManager) } @Bean + @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(DriftReportStore::class) - fun driftReportStore(persistenceManager: PersistenceManager): DriftReportStore { - logger.debug("Wiring default DriftReportStore: DrivineDriftReportStore") + fun drivineDriftReportStore(persistenceManager: PersistenceManager): DriftReportStore { + logger.debug("Wiring graph DriftReportStore: DrivineDriftReportStore") return DrivineDriftReportStore(persistenceManager) } @Bean + @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(ObservedSchemaSource::class) - fun observedSchemaSource(persistenceManager: PersistenceManager): ObservedSchemaSource { - logger.debug("Wiring default ObservedSchemaSource: DrivineObservedSchemaSource") + fun drivineObservedSchemaSource(persistenceManager: PersistenceManager): ObservedSchemaSource { + logger.debug("Wiring graph ObservedSchemaSource: DrivineObservedSchemaSource") return DrivineObservedSchemaSource(persistenceManager) } + // ---- In-memory backend (default) ---- + + /** + * Schema history for a host with no graph. It keeps stamps and the swept baseline in the JVM, so + * a declaration can be stamped and compared against its predecessor before there is a database. + * + * There is no in-memory counterpart to the other two graph beans, and that is the honest answer + * for both. A drift log is a durable record an operator reads days later, and a heap map is not + * one. An observed-schema source reports what a live graph holds, and a host with no graph has + * no graph to report on. + */ + @Bean + @ConditionalOnMissingBean(MetamodelVersionStore::class) + fun inMemoryMetamodelVersionStore(): SweptBaselineStore { + logger.debug("Wiring in-memory MetamodelVersionStore: InMemoryMetamodelVersionStore") + return InMemoryMetamodelVersionStore() + } + + // ---- Pure-JVM collaborators, both backends ---- + /** * The shipped differ, registered under its concrete type so it resolves as both a - * `MetamodelDiffer` (two declarations compared) and a [DeclaredObservedDiffer] (a declaration + * [MetamodelDiffer] (two declarations compared) and a [DeclaredObservedDiffer] (a declaration * compared against a live graph). * - * Backing off keys on [DeclaredObservedDiffer] alone, because that is the collaborator the - * runner needs. A consumer who supplies their own drift differ takes this one's place entirely. - * A consumer who supplies only a `MetamodelDiffer`, which is a legitimate thing to want on its - * own, still gets a working drift check. + * It backs off as soon as the application supplies either one. That looks strict for a host that + * supplied only a [MetamodelDiffer], since the declaration-against-graph question is then + * unanswered by any bean — but [driftCheckRunner] fills that role with its own + * [StructuralMetamodelDiffer] and both differs are honoured. Registering this bean anyway would + * put two [MetamodelDiffer] candidates on the context, and the consumer's bean would lose to an + * ambiguity nobody asked for. */ @Bean - @ConditionalOnMissingBean(DeclaredObservedDiffer::class) + @ConditionalOnMissingBean(value = [DeclaredObservedDiffer::class, MetamodelDiffer::class]) fun structuralMetamodelDiffer(): StructuralMetamodelDiffer { logger.debug("Wiring default differ: StructuralMetamodelDiffer") return StructuralMetamodelDiffer() @@ -169,95 +223,113 @@ class MetamodelAutoConfiguration { return MentionTypeDriftQuarantinePolicy() } + // ---- The sweep, for a host that decides to act on a check ---- + /** - * The `quarantine` tier: the real runner, which honours `run(dryRun = false)`. + * The reference sweep over whatever [PropositionStore] the application has, wired with the + * application's own event listeners so a quarantine it performs is heard downstream. + * + * Backs off when a bean already implements [DriftSweepCapable], which is how a durable store + * that can push the candidate query down into its backend takes over: it implements the + * interface itself and this default steps aside. * - * Mutually exclusive with [observeOnlyDriftCheckRunner] by property value, so which one you get - * doesn't depend on the order the two are listed in. + * [propositionStore] is the base persistence port. A sweep reads one context and saves, and uses + * none of the vector search, graph traversal or temporal query `PropositionRepository` adds, so + * asking for the wider interface would shut a plain store-and-retrieve backend out of governance + * over capabilities it never touches. A `PropositionRepository` satisfies this parameter anyway. + * + * Registering this bean sweeps nothing. It is the object a host calls when it has read a check + * and decided to act. + * + * @param listeners Every `DiceEventListener` bean the application registered. Each status + * transition the sweep performs is announced to all of them, which is what lets + * `ProjectionLineageStaleCascade` mark the projection records derived from a quarantined + * proposition stale. */ @Bean - @ConditionalOnMissingBean(DriftCheckRunner::class) - @ConditionalOnProperty( - prefix = "embabel.dice.metamodel.drift", - name = ["mode"], - havingValue = "quarantine", - ) - fun quarantiningDriftCheckRunner( - declaredSchemaSource: DeclaredSchemaSource, - versionStore: MetamodelVersionStore, - observedSchemaSource: ObservedSchemaSource, - differ: DeclaredObservedDiffer, - driftReportStore: DriftReportStore, - quarantinePolicy: DriftQuarantinePolicy, + @ConditionalOnBean(PropositionStore::class) + @ConditionalOnMissingBean(DriftSweepCapable::class) + fun propositionStoreDriftSweep( propositionStore: PropositionStore, - ): DriftCheckRunner { - logger.info( - "Metamodel drift checking wired in QUARANTINE mode: run(dryRun = false) will move " + - "stranded propositions to STALE. Nothing runs on a schedule; a caller decides when.", - ) - return defaultRunner( - declaredSchemaSource, versionStore, observedSchemaSource, differ, - driftReportStore, quarantinePolicy, propositionStore, - ) + listeners: ObjectProvider, + ): DriftSweepCapable { + val listener = compositeListener(listeners) + logger.debug("Wiring default DriftSweepCapable: PropositionStoreDriftSweep") + return PropositionStoreDriftSweep(propositionStore, listener) } + // ---- The check ---- + /** - * The default `observe` tier: the real runner behind an [ObserveOnlyDriftCheckRunner], so - * turning this context's checks live takes a property change. + * The shipped runner. It stamps the declaration, snapshots the graph, runs both comparisons and + * writes a report, and it touches no proposition on any path. + * + * Registered only when there is something to observe and somewhere to write the answer, which is + * why it is conditional on both [ObservedSchemaSource] and [DriftReportStore]. Under the default + * in-memory backend neither exists, and the context starts with the rest of the loop wired. + * + * ## Two differs, resolved independently + * + * A check asks two different questions and each has its own collaborator: [DeclaredObservedDiffer] + * compares the declaration against the live graph, and [MetamodelDiffer] compares the declaration + * against the baseline a sweep last reconciled. One object commonly answers both — the shipped + * [StructuralMetamodelDiffer] does — so the two are looked up separately and the same bean is + * free to satisfy each. An application that supplies a distinct bean for either role gets that + * bean used for that role, and whichever role no bean fills falls back to a + * [StructuralMetamodelDiffer] built here. Looking up one interface and testing the result for + * the other is what let a consumer's [MetamodelDiffer] be silently ignored. + * + * @param declaredObservedDiffers Candidates for the declaration-against-graph comparison. + * @param metamodelDiffers Candidates for the declaration-against-baseline comparison. */ @Bean @ConditionalOnMissingBean(DriftCheckRunner::class) + @ConditionalOnBean(value = [ObservedSchemaSource::class, DriftReportStore::class]) @ConditionalOnProperty( prefix = "embabel.dice.metamodel.drift", name = ["mode"], havingValue = "observe", matchIfMissing = true, ) - fun observeOnlyDriftCheckRunner( + fun driftCheckRunner( declaredSchemaSource: DeclaredSchemaSource, versionStore: MetamodelVersionStore, observedSchemaSource: ObservedSchemaSource, - differ: DeclaredObservedDiffer, driftReportStore: DriftReportStore, - quarantinePolicy: DriftQuarantinePolicy, - propositionStore: PropositionStore, + declaredObservedDiffers: ObjectProvider, + metamodelDiffers: ObjectProvider, ): DriftCheckRunner { - logger.info("Metamodel drift checking wired in OBSERVE mode: checks report, nothing is quarantined") - return ObserveOnlyDriftCheckRunner( - defaultRunner( - declaredSchemaSource, versionStore, observedSchemaSource, differ, - driftReportStore, quarantinePolicy, propositionStore, - ), + // One fallback object, shared by both roles and built only if a role needs it. + val fallback = lazy { StructuralMetamodelDiffer() } + val differ = declaredObservedDiffers.getIfUnique() ?: fallback.value + val metamodelDiffer = metamodelDiffers.getIfUnique() ?: fallback.value + + logger.info( + "Metamodel drift checking wired: a check reports and quarantines nothing. Nothing runs " + + "on a schedule; the application decides when to call it.", + ) + return DefaultDriftCheckRunner( + declaredSchemaSource = declaredSchemaSource, + versionStore = versionStore, + observedSchemaSource = observedSchemaSource, + differ = differ, + metamodelDiffer = metamodelDiffer, + driftReportStore = driftReportStore, ) } /** - * Builds the shipped runner. A plain function rather than a shared `@Bean`, because - * `@AutoConfiguration` runs with `proxyBeanMethods = false` — a `@Bean` method called from - * another one would build a second, unmanaged instance. Only one of the two runner beans is - * ever registered, so there is nothing to share anyway. - * - * [propositionStore] is the base `PropositionStore` port. A drift check reads propositions by - * context or in bulk and saves them back, and uses none of the vector search, graph traversal - * or temporal query that `PropositionRepository` adds. Asking for the wider interface here - * would shut a plain store-and-retrieve backend out of governance over capabilities it is - * never asked to use, and a `PropositionRepository` satisfies this parameter anyway. + * Every application listener as one listener. [CompositeDiceEventListener] wraps each delivery so + * a throwing listener can never abort the sweep that emitted the event, and an application with + * no listener bean gets the no-op. */ - private fun defaultRunner( - declaredSchemaSource: DeclaredSchemaSource, - versionStore: MetamodelVersionStore, - observedSchemaSource: ObservedSchemaSource, - differ: DeclaredObservedDiffer, - driftReportStore: DriftReportStore, - quarantinePolicy: DriftQuarantinePolicy, - propositionStore: PropositionStore, - ): DriftCheckRunner = DefaultDriftCheckRunner( - declaredSchemaSource = declaredSchemaSource, - versionStore = versionStore, - observedSchemaSource = observedSchemaSource, - differ = differ, - driftReportStore = driftReportStore, - quarantinePolicy = quarantinePolicy, - propositionStore = propositionStore, - ) + private fun compositeListener(listeners: ObjectProvider): DiceEventListener { + val all = listeners.orderedStream().toList() + if (all.isEmpty()) { + logger.debug("No DiceEventListener bean on the context; drift quarantine will announce to nobody") + return DiceEventListener.DEV_NULL + } + logger.debug("Drift quarantine will announce status transitions to {} listener(s)", all.size) + return CompositeDiceEventListener(all) + } } diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt index 4b9bf024..81148c7a 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt @@ -18,31 +18,28 @@ package com.embabel.dice.storage.autoconfigure import org.springframework.boot.context.properties.ConfigurationProperties /** - * How far the schema-governance loop is allowed to go. + * Whether the wiring registers a drift-check runner. * - * Governance escalates in tiers, and an operator picks the tier. Each tier is useful on its own: an - * application can report drift for a year and quarantine nothing. + * There are two settings because there are two useful answers. Some applications want schema + * history and nothing else; most want the check as well. Neither setting can quarantine anything: a + * check reads, compares and writes a report, and moving a proposition is a separate call a host + * makes on `DriftSweepCapable` at a moment it picks. */ enum class DriftMode { /** - * No drift checking at all. Stamps and stores are still wired, so an application can record and - * read schema versions, but nothing compares the declaration against a live graph. + * Stamps and stores only. The version store, the drift log and the differ are wired, so an + * application can record and read schema versions, and no runner bean is registered, so nothing + * compares the declaration against a live graph. */ OFF, /** - * The default. Check and report, touching no proposition. Ask the runner wired here for a live - * run and it downgrades to a dry run, logs the downgrade, and reports back `dryRun = true`. + * The default. A [com.embabel.dice.metamodel.DriftCheckRunner] bean is registered. Ask it for a + * check and it stamps the declaration, snapshots the graph, compares, and writes a drift report. + * It touches no proposition on any path. */ OBSERVE, - - /** - * Check, report, and let a caller quarantine. The runner honours `run(dryRun = false)`, which - * moves stranded propositions to `STALE` with a reason. Nothing runs on its own; a caller has - * to ask. - */ - QUARANTINE, } /** @@ -61,7 +58,7 @@ data class MetamodelProperties( */ val enabled: Boolean = true, - /** Drift checking: how far a check is allowed to go. */ + /** Drift checking: whether a runner is wired at all. */ val drift: DriftProperties = DriftProperties(), ) { @@ -69,11 +66,10 @@ data class MetamodelProperties( data class DriftProperties( /** - * The escalation tier: `off`, `observe` (the default), or `quarantine`. See [DriftMode]. + * `off` or `observe` (the default). See [DriftMode]. * - * The default is `observe`. Reporting is safe to leave running indefinitely; changing - * proposition state is a decision somebody makes on purpose. Defaulting to `quarantine` - * would let a mistyped schema strand real knowledge on the next check. + * `off` is for an application that wants schema stamps and history without a drift check + * running against its graph. */ val mode: DriftMode = DriftMode.OBSERVE, ) diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt deleted file mode 100644 index 69da004e..00000000 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/ObserveOnlyDriftCheckRunner.kt +++ /dev/null @@ -1,55 +0,0 @@ -/* - * Copyright 2024-2026 Embabel Pty Ltd. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.embabel.dice.storage.autoconfigure - -import com.embabel.agent.core.ContextId -import com.embabel.dice.metamodel.DriftCheckResult -import com.embabel.dice.metamodel.DriftCheckRunner -import org.slf4j.LoggerFactory - -/** - * A [DriftCheckRunner] that only observes. Checks run and reports are written; a request for a live - * run is downgraded to a dry one. - * - * This is what `embabel.dice.metamodel.drift.mode=observe` wires, so the observe tier is enforced by - * the wiring. A dry-run default on the method only protects the caller who passes no argument, and a - * scheduler, an admin endpoint, or a stray `run(dryRun = false)` in a script all reach past it. - * Wrapping the real runner holds the guarantee whoever calls and whatever they pass, and switching - * the tier is then a config change. - * - * The downgrade logs a warning and returns normally. Throwing would take down a scheduled job for - * asking a reasonable question in the wrong environment. The report is written, - * [DriftCheckResult.dryRun] comes back `true` and [DriftCheckResult.quarantinedCount] is `0`, so a - * caller can tell it was downgraded without reading the log. - * - * @param delegate The real runner, which decides everything except whether the run is live. - */ -internal class ObserveOnlyDriftCheckRunner( - private val delegate: DriftCheckRunner, -) : DriftCheckRunner { - - private val logger = LoggerFactory.getLogger(ObserveOnlyDriftCheckRunner::class.java) - - override fun run(dryRun: Boolean, contextId: ContextId?): DriftCheckResult { - if (!dryRun) { - logger.warn( - "Live drift check requested but the drift mode is 'observe'; running dry instead. " + - "Set embabel.dice.metamodel.drift.mode=quarantine to allow quarantining.", - ) - } - return delegate.run(dryRun = true, contextId = contextId) - } -} diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt index 109a5353..1d5b4f69 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt @@ -17,8 +17,11 @@ package com.embabel.dice.storage.autoconfigure import com.embabel.dice.metamodel.DeclaredSchemaSource import com.embabel.dice.metamodel.DriftCheckRunner +import com.embabel.dice.metamodel.DriftQuarantinePolicy import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.DriftSweepCapable import com.embabel.dice.metamodel.MetamodelVersionStore +import com.embabel.dice.metamodel.support.DefaultDriftCheckRunner import com.embabel.dice.proposition.PropositionStatus import com.embabel.dice.storage.DrivineDriftReportStore import com.embabel.dice.storage.DrivineMetamodelVersionStore @@ -51,12 +54,12 @@ import org.springframework.test.context.DynamicPropertySource * The drift comes from the graph: a `(:Ghost)` node nobody declared. The proposition side stays in * memory ([MapPropositionStore]), since this test is about the auto-configured Drivine stores and * the runner that sequences them, and `dice-storage`'s own integration tests already cover - * quarantining through a graph-backed repository. It also exercises the runner's use of the base - * `PropositionStore` port. + * quarantining through a graph-backed repository. It also exercises the wired sweep's use of the + * base `PropositionStore` port. */ @SpringBootTest( classes = [MetamodelIntegrationTestApplication::class], - properties = ["embabel.dice.metamodel.drift.mode=quarantine"], + properties = ["embabel.dice.store.type=graph"], ) class MetamodelAutoConfigurationIntegrationTest { @@ -78,6 +81,12 @@ class MetamodelAutoConfigurationIntegrationTest { @Autowired private lateinit var propositionStore: MapPropositionStore + @Autowired + private lateinit var driftSweep: DriftSweepCapable + + @Autowired + private lateinit var quarantinePolicy: DriftQuarantinePolicy + @Autowired private lateinit var persistenceManager: PersistenceManager @@ -98,14 +107,13 @@ class MetamodelAutoConfigurationIntegrationTest { fun `the auto-configured beans are the Drivine ones`() { assertThat(versionStore).isInstanceOf(DrivineMetamodelVersionStore::class.java) assertThat(driftReportStore).isInstanceOf(DrivineDriftReportStore::class.java) - assertThat(runner).isNotInstanceOf(ObserveOnlyDriftCheckRunner::class.java) + assertThat(runner).isInstanceOf(DefaultDriftCheckRunner::class.java) } @Test - fun `a live run stamps the schema, finds the undeclared type, and persists a report`() { - val result = runner.run(dryRun = false, contextId = null) + fun `a check stamps the schema, finds the undeclared type, and persists a report`() { + val result = runner.run() - assertThat(result.dryRun).isFalse() assertThat(result.driftedEntityTypes).contains("Ghost") // The stamp resolves out of the database by the hash the report carries. That is what @@ -120,9 +128,28 @@ class MetamodelAutoConfigurationIntegrationTest { assertThat(reports.single().driftedEntityTypes).contains("Ghost") assertThat(reports.single().versionHash).isEqualTo(result.report.versionHash) - // The tier did what it says: the proposition mentioning Ghost is quarantined, the one - // mentioning Person is left alone. - assertThat(result.quarantinedCount).isEqualTo(1) + // The check moved nothing. Every proposition is where it was. + assertThat(propositionStore.findByStatus(PropositionStatus.ACTIVE)).hasSize(2) + assertThat(propositionStore.findByStatus(PropositionStatus.STALE)).isEmpty() + } + + /** + * The second half of the loop, as a host performs it: read the check, decide, then call the + * sweep. The auto-configuration wires the sweep and never calls it, so this method is the only + * thing in the whole test that can move a proposition. + */ + @Test + fun `a host that acts on the check quarantines through the wired sweep`() { + val result = runner.run() + + val swept = driftSweep.sweep( + diff = result.quarantineDiff, + policy = quarantinePolicy, + contextId = MetamodelTestFixtures.CONTEXT_ID, + ) + + // The proposition mentioning Ghost is quarantined, the one mentioning Person is left alone. + assertThat(swept.quarantined).hasSize(1) assertThat(propositionStore.findByStatus(PropositionStatus.STALE)).hasSize(1) assertThat(propositionStore.findByStatus(PropositionStatus.ACTIVE)).hasSize(1) } @@ -130,12 +157,13 @@ class MetamodelAutoConfigurationIntegrationTest { /** * Stamping a version and writing a report both add nodes to the graph the next check observes. * If those labels came back as drift, every check would report drift caused by the previous - * check. + * check. `DiceOwnedSchema` in `dice-storage` is what keeps them out, and this asserts that the + * auto-configured stores and observed-schema source really do line up with it. * - * `_DrivineSchema` is the same problem from outside dice: Drivine's `SchemaManager` writes that - * inventory node when it applies a `SchemaCatalog`, and no application can silence it by - * declaring it. `DrivineObservedSchemaSource` excludes it by shape along with dice's own - * bookkeeping labels. + * Drivine's `SchemaManager` writes a `_DrivineSchema` inventory node whenever it applies a + * `SchemaCatalog`, and that label is currently reported as drift, since `DiceOwnedSchema` + * describes dice's own labels and knows nothing about Drivine's. Fixing it is a `dice-storage` + * change and there is no assertion for it here. */ @Test fun `governance never reports its own bookkeeping as drift`() { @@ -144,7 +172,6 @@ class MetamodelAutoConfigurationIntegrationTest { assertThat(second.driftedEntityTypes).contains("Ghost") assertThat(second.driftedEntityTypes).doesNotContainAnyElementsOf(MetamodelSchema.LABELS) - assertThat(second.driftedEntityTypes).doesNotContain("_DrivineSchema") } } diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt index ec4d6d71..b889535f 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationTest.kt @@ -24,15 +24,24 @@ import com.embabel.dice.metamodel.DriftCheckResult import com.embabel.dice.metamodel.DriftCheckRunner import com.embabel.dice.metamodel.DriftQuarantinePolicy import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.DriftSweepCapable +import com.embabel.dice.metamodel.InMemoryMetamodelVersionStore +import com.embabel.dice.metamodel.MetamodelChange import com.embabel.dice.metamodel.MetamodelDiff import com.embabel.dice.metamodel.MetamodelDiffer +import com.embabel.dice.metamodel.MetamodelVersion import com.embabel.dice.metamodel.MetamodelVersionStore import com.embabel.dice.metamodel.ObservedSchema import com.embabel.dice.metamodel.ObservedSchemaSource import com.embabel.dice.metamodel.QuarantineResult import com.embabel.dice.metamodel.support.DefaultDriftCheckRunner import com.embabel.dice.metamodel.support.MentionTypeDriftQuarantinePolicy +import com.embabel.dice.metamodel.support.PropositionStoreDriftSweep import com.embabel.dice.metamodel.support.StructuralMetamodelDiffer +import com.embabel.dice.projection.lineage.InMemoryProjectionRecordStore +import com.embabel.dice.projection.lineage.ProjectionLifecycle +import com.embabel.dice.projection.lineage.ProjectionLineageStaleCascade +import com.embabel.dice.projection.lineage.ProjectionRecord import com.embabel.dice.proposition.Proposition import com.embabel.dice.proposition.PropositionRepository import com.embabel.dice.proposition.PropositionStatus @@ -53,20 +62,22 @@ import org.springframework.core.io.support.PathMatchingResourcePatternResolver /** * Wiring tests for [MetamodelAutoConfiguration]. No Spring Boot app and no database: the - * auto-configuration, a stub `PersistenceManager`, and whichever governance beans a given test - * wants the application to have supplied. + * auto-configuration, a stub `PersistenceManager` where the graph backend is selected, and whichever + * governance beans a given test wants the application to have supplied. * * These cover what the wiring alone can get wrong: whether governance stays absent until somebody - * declares a schema, whether a consumer's bean wins whichever declaration order it arrives in, and - * whether each drift tier produces a runner that behaves like that tier. + * declares a schema, whether a consumer's bean wins whichever declaration order it arrives in, + * whether a host with no graph still starts, and whether a quarantine is heard by the listeners the + * application registered. */ class MetamodelAutoConfigurationTest { private val autoConfiguration = AutoConfigurations.of(MetamodelAutoConfiguration::class.java) - /** The application declared a schema and has a graph connection: the normal case. */ + /** The application declared a schema and selected the graph backend: the normal case. */ private val runner = ApplicationContextRunner() .withConfiguration(autoConfiguration) + .withPropertyValues(GRAPH_BACKEND) .withUserConfiguration(DeclaredSchemaConfig::class.java) // ---- The opt-in gate ---- @@ -75,6 +86,7 @@ class MetamodelAutoConfigurationTest { fun `no DeclaredSchemaSource bean means no metamodel beans at all`() { ApplicationContextRunner() .withConfiguration(autoConfiguration) + .withPropertyValues(GRAPH_BACKEND) .withUserConfiguration(NoDeclaredSchemaConfig::class.java) .run { ctx -> assertThat(ctx).hasNotFailed() @@ -85,6 +97,7 @@ class MetamodelAutoConfigurationTest { assertThat(ctx).doesNotHaveBean(ObservedSchemaSource::class.java) assertThat(ctx).doesNotHaveBean(DeclaredObservedDiffer::class.java) assertThat(ctx).doesNotHaveBean(DriftQuarantinePolicy::class.java) + assertThat(ctx).doesNotHaveBean(DriftSweepCapable::class.java) assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) assertThat(ctx).doesNotHaveBean(SchemaCatalog::class.java) } @@ -100,7 +113,8 @@ class MetamodelAutoConfigurationTest { assertThat(ctx.getBean()).isInstanceOf(DrivineObservedSchemaSource::class.java) assertThat(ctx.getBean()) .isInstanceOf(MentionTypeDriftQuarantinePolicy::class.java) - assertThat(ctx).hasSingleBean(DriftCheckRunner::class.java) + assertThat(ctx.getBean()).isInstanceOf(PropositionStoreDriftSweep::class.java) + assertThat(ctx.getBean()).isInstanceOf(DefaultDriftCheckRunner::class.java) assertThat(ctx).hasSingleBean(SchemaCatalog::class.java) } } @@ -125,15 +139,109 @@ class MetamodelAutoConfigurationTest { .run { ctx -> assertThat(ctx).hasNotFailed() assertThat(ctx).doesNotHaveBean(MetamodelAutoConfiguration::class.java) + assertThat(ctx).doesNotHaveBean(MetamodelProperties::class.java) assertThat(ctx).doesNotHaveBean(MetamodelVersionStore::class.java) + assertThat(ctx).doesNotHaveBean(DriftReportStore::class.java) + assertThat(ctx).doesNotHaveBean(ObservedSchemaSource::class.java) + assertThat(ctx).doesNotHaveBean(DeclaredObservedDiffer::class.java) + assertThat(ctx).doesNotHaveBean(MetamodelDiffer::class.java) + assertThat(ctx).doesNotHaveBean(DriftQuarantinePolicy::class.java) + assertThat(ctx).doesNotHaveBean(DriftSweepCapable::class.java) assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) + assertThat(ctx).doesNotHaveBean(SchemaCatalog::class.java) // The DeclaredSchemaSource the application supplied stays; only the metamodel // beans are removed. assertThat(ctx).hasSingleBean(DeclaredSchemaSource::class.java) } } - // ---- The differ, resolvable under both interfaces ---- + // ---- Backend selection ---- + + @Test + fun `a declared schema starts under the default in-memory backend, with no PersistenceManager`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryBackendConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).doesNotHaveBean(PersistenceManager::class.java) + + // What a host with no graph gets: schema history, the comparisons, the policy, and + // the sweep it can call once it has a diff. + assertThat(ctx.getBean()) + .isInstanceOf(InMemoryMetamodelVersionStore::class.java) + assertThat(ctx).hasSingleBean(StructuralMetamodelDiffer::class.java) + assertThat(ctx).hasSingleBean(DriftQuarantinePolicy::class.java) + assertThat(ctx.getBean()).isInstanceOf(PropositionStoreDriftSweep::class.java) + + // What it does not get, and why: there is no live graph to observe, so there is + // nothing for a check to ask about and no drift log to write the answer to. + assertThat(ctx).doesNotHaveBean(ObservedSchemaSource::class.java) + assertThat(ctx).doesNotHaveBean(DriftReportStore::class.java) + assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) + // The Neo4j constraints are DDL for the Drivine stores, which are not here either. + assertThat(ctx).doesNotHaveBean(SchemaCatalog::class.java) + } + } + + // ---- Status transitions reach the application's listeners ---- + + @Test + fun `a quarantine through the wired sweep drives the projection lineage cascade`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(CascadeListeningConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + val propositions = ctx.getBean() + val ghost = propositions.findAll().single { proposition -> + proposition.mentions.any { it.type == "Ghost" } + } + val records = ctx.getBean() + records.record( + ProjectionRecord( + propositionId = ghost.id, + target = "graph", + lifecycle = ProjectionLifecycle.PROJECTED, + runId = "run-1", + ), + ) + + // A host acting on a schema change that dropped `Ghost`. Nothing in DICE called this. + val result = ctx.getBean().sweep( + diff = MetamodelTestFixtures.diffRemoving("Ghost"), + policy = ctx.getBean(), + contextId = MetamodelTestFixtures.CONTEXT_ID, + ) + + assertThat(result.quarantined).hasSize(1) + assertThat(propositions.findById(ghost.id)!!.status).isEqualTo(PropositionStatus.STALE) + // The point of the test: the cascade heard the transition, so the projection record + // derived from that proposition is stale too. + assertThat(records.findByProposition(ghost.id).single().lifecycle) + .isEqualTo(ProjectionLifecycle.STALE) + } + } + + @Test + fun `the sweep still works when the application registered no listener at all`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(InMemoryBackendConfig::class.java) + .run { ctx -> + val propositions = ctx.getBean() + val result = ctx.getBean().sweep( + diff = MetamodelTestFixtures.diffRemoving("Ghost"), + policy = ctx.getBean(), + contextId = MetamodelTestFixtures.CONTEXT_ID, + ) + + assertThat(result.quarantined).hasSize(1) + assertThat(propositions.findByStatus(PropositionStatus.STALE)).hasSize(1) + } + } + + // ---- The two differ roles ---- @Test fun `the default differ resolves as both MetamodelDiffer and DeclaredObservedDiffer`() { @@ -144,6 +252,52 @@ class MetamodelAutoConfigurationTest { } } + @Test + fun `a distinct MetamodelDiffer bean and DeclaredObservedDiffer bean are both used`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(TwoDifferConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + // The declared-against-baseline comparison only runs when a sweep has completed, so + // give it a baseline to compare against. + ctx.getBean() + .markSwept(MetamodelTestFixtures.declaredSchema("Person", "Retired").version) + + val result = ctx.getBean().run() + + val declaredObserved = ctx.getBean() + val metamodel = ctx.getBean() + assertThat(declaredObserved.calls).isEqualTo(1) + assertThat(metamodel.calls).isEqualTo(1) + // Each answer reaches the report under its own heading, so neither differ can be + // standing in for the other. + assertThat(result.driftedEntityTypes).containsExactly(RecordingDeclaredObservedDiffer.MARKER) + assertThat(result.declaredDiff!!.removedEntityTypes) + .containsExactly(RecordingMetamodelDiffer.MARKER) + } + } + + @Test + fun `a consumer MetamodelDiffer alone still leaves the declared-against-graph comparison working`() { + ApplicationContextRunner() + .withConfiguration(autoConfiguration) + .withUserConfiguration(MetamodelDifferOnlyConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + ctx.getBean() + .markSwept(MetamodelTestFixtures.declaredSchema("Person", "Retired").version) + + val result = ctx.getBean().run() + + assertThat(ctx.getBean().calls).isEqualTo(1) + assertThat(result.declaredDiff!!.removedEntityTypes) + .containsExactly(RecordingMetamodelDiffer.MARKER) + // The shipped structural differ filled the role nobody supplied a bean for. + assertThat(result.driftedEntityTypes).containsExactly("Ghost") + } + } + // ---- Consumer beans win, in either declaration order ---- @Test @@ -173,6 +327,15 @@ class MetamodelAutoConfigurationTest { } } + @Test + fun `a consumer sweep wins whether it is registered before or after`() { + bothOrders(CustomSweepConfig::class.java) { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).hasSingleBean(DriftSweepCapable::class.java) + assertThat(ctx.getBean()).isInstanceOf(CustomSweep::class.java) + } + } + @Test fun `consumer stores win, and then no Drivine connection is needed at all`() { ApplicationContextRunner() @@ -185,57 +348,40 @@ class MetamodelAutoConfigurationTest { .isInstanceOf(RecordingMetamodelVersionStore::class.java) assertThat(ctx.getBean()).isInstanceOf(RecordingDriftReportStore::class.java) assertThat(ctx.getBean()).isInstanceOf(FixedObservedSchemaSource::class.java) + assertThat(ctx).hasSingleBean(DriftCheckRunner::class.java) } } // ---- The narrow proposition port ---- @Test - fun `the runner wires against a bare PropositionStore, with no PropositionRepository in sight`() { + fun `the sweep wires against a bare PropositionStore, with no PropositionRepository in sight`() { ApplicationContextRunner() .withConfiguration(autoConfiguration) .withUserConfiguration(InMemoryGovernanceConfig::class.java) .run { ctx -> assertThat(ctx).hasNotFailed() assertThat(ctx).doesNotHaveBean(PropositionRepository::class.java) - assertThat(ctx).hasSingleBean(DriftCheckRunner::class.java) + assertThat(ctx).hasSingleBean(DriftSweepCapable::class.java) } } - // ---- The drift tiers ---- + // ---- A check reports and changes nothing ---- @Test - fun `mode off leaves the stores wired and no runner`() { - ApplicationContextRunner() - .withConfiguration(autoConfiguration) - .withUserConfiguration(InMemoryGovernanceConfig::class.java) - .withPropertyValues("embabel.dice.metamodel.drift.mode=off") - .run { ctx -> - assertThat(ctx).hasNotFailed() - assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.OFF) - assertThat(ctx).hasSingleBean(MetamodelVersionStore::class.java) - assertThat(ctx).hasSingleBean(DriftReportStore::class.java) - assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) - } - } - - @Test - fun `observe is the default tier and downgrades a live run to a dry one`() { + fun `observe is the default mode and a check leaves every proposition alone`() { ApplicationContextRunner() .withConfiguration(autoConfiguration) .withUserConfiguration(InMemoryGovernanceConfig::class.java) .run { ctx -> assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.OBSERVE) - assertThat(ctx.getBean()).isInstanceOf(ObserveOnlyDriftCheckRunner::class.java) + assertThat(ctx.getBean()).isInstanceOf(DefaultDriftCheckRunner::class.java) - // Ask for a live run. 'Ghost' is observed and undeclared, so the quarantine tier - // would act here. - val result = ctx.getBean().run(dryRun = false, contextId = null) + // 'Ghost' is observed and undeclared, so there is real drift to report. + val result = ctx.getBean().run() - assertThat(result.dryRun).isTrue() assertThat(result.hasDrift).isTrue() assertThat(result.driftedEntityTypes).containsExactly("Ghost") - assertThat(result.quarantinedCount).isZero() val store = ctx.getBean() assertThat(store.saved).hasSize(1) @@ -247,50 +393,29 @@ class MetamodelAutoConfigurationTest { } @Test - fun `explicit observe mode wires the same observe-only runner`() { + fun `explicit observe mode wires the same runner`() { ApplicationContextRunner() .withConfiguration(autoConfiguration) .withUserConfiguration(InMemoryGovernanceConfig::class.java) .withPropertyValues("embabel.dice.metamodel.drift.mode=observe") .run { ctx -> - assertThat(ctx.getBean()).isInstanceOf(ObserveOnlyDriftCheckRunner::class.java) - } - } - - @Test - fun `quarantine mode wires the real runner and a live run really quarantines`() { - ApplicationContextRunner() - .withConfiguration(autoConfiguration) - .withUserConfiguration(InMemoryGovernanceConfig::class.java) - .withPropertyValues("embabel.dice.metamodel.drift.mode=quarantine") - .run { ctx -> - assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.QUARANTINE) assertThat(ctx.getBean()).isInstanceOf(DefaultDriftCheckRunner::class.java) - - val result = ctx.getBean().run(dryRun = false, contextId = null) - - assertThat(result.dryRun).isFalse() - assertThat(result.driftedEntityTypes).containsExactly("Ghost") - assertThat(result.quarantinedCount).isEqualTo(1) - - val store = ctx.getBean() - assertThat(store.findByStatus(PropositionStatus.STALE)).hasSize(1) - assertThat(store.findByStatus(PropositionStatus.ACTIVE)).hasSize(1) } } @Test - fun `quarantine mode still leaves a dry run harmless`() { + fun `mode off leaves the stores and the sweep wired, and no runner`() { ApplicationContextRunner() .withConfiguration(autoConfiguration) .withUserConfiguration(InMemoryGovernanceConfig::class.java) - .withPropertyValues("embabel.dice.metamodel.drift.mode=quarantine") + .withPropertyValues("embabel.dice.metamodel.drift.mode=off") .run { ctx -> - val result = ctx.getBean().run() - - assertThat(result.dryRun).isTrue() - assertThat(result.quarantinedCount).isZero() - assertThat(ctx.getBean().findByStatus(PropositionStatus.STALE)).isEmpty() + assertThat(ctx).hasNotFailed() + assertThat(ctx.getBean().drift.mode).isEqualTo(DriftMode.OFF) + assertThat(ctx).hasSingleBean(MetamodelVersionStore::class.java) + assertThat(ctx).hasSingleBean(DriftReportStore::class.java) + assertThat(ctx).hasSingleBean(DriftSweepCapable::class.java) + assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) } } @@ -333,15 +458,23 @@ class MetamodelAutoConfigurationTest { assertions: (org.springframework.boot.test.context.assertj.AssertableApplicationContext) -> Unit, ) { ApplicationContextRunner() + .withPropertyValues(GRAPH_BACKEND) .withUserConfiguration(DeclaredSchemaConfig::class.java, userConfiguration) .withConfiguration(autoConfiguration) .run(assertions) ApplicationContextRunner() + .withPropertyValues(GRAPH_BACKEND) .withConfiguration(autoConfiguration) .withUserConfiguration(DeclaredSchemaConfig::class.java, userConfiguration) .run(assertions) } + + private companion object { + + /** Selects the Drivine/Neo4j backend, the same switch the proposition store reads. */ + const val GRAPH_BACKEND = "embabel.dice.store.type=graph" + } } /** A graph-connected application that has declared a schema: the Drivine defaults apply. */ @@ -369,11 +502,58 @@ internal open class NoDeclaredSchemaConfig { open fun propositionStore(): MapPropositionStore = MapPropositionStore() } +/** + * An application on the default in-memory backend that declared a schema and nothing else. No + * `PersistenceManager` anywhere, which is the whole point: declaring a schema must not drag a graph + * connection in behind it. + */ +@Configuration(proxyBeanMethods = false) +internal open class InMemoryBackendConfig { + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun propositionStore(): MapPropositionStore = MapPropositionStore( + listOf( + MetamodelTestFixtures.proposition("Ada haunts the archive", mentionType = "Ghost"), + MetamodelTestFixtures.proposition("Ada wrote the notes", mentionType = "Person"), + ), + ) +} + +/** + * The same host, plus the lineage cascade registered as an ordinary `DiceEventListener` bean. This + * is what a real application does when it wants derived projection records to follow their + * proposition. + */ +@Configuration(proxyBeanMethods = false) +internal open class CascadeListeningConfig { + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun propositionStore(): MapPropositionStore = MapPropositionStore( + listOf( + MetamodelTestFixtures.proposition("Ada haunts the archive", mentionType = "Ghost"), + MetamodelTestFixtures.proposition("Ada wrote the notes", mentionType = "Person"), + ), + ) + + @Bean + open fun projectionRecordStore(): InMemoryProjectionRecordStore = InMemoryProjectionRecordStore() + + @Bean + open fun projectionLineageStaleCascade( + recordStore: InMemoryProjectionRecordStore, + ): ProjectionLineageStaleCascade = ProjectionLineageStaleCascade(recordStore) +} + /** * An application that brought its own governance stores, so no Drivine connection is needed and the * whole loop runs in memory. The observed schema holds a `Ghost` type the declaration never - * mentions, and one of the two propositions mentions it, so a live run has one thing to quarantine - * and one to leave alone. + * mentions, so a check has real drift to report. */ @Configuration(proxyBeanMethods = false) internal open class InMemoryGovernanceConfig { @@ -399,6 +579,94 @@ internal open class InMemoryGovernanceConfig { ) } +/** Records that it was asked, and answers with a name no other collaborator could have produced. */ +internal class RecordingDeclaredObservedDiffer : DeclaredObservedDiffer { + + var calls = 0 + private set + + override fun diffAgainstObserved(declared: DeclaredSchema, observed: ObservedSchema): DeclaredObservedDiff { + calls++ + return DeclaredObservedDiff( + declared = declared, + observedSchema = observed, + driftedEntityTypes = setOf(MARKER), + driftedRelationshipTypes = emptySet(), + unobservedEntityTypes = emptySet(), + unobservedRelationshipTypes = emptySet(), + ) + } + + companion object { + + const val MARKER = "AnsweredByTheConsumerDeclaredObservedDiffer" + } +} + +/** The same trick for the other role: its answer carries a name only it can produce. */ +internal class RecordingMetamodelDiffer : MetamodelDiffer { + + var calls = 0 + private set + + override fun diff(from: MetamodelVersion, to: MetamodelVersion): MetamodelDiff { + calls++ + return MetamodelDiff( + fromVersion = from, + toVersion = to, + changes = listOf(MetamodelChange.EntityTypeRemoved(MARKER)), + ) + } + + companion object { + + const val MARKER = "AnsweredByTheConsumerMetamodelDiffer" + } +} + +/** An application supplying a distinct bean for each of the two differ roles. */ +@Configuration(proxyBeanMethods = false) +internal open class TwoDifferConfig { + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun versionStore(): InMemoryMetamodelVersionStore = InMemoryMetamodelVersionStore() + + @Bean + open fun driftReportStore(): RecordingDriftReportStore = RecordingDriftReportStore() + + @Bean + open fun observedSchemaSource(): FixedObservedSchemaSource = FixedObservedSchemaSource() + + @Bean + open fun consumerDeclaredObservedDiffer(): RecordingDeclaredObservedDiffer = RecordingDeclaredObservedDiffer() + + @Bean + open fun consumerMetamodelDiffer(): RecordingMetamodelDiffer = RecordingMetamodelDiffer() +} + +/** An application that only wanted to replace the declaration-against-baseline comparison. */ +@Configuration(proxyBeanMethods = false) +internal open class MetamodelDifferOnlyConfig { + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun versionStore(): InMemoryMetamodelVersionStore = InMemoryMetamodelVersionStore() + + @Bean + open fun driftReportStore(): RecordingDriftReportStore = RecordingDriftReportStore() + + @Bean + open fun observedSchemaSource(): FixedObservedSchemaSource = FixedObservedSchemaSource() + + @Bean + open fun consumerMetamodelDiffer(): RecordingMetamodelDiffer = RecordingMetamodelDiffer() +} + internal class CustomDiffer : DeclaredObservedDiffer { override fun diffAgainstObserved(declared: DeclaredSchema, observed: ObservedSchema): DeclaredObservedDiff = DeclaredObservedDiff( @@ -419,6 +687,9 @@ internal open class CustomDifferConfig { } internal class CustomPolicy : DriftQuarantinePolicy { + + override fun candidateMentionTypes(diff: MetamodelDiff): Set = emptySet() + override fun evaluate(diff: MetamodelDiff, propositions: Iterable): QuarantineResult = QuarantineResult(conforming = emptyList(), quarantined = emptyList()) } @@ -431,7 +702,7 @@ internal open class CustomPolicyConfig { } internal class CustomRunner : DriftCheckRunner { - override fun run(dryRun: Boolean, contextId: ContextId?): DriftCheckResult = + override fun run(contextId: ContextId?): DriftCheckResult = throw UnsupportedOperationException("never called; this test only asks which bean won") } @@ -441,3 +712,26 @@ internal open class CustomRunnerConfig { @Bean open fun customRunner(): DriftCheckRunner = CustomRunner() } + +/** A store-side sweep of the kind a durable backend implements once it can push the query down. */ +internal class CustomSweep : DriftSweepCapable { + + override fun quarantineCandidates( + contextId: ContextId, + mentionTypes: Set, + limit: Int, + afterId: String?, + ): List = emptyList() + + override fun applyQuarantine(decision: com.embabel.dice.metamodel.QuarantineDecision.Quarantined): Proposition = + decision.proposition + + override fun releaseFromQuarantine(propositionId: String): Proposition? = null +} + +@Configuration(proxyBeanMethods = false) +internal open class CustomSweepConfig { + + @Bean + open fun customSweep(): DriftSweepCapable = CustomSweep() +} diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt index 9e254802..12d9c31c 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelTestFixtures.kt @@ -21,6 +21,8 @@ import com.embabel.dice.metamodel.DeclaredSchema import com.embabel.dice.metamodel.DeclaredSchemaSource import com.embabel.dice.metamodel.DriftReport import com.embabel.dice.metamodel.DriftReportStore +import com.embabel.dice.metamodel.MetamodelChange +import com.embabel.dice.metamodel.MetamodelDiff import com.embabel.dice.metamodel.MetamodelVersion import com.embabel.dice.metamodel.MetamodelVersionStore import com.embabel.dice.metamodel.ObservedSchema @@ -34,10 +36,10 @@ import java.time.Instant /** * Hand-written stand-ins for the governance collaborators, used by the wiring tests. * - * These are real objects, because half of what the tests check is behaviour: whether the observe - * tier refused to quarantine, whether a report was written. A mock that records calls can only - * answer that by restating the runner's logic in the assertions. These keep their state in a list - * the test reads afterwards. + * These are real objects, because half of what the tests check is behaviour: whether a check left + * every proposition alone, whether a report was written, whether a sweep announced what it did. A + * mock that records calls can only answer that by restating the runner's own logic in the + * assertions. These keep their state in a list the test reads afterwards. */ internal object MetamodelTestFixtures { @@ -58,6 +60,16 @@ internal object MetamodelTestFixtures { relationshipTypeNames = emptySet(), ) + /** + * A diff that drops [typeName] from the declaration: the shape of schema change a host acts on + * with a sweep, since data already extracted under that type has nothing describing it any more. + */ + fun diffRemoving(typeName: String): MetamodelDiff = MetamodelDiff( + fromVersion = declaredSchema("Person", typeName).version, + toVersion = declaredSchema("Person").version, + changes = listOf(MetamodelChange.EntityTypeRemoved(typeName)), + ) + /** A proposition mentioning [mentionType], so the quarantine policy has something to catch. */ fun proposition(text: String, mentionType: String): Proposition = Proposition( @@ -127,7 +139,7 @@ internal class RecordingDriftReportStore : DriftReportStore { /** * A [PropositionStore] over an in-memory map. It implements the base port and nothing more, so a - * context holding one has no `PropositionRepository`. That is what shows the drift runner asks only + * context holding one has no `PropositionRepository`. That is what shows the wired sweep asks only * for the narrow port. */ internal class MapPropositionStore(private val initial: List = emptyList()) : PropositionStore { diff --git a/docs/design/metamodel-wiring.md b/docs/design/metamodel-wiring.md index bdb5592c..6a17c0ca 100644 --- a/docs/design/metamodel-wiring.md +++ b/docs/design/metamodel-wiring.md @@ -14,14 +14,17 @@ brings an `EntityManagerFactory` and the repository infrastructure with it, and ```mermaid flowchart TD A{"DeclaredSchemaSource
bean present?"} - A -- no --> Z["Nothing. No stores, no differ,
no runner, no cost."] + A -- no --> Z["Nothing. No stores, no differ,
no runner, no sweep, no cost."] A -- yes --> B{"embabel.dice.metamodel
.enabled"} B -- false --> Z - B -- true --> C["Version store, drift log,
observed-schema source,
differ, quarantine policy,
schema constraints"] - C --> D{"embabel.dice.metamodel
.drift.mode"} - D -- off --> E["No runner.
Stamps and history only."] - D -- "observe (default)" --> F["ObserveOnlyDriftCheckRunner
reports; touches no proposition"] - D -- quarantine --> G["DefaultDriftCheckRunner
run(dryRun = false) marks stranded
propositions STALE"] + B -- true --> C{"embabel.dice.store.type"} + C -- graph --> D["Drivine version store,
drift log, observed-schema
source, schema constraints"] + C -- "in-memory (default)" --> E["In-memory version store.
No drift log, nothing
to observe."] + D --> F["Differ, quarantine policy,
DriftSweepCapable"] + E --> F + F --> G{"embabel.dice.metamodel
.drift.mode"} + G -- off --> H["No runner.
Stamps and history only."] + G -- "observe (default)" --> I["DriftCheckRunner,
where there is a graph
to observe"] ``` There is no sensible default declared schema, and both ways of guessing one are bad. Govern @@ -34,34 +37,94 @@ in the consumer's own code, where a reader can see it. in one environment while the bean stays in place. It is consulted only once the bean exists, so it changes nothing for an application that never declared a schema. -## Drift tiers +## Nothing runs, and nothing quarantines -`embabel.dice.metamodel.drift.mode` sets how far a check may go. It maps the three tiers in -[metamodel-drift.md](metamodel-drift.md) onto one property value each: +Every bean the auto-configuration registers is a capability sitting still. + +There is no scheduler and no startup hook. A check happens when the application calls +`DriftCheckRunner.run()`, from a cron, an admin endpoint or a migration script. The check declares, +stamps, observes, runs both comparisons and writes a `DriftReport`, and no path through it moves a +proposition. + +Acting on what a check found is a second, deliberate step. The application reads +`DriftCheckResult.quarantineDiff`, decides, and calls `DriftSweepCapable.sweep` once per context it +means to reconcile. The wiring supplies the object that performs a sweep and never calls it. + +```kotlin +val result = runner.run() +if (result.hasAnyChange) { + val swept = sweep.sweep(result.quarantineDiff, policy, contextId) + log.info("quarantined {} proposition(s)", swept.quarantined.size) +} +``` + +`embabel.dice.metamodel.drift.mode` picks whether the runner bean exists at all: | Mode | What you get | What it can change | | --- | --- | --- | | `off` | Stores and stamps. Schema history is recorded and readable. | Nothing | -| `observe` (default) | A runner that checks and writes reports | Nothing | -| `quarantine` | The real runner | Marks stranded propositions `STALE`, with a reason | +| `observe` (default) | The same, plus a runner that checks and writes reports | Nothing | + +`observe` is the default because reporting is safe to leave running indefinitely. `off` is there for +an application that wants schema stamps and version history without a check running against its +graph. -The default is `observe`. Reporting is safe to leave running indefinitely; changing proposition -state is a decision somebody makes on purpose. +There is no `quarantine` mode, and there is no property anywhere that turns one on. Quarantine +happens because a host called `sweep`, and a configuration value that could make DICE move +propositions by itself would be exactly the surprise this shape exists to remove. -Which mistakes are recoverable drives that choice. A typo in a declared type name under `observe` -produces a report naming a type you expected to see, which takes five minutes to fix. The same typo -under `quarantine` marks every proposition mentioning that type stale on the next check. Quarantine -is non-destructive, so that is recoverable too, but only if somebody notices. +## Status transitions reach your listeners -The `observe` guarantee lives in the wiring. The bean you get is an `ObserveOnlyDriftCheckRunner` -wrapping the real one; it downgrades `run(dryRun = false)` to a dry run, logs a warning, and returns -a result whose `dryRun` is `true`. A dry-run default on the method only protects the caller who -passes no arguments, and a scheduler, an admin endpoint, or one line in a script all reach straight -past it. Making the tier a property of the bean holds the guarantee whoever calls, and moving -between tiers is then a config change. +A sweep moves a proposition to `STALE`, and things downstream need to hear about it. +`ProjectionLineageStaleCascade` marks every projection record derived from that proposition stale, +and it can only do so if the transition is announced. -Nothing is scheduled here. This wires the capability to run a check; when one runs is the -application's call. +So the sweep is built with every `DiceEventListener` bean on the context, fanned out through a +`CompositeDiceEventListener`, which makes each delivery exception-safe: a listener that throws is +logged and the remaining listeners still hear the event. Registering the cascade is a one-liner in +the application: + +```kotlin +@Bean +fun projectionLineageStaleCascade(records: ProjectionRecordStore) = + ProjectionLineageStaleCascade(records) +``` + +An application with no listener bean gets the no-op listener and everything else behaves the same. + +## Backend selection follows the store + +The Drivine/Neo4j governance beans register under `embabel.dice.store.type=graph`, the same switch +`DiceStorageAutoConfiguration` reads for the proposition store, and they are declared before their +in-memory counterparts so the fallback resolves by registration order. + +Declaring a schema on the default in-memory backend starts cleanly, with no `PersistenceManager` +anywhere. What such an application gets: + +| Piece | In-memory backend | Graph backend | +| --- | --- | --- | +| `MetamodelVersionStore` | `InMemoryMetamodelVersionStore` | `DrivineMetamodelVersionStore` | +| `DriftReportStore` | — | `DrivineDriftReportStore` | +| `ObservedSchemaSource` | — | `DrivineObservedSchemaSource` | +| `DeclaredObservedDiffer` / `MetamodelDiffer` | `StructuralMetamodelDiffer` | same | +| `DriftQuarantinePolicy` | `MentionTypeDriftQuarantinePolicy` | same | +| `DriftSweepCapable` | `PropositionStoreDriftSweep` | same | +| `DriftCheckRunner` | — | `DefaultDriftCheckRunner` | +| `SchemaCatalog` | — | metamodel uniqueness constraints | + +The two blanks in the middle column are the honest answer for a host with no graph. An +observed-schema source reports what a live graph holds, and there is no live graph here; a drift log +is a durable record an operator reads days later, and a heap map is not one. With nothing to +observe, a check has no question to ask, so no runner is registered either. + +That still leaves the first tier of governance fully working: such an application can stamp its +declaration, keep a version history, compare two declarations through the differ, and sweep on a +diff it built itself. It moves to the graph backend when it wants live drift detection. + +The metamodel uniqueness constraints ride along as a `SchemaCatalog` bean, the same way the +proposition and lineage constraints do; Drivine's `SchemaManager` applies them idempotently on +startup. The stores' MERGEs are only race-free under them, so that bean carries no +`@ConditionalOnMissingBean`. Catalogs accumulate, so an application adding its own gets both. ## Replacing a default @@ -78,52 +141,45 @@ registering through the imports file makes "your bean wins" a guarantee. `MetamodelAutoConfigurationTest` checks it from both directions, registering the consumer's configuration before and after the auto-configuration and asserting the same result. -The replaceable pieces: +`DriftSweepCapable` is the one worth calling out. The shipped `PropositionStoreDriftSweep` works on +any `PropositionStore` and does its mention-type filtering, ordering and paging in the JVM. A +durable store that can push all three into a query implements `DriftSweepCapable` itself, and this +default then steps aside. -| Type | Default | Backed by | -| --- | --- | --- | -| `MetamodelVersionStore` | `DrivineMetamodelVersionStore` | Neo4j | -| `DriftReportStore` | `DrivineDriftReportStore` | Neo4j | -| `ObservedSchemaSource` | `DrivineObservedSchemaSource` | Neo4j | -| `DeclaredObservedDiffer` | `StructuralMetamodelDiffer` | pure JVM | -| `DriftQuarantinePolicy` | `MentionTypeDriftQuarantinePolicy` | pure JVM | -| `DriftCheckRunner` | per `drift.mode`, above | — | - -The differ is registered under its concrete type, because `StructuralMetamodelDiffer` answers two -questions — declaration against declaration (`MetamodelDiffer`) and declaration against a live graph -(`DeclaredObservedDiffer`) — and both need to resolve. Backing off keys on `DeclaredObservedDiffer` -alone, since that is the collaborator the runner needs. An application supplying only a -`MetamodelDiffer`, which is a legitimate thing to want on its own, still gets a working drift check. +## The two differ roles are resolved separately -The metamodel uniqueness constraints ride along as a `SchemaCatalog` bean, the same way the -proposition and lineage constraints do; Drivine's `SchemaManager` applies them idempotently on -startup. The stores' MERGEs are only race-free under them, so that bean carries no -`@ConditionalOnMissingBean`. Catalogs accumulate, so an application adding its own gets both. +A check asks two different questions, and each has its own interface: + +- `DeclaredObservedDiffer` — what the live graph holds that this declaration doesn't recognise. +- `MetamodelDiffer` — what moved in the declaration itself since the baseline a sweep last + reconciled. + +The shipped `StructuralMetamodelDiffer` answers both, so it is registered under its concrete type +and resolves as either interface. The runner looks the two up independently, which means one object +can fill both roles, a consumer can replace either role on its own, and a consumer can supply two +distinct beans and have both used. Whichever role no bean fills falls back to a +`StructuralMetamodelDiffer` the runner builds for itself. -## The runner takes the narrow port +The default differ bean backs off as soon as the application supplies either interface. That keeps a +consumer's `MetamodelDiffer` from competing with the shipped differ for the same injection point, +which is how such a bean could end up quietly ignored. -`DefaultDriftCheckRunner` is wired against `PropositionStore`, the base persistence port. +## The sweep takes the narrow port + +`PropositionStoreDriftSweep` is wired against `PropositionStore`, the base persistence port. `PropositionRepository` adds vector search, graph traversal, temporal query, and core search -operations on top, and a drift check uses none of them: it reads propositions by context or in bulk -and saves the flagged copies back. +operations on top, and a sweep uses none of them: it reads one context's propositions and saves the +flagged copies back. Asking for the wider interface at the wiring layer would shut a plain store-and-retrieve backend out of governance over capabilities it is never asked to use, and would do so as a missing `PropositionRepository` bean at startup. A `PropositionRepository` satisfies the narrow parameter, so asking for less costs nothing. -## A missing PersistenceManager fails startup - -The defaults are Drivine/Neo4j-backed and are not gated on a `PersistenceManager` being present. An -application that declared a schema without a graph connection fails at startup with the missing bean -named. Wiring nothing when the connection is missing would leave somebody believing governance is -running when it isn't. An application that wants the loop without Drivine supplies its own three -stores; they are `@ConditionalOnMissingBean` like everything else, and then no `PersistenceManager` -is asked for at all. - ## Property reference | Property | Default | Meaning | | --- | --- | --- | | `embabel.dice.metamodel.enabled` | `true` | Kill switch. Consulted only when a `DeclaredSchemaSource` bean exists | -| `embabel.dice.metamodel.drift.mode` | `observe` | `off`, `observe`, or `quarantine` — see above | +| `embabel.dice.metamodel.drift.mode` | `observe` | `off` or `observe` — whether a `DriftCheckRunner` bean is registered | +| `embabel.dice.store.type` | `in-memory` | Shared with the proposition store. `graph` selects the Drivine-backed governance stores | From 59beb5b1427b3b6acb56258a6649471439634942 Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Wed, 2 Sep 2026 04:41:10 -0400 Subject: [PATCH 04/10] Give the governance loop its operator surface Drift reports, the declared version, and quarantine release were reachable only from a debugger: the report store's bounded reads had no caller and release existed as an SPI with no door. One GovernanceOperationsService now answers reads (latest reports global and per context, the current declared version), runs a check that returns the full impact a sweep would evaluate, and releases a held proposition through the recorded-prior-status path, refusing across contexts. GovernanceController exposes it under /api/v1/metamodel and GovernanceTools exposes it to agents, both following the discovery surface's shape, wired in the autoconfigure module under the governance conditions and overridable per bean. Per-report release is absent: a DriftReport carries no identity a reason could name, and that modeling belongs to its own slice. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- CHANGELOG.md | 42 ++ dice-storage-autoconfigure/pom.xml | 11 + .../GovernanceHttpAutoConfiguration.kt | 82 +++ .../MetamodelAutoConfiguration.kt | 78 ++- ...ot.autoconfigure.AutoConfiguration.imports | 3 +- ...GovernanceOperatorAutoConfigurationTest.kt | 309 +++++++++++ ...tamodelAutoConfigurationIntegrationTest.kt | 8 +- .../MetamodelAutoConfigurationTest.kt | 16 +- .../com/embabel/dice/agent/GovernanceTools.kt | 179 ++++++ .../embabel/dice/governance/GovernanceDtos.kt | 337 ++++++++++++ .../governance/GovernanceOperationsService.kt | 231 ++++++++ .../dice/web/rest/GovernanceController.kt | 172 ++++++ .../embabel/dice/agent/GovernanceToolsTest.kt | 253 +++++++++ .../GovernanceOperationsServiceTest.kt | 513 ++++++++++++++++++ .../dice/web/rest/GovernanceControllerTest.kt | 353 ++++++++++++ docs/design/metamodel-wiring.md | 82 +++ 16 files changed, 2651 insertions(+), 18 deletions(-) create mode 100644 dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceHttpAutoConfiguration.kt create mode 100644 dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceOperatorAutoConfigurationTest.kt create mode 100644 dice/src/main/kotlin/com/embabel/dice/agent/GovernanceTools.kt create mode 100644 dice/src/main/kotlin/com/embabel/dice/governance/GovernanceDtos.kt create mode 100644 dice/src/main/kotlin/com/embabel/dice/governance/GovernanceOperationsService.kt create mode 100644 dice/src/main/kotlin/com/embabel/dice/web/rest/GovernanceController.kt create mode 100644 dice/src/test/kotlin/com/embabel/dice/agent/GovernanceToolsTest.kt create mode 100644 dice/src/test/kotlin/com/embabel/dice/governance/GovernanceOperationsServiceTest.kt create mode 100644 dice/src/test/kotlin/com/embabel/dice/web/rest/GovernanceControllerTest.kt diff --git a/CHANGELOG.md b/CHANGELOG.md index ef9c2bde..4c406690 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -575,6 +575,39 @@ and the consumer PRs that deliver it). module's `SchemaCatalog` bean supplies, and a `PersistenceManager` on the context; a `PropositionStore` brings the sweep with it, and its absence leaves the rest of the loop working. +- An operator surface for schema governance. Until now the loop produced two kinds of inspectable + state — drift reports and quarantined propositions — and offered no way to reach either outside a + debugger. `GovernanceOperationsService` in `dice` is the one way in: `latestReports` and + `reportsInContext` read the drift log, `currentDeclaredVersion` reports the declaration in force + along with whether it has been stamped and which version the last completed sweep reconciled + against, `runCheck` runs a check, and `releaseProposition` lifts one quarantine hold. + `GovernanceController` puts it on HTTP under `/api/v1/metamodel` and `GovernanceTools` exposes the + same five operations as `@LlmTool` agent tools; both call the one service, so the two front ends + cannot answer differently. + Reads are bounded: `limit` defaults to 20, must fall between 1 and 200, and a value outside that + answers `400` naming the bound. `since` takes an ISO-8601 instant. A check reports and moves no + proposition, so its response carries the full impact a sweep would evaluate — both drift sets, the + declaration's own movement in `declaredDiff`, and the two merged into `sweepImpact`. A release is + scoped by the context in its path before it writes, so a proposition in another context answers + `404` untouched; a successful release restores the status the proposition carried before quarantine + and answers the state it is in afterwards. + Wiring: `MetamodelAutoConfiguration` registers the service and the tools under the governance + conditions plus a `DriftReportStore`, `DriftCheckRunner`, `DriftSweepCapable` and + `PropositionStore` on the context, and the new `GovernanceHttpAutoConfiguration` registers the + controller when the application is a servlet web application with Spring MVC on the classpath. All + three are `@ConditionalOnMissingBean`. A host that wants the loop with no HTTP surface excludes one + auto-configuration by name: + `spring.autoconfigure.exclude: com.embabel.dice.storage.autoconfigure.GovernanceHttpAutoConfiguration`. + Building the context stamps nothing, writes no report and moves no proposition. + There is deliberately no "release everything this report quarantined" operation. Nothing in the + model ties a quarantined proposition back to the report whose application held it: a `DriftReport` + has no identity beyond its natural key, and the reason a sweep writes names the two schemas and + nothing about the check. See `docs/design/metamodel-wiring.md`. + **Compatibility: additive.** New types and one new auto-configuration; no existing symbol changes + shape or behavior, and an application with no `DeclaredSchemaSource` bean sees no change. The + controller is not component-scanned, so nothing appears on an application's HTTP surface unless the + governance loop is wired. + - Optional source revisions in the `dice` core provenance model, the first slice of DICE #64. `ProvenanceEntry` gains a sixth field, `sourceRevision`: an opaque, provider-defined string, non-blank when present, recording which version of a source a claim was read from. @@ -655,6 +688,15 @@ and the consumer PRs that deliver it). ### Fixed +- `MetamodelAutoConfiguration` and the metamodel wiring tests referenced the drift quarantine types + at their former home in `com.embabel.dice.metamodel(.support)`. They moved to + `com.embabel.dice.spi` in the `dice` module when quarantine was given its own + `PropositionStatus.QUARANTINED`, and the wiring was left pointing at the old package, so + `dice-storage-autoconfigure` did not compile from clean. The affected tests also still asserted + `PropositionStatus.STALE` after a drift sweep. Imports corrected and the post-sweep assertions + moved to `QUARANTINED`. **Compatibility: additive.** No shipped symbol changes; the module now + builds from a clean tree. + - `:Source.display` in the graph projection is write-once. A `:Source` node is global — one locator key is one node across every context that cites it — and `display` used to be refreshed on every write, so whichever writer ran last owned the label every other context read. It is now set on diff --git a/dice-storage-autoconfigure/pom.xml b/dice-storage-autoconfigure/pom.xml index 2acf3ee5..42d70637 100644 --- a/dice-storage-autoconfigure/pom.xml +++ b/dice-storage-autoconfigure/pom.xml @@ -117,6 +117,17 @@ test + + + jakarta.servlet + jakarta.servlet-api + test + + + + org.springframework + spring-webmvc + test + + + + org.jetbrains + annotations + 26.0.1 + + org.slf4j slf4j-api diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceHttpAutoConfiguration.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceHttpAutoConfiguration.kt deleted file mode 100644 index 3a483faf..00000000 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceHttpAutoConfiguration.kt +++ /dev/null @@ -1,111 +0,0 @@ -/* - * Copyright 2024-2026 Embabel Pty Ltd. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.embabel.dice.storage.autoconfigure - -import com.embabel.dice.governance.GovernanceOperationsService -import com.embabel.dice.metamodel.DeclaredSchemaSource -import com.embabel.dice.web.rest.GovernanceController -import org.slf4j.LoggerFactory -import org.springframework.boot.autoconfigure.AutoConfiguration -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean -import org.springframework.boot.autoconfigure.condition.ConditionalOnClass -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty -import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication -import org.springframework.context.annotation.Bean - -/** - * Puts the governance operator surface on HTTP. - * - * It registers one bean, [GovernanceController], and it registers it only for a host that asked for - * the governance loop. Four things have to hold at once: the application supplies a - * [DeclaredSchemaSource] bean, `embabel.dice.metamodel.enabled` is absent or `true`, the loop wired - * a [GovernanceOperationsService], and this is a servlet web application with Spring MVC on the - * classpath. Miss any one of them and the routes under `/api/v1/metamodel` do not exist. Under - * `drift.mode=off` there is no check to run, so there is no service and no controller either. - * - * ## The opt-in is stated here as well as on the loop - * - * A declared schema and the kill switch are the gate on [MetamodelAutoConfiguration], which is where - * the service comes from, so a context that reaches this class has satisfied them once already. They - * are repeated because this is the part of governance a host meets from outside — six public routes, - * one of them a write — and leaning on the service alone leaves those routes one hand-wired - * `GovernanceOperationsService` bean away from appearing in an application that declared no schema. - * The condition that opens a public surface belongs on the class that opens it. - * - * ## How the ordering is guaranteed - * - * `@ConditionalOnBean` sees only the bean definitions registered by the time it runs, so an - * auto-configuration asking about a bean that another auto-configuration contributes has to run - * afterwards. `@AutoConfiguration(after = [MetamodelAutoConfiguration::class])` is what arranges - * that: Spring Boot sorts auto-configuration classes on those declarations before it evaluates a - * single condition, so [MetamodelAutoConfiguration] has already had its say about - * [GovernanceOperationsService] when this class is asked. The [DeclaredSchemaSource] half needs no - * ordering, because it is the host's own bean and Spring Boot registers every application bean - * definition before it processes any auto-configuration at all. - * - * ## Turning the HTTP surface off - * - * The operator surface is worth having through an agent's tools or a host's own code without opening - * a public endpoint. This lives in its own auto-configuration so that choice is one line, and the - * service and the agent tools stay wired: - * - * ```yaml - * spring: - * autoconfigure: - * exclude: com.embabel.dice.storage.autoconfigure.GovernanceHttpAutoConfiguration - * ``` - * - * A host that wants the endpoints on a different path, behind extra authorization, or shaped - * differently supplies its own [GovernanceController] bean; `@ConditionalOnMissingBean` backs this - * one off. - * - * ## What is on the endpoints - * - * The routes read and run; one of them writes. `POST .../quarantine/{propositionId}/release` lifts a - * quarantine hold, so it changes stored data and should sit behind whatever authorization the host - * puts on its administrative routes. The reads and the drift check move no proposition, though a - * check does write a drift report and stamp the declared version. - */ -@AutoConfiguration(after = [MetamodelAutoConfiguration::class]) -@ConditionalOnClass(name = [REST_CONTROLLER]) -@ConditionalOnWebApplication(type = ConditionalOnWebApplication.Type.SERVLET) -@ConditionalOnBean(value = [DeclaredSchemaSource::class, GovernanceOperationsService::class]) -@ConditionalOnProperty( - prefix = "embabel.dice.metamodel", - name = ["enabled"], - havingValue = "true", - matchIfMissing = true, -) -class GovernanceHttpAutoConfiguration { - - private val logger = LoggerFactory.getLogger(GovernanceHttpAutoConfiguration::class.java) - - @Bean - @ConditionalOnMissingBean(GovernanceController::class) - fun governanceController(operations: GovernanceOperationsService): GovernanceController { - logger.info("Metamodel governance REST surface wired under /api/v1/metamodel") - return GovernanceController(operations) - } -} - -/** - * Spring MVC's marker, named as text. Spring MVC is a `provided` dependency, so a consumer can run - * DICE without it; naming the class by text keeps this configuration loadable on such a classpath, - * because `@ConditionalOnClass` is read from the bytecode and the annotated class is never loaded - * when the answer is no. - */ -private const val REST_CONTROLLER = "org.springframework.web.bind.annotation.RestController" diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt index fad82034..77137606 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt @@ -15,7 +15,6 @@ */ package com.embabel.dice.storage.autoconfigure -import com.embabel.dice.agent.GovernanceTools import com.embabel.dice.common.CompositeDiceEventListener import com.embabel.dice.common.DiceEventListener import com.embabel.dice.governance.GovernanceOperationsService @@ -43,6 +42,7 @@ import com.embabel.dice.storage.DrivineObservedSchemaSource import com.embabel.dice.storage.MetamodelSchema import org.drivine.manager.PersistenceManager import org.drivine.schema.SchemaCatalog +import org.jetbrains.annotations.ApiStatus import org.slf4j.LoggerFactory import org.springframework.beans.factory.ObjectProvider import org.springframework.boot.autoconfigure.AutoConfiguration @@ -120,6 +120,7 @@ import org.springframework.context.annotation.Bean * lineage constraints do; Drivine's `SchemaManager` applies them idempotently on startup. The * stores' MERGEs are only race-free under them, so they are required wherever the graph stores are. */ +@ApiStatus.Experimental @AutoConfiguration(after = [DiceStorageAutoConfiguration::class]) @ConditionalOnBean(DeclaredSchemaSource::class) @ConditionalOnProperty( @@ -355,8 +356,13 @@ class MetamodelAutoConfiguration { * runner there is no check to run, and with no sweep there is no hold to lift. Under the default * in-memory backend, and under `drift.mode=off`, none of those exist and neither does this bean. * - * Registering it runs nothing. It is the object a host calls, or hands to - * `GovernanceController` and [GovernanceTools]. + * Registering it runs nothing. It is the object a host calls, or hands to `GovernanceController` + * and `GovernanceTools`. + * + * It is also the only governance bean either front end needs. `GovernanceController` arrives + * with `DiceRestConfiguration`, the one import that opens any DICE REST surface, and switches + * itself on when this bean exists. `GovernanceTools` is constructed by the host, the way every + * DICE tool object is; nothing here registers one. */ @Bean @ConditionalOnBean( @@ -391,22 +397,6 @@ class MetamodelAutoConfiguration { ) } - /** - * The governance operations as agent tools, for a host that registers them with its own MCP - * server or tool set. - * - * No web application is needed, so this sits beside the service and outside - * `GovernanceHttpAutoConfiguration`. A host that wants no agent surface supplies its own bean of - * this type, or takes the tools and registers none of them. - */ - @Bean - @ConditionalOnBean(GovernanceOperationsService::class) - @ConditionalOnMissingBean(GovernanceTools::class) - fun governanceTools(operations: GovernanceOperationsService): GovernanceTools { - logger.debug("Wiring the metamodel governance agent tools") - return GovernanceTools(operations) - } - /** * Every application listener as one listener. [CompositeDiceEventListener] wraps each delivery so * a throwing listener can never abort the sweep that emitted the event, and an application with diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt index 81148c7a..f6f8df9b 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt @@ -15,6 +15,7 @@ */ package com.embabel.dice.storage.autoconfigure +import org.jetbrains.annotations.ApiStatus import org.springframework.boot.context.properties.ConfigurationProperties /** @@ -25,6 +26,7 @@ import org.springframework.boot.context.properties.ConfigurationProperties * check reads, compares and writes a report, and moving a proposition is a separate call a host * makes on `DriftSweepCapable` at a moment it picks. */ +@ApiStatus.Experimental enum class DriftMode { /** @@ -49,6 +51,7 @@ enum class DriftMode { * supplies a `DeclaredSchemaSource` bean, whatever these properties say. They control what happens * once one exists. */ +@ApiStatus.Experimental @ConfigurationProperties(prefix = "embabel.dice.metamodel") data class MetamodelProperties( diff --git a/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports index f8b8b009..46249c08 100644 --- a/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ b/dice-storage-autoconfigure/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports @@ -1,5 +1,4 @@ com.embabel.dice.storage.autoconfigure.DiceStorageAutoConfiguration com.embabel.dice.storage.autoconfigure.DiceDecaySchedulingConfiguration com.embabel.dice.storage.autoconfigure.CollectorAutoConfiguration -com.embabel.dice.storage.autoconfigure.MetamodelAutoConfiguration -com.embabel.dice.storage.autoconfigure.GovernanceHttpAutoConfiguration \ No newline at end of file +com.embabel.dice.storage.autoconfigure.MetamodelAutoConfiguration \ No newline at end of file diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceOperatorAutoConfigurationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceOperatorAutoConfigurationTest.kt index 34bfd022..deda6773 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceOperatorAutoConfigurationTest.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/GovernanceOperatorAutoConfigurationTest.kt @@ -20,9 +20,13 @@ import com.embabel.dice.governance.GovernanceOperationsService import com.embabel.dice.metamodel.DeclaredSchemaSource import com.embabel.dice.metamodel.DriftCheckRunner import com.embabel.dice.proposition.PropositionStatus +import com.embabel.dice.proposition.store.InMemoryPropositionRepository +import com.embabel.dice.web.rest.DiceRestConfiguration import com.embabel.dice.web.rest.GovernanceController import org.assertj.core.api.Assertions.assertThat +import org.drivine.manager.PersistenceManager import org.junit.jupiter.api.Test +import org.mockito.kotlin.mock import org.springframework.beans.factory.getBean import org.springframework.boot.autoconfigure.AutoConfigurations import org.springframework.boot.test.context.runner.ApplicationContextRunner @@ -30,62 +34,72 @@ import org.springframework.boot.test.context.runner.WebApplicationContextRunner import org.springframework.context.ApplicationContext import org.springframework.context.annotation.Bean import org.springframework.context.annotation.Configuration +import org.springframework.context.annotation.Import import org.springframework.core.io.support.PathMatchingResourcePatternResolver import org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping /** - * Wiring tests for the governance operator surface: the service and the agent tools from - * [MetamodelAutoConfiguration], and the REST controller from [GovernanceHttpAutoConfiguration]. + * Wiring tests for the governance operator surface: the service from [MetamodelAutoConfiguration], + * and the REST controller that arrives with a host's `@Import(DiceRestConfiguration)`. * - * What the wiring alone can get wrong is whether the surface appears when there is something behind - * it, disappears with the rest of the loop, backs off for a consumer's own bean, and stays inert — - * registering these beans must never run a check or write to a store. + * Two host decisions have to line up before an operator can reach governance over HTTP — import the + * DICE REST surface, and wire the governance loop — and these tests pin all four combinations. The + * agent tools are here too, for the opposite reason: no context may hold a `GovernanceTools` bean, + * because a host builds those itself. */ class GovernanceOperatorAutoConfigurationTest { - private val autoConfigurations = AutoConfigurations.of( - MetamodelAutoConfiguration::class.java, - GovernanceHttpAutoConfiguration::class.java, - ) + private val autoConfigurations = AutoConfigurations.of(MetamodelAutoConfiguration::class.java) - /** A host with the whole loop in memory: stores, an observed schema, propositions. */ + /** A host with the whole loop in memory and no REST import at all. */ private val runner = ApplicationContextRunner() .withConfiguration(autoConfigurations) .withUserConfiguration(InMemoryGovernanceConfig::class.java) + /** The same loop in a servlet web application that imported the DICE REST surface. */ private val webRunner = WebApplicationContextRunner() .withConfiguration(autoConfigurations) - .withUserConfiguration(InMemoryGovernanceConfig::class.java, EndpointMappingConfig::class.java) + .withUserConfiguration(RestImportingGovernanceConfig::class.java, EndpointMappingConfig::class.java) - // ---- Present when the loop is ---- + // ---- Import plus a wired loop ---- + /** + * The happy path, pinned as URLs. A host that imported [DiceRestConfiguration] and declared a + * schema gets exactly these six routes, which is the surface an operator is offered and the + * surface a consumer's endpoint snapshot records. + */ @Test - fun `the governance loop brings the operator service and the agent tools with it`() { - runner.run { ctx -> + fun `importing DICE REST with the loop wired opens all six routes`() { + webRunner.run { ctx -> assertThat(ctx).hasNotFailed() assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) - assertThat(ctx).hasSingleBean(GovernanceTools::class.java) + assertThat(ctx).hasSingleBean(GovernanceController::class.java) + assertThat(metamodelEndpoints(ctx)).containsExactlyInAnyOrder(*ALL_SIX_ROUTES) } } /** - * The happy path, pinned as URLs. A host that declared a schema and runs a servlet web - * application gets exactly these six routes, which is the surface an operator is offered and the - * surface a consumer's endpoint snapshot records. + * The condition is asked late enough to see a bean the auto-configuration contributed. + * + * `@ConditionalOnBean` on an imported class is answered while the importing configuration class + * is read, and Spring Boot registers auto-configuration bean definitions after that. So a plain + * entry in `DiceRestConfiguration`'s `@Import` list would answer "no service" for every host + * whose loop came from `MetamodelAutoConfiguration` — which is every host that follows the + * documented wiring. `GovernanceControllerImport` defers the import to the end of the round, + * and this test is what would catch that deferral being dropped: the service below exists only + * because the auto-configuration built it. */ @Test - fun `the REST controller appears in a servlet web application, with all six routes`() { + fun `the controller sees a service the auto-configuration contributed`() { webRunner.run { ctx -> - assertThat(ctx).hasNotFailed() - assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) + assertThat(ctx.getBeanDefinitionNames()).contains("governanceOperationsService") assertThat(ctx).hasSingleBean(GovernanceController::class.java) - assertThat(metamodelEndpoints(ctx)).containsExactlyInAnyOrder(*ALL_SIX_ROUTES) } } /** - * The service and the tools are useful to a host with no HTTP at all, so only the controller is - * gated on a servlet web application. + * The service is useful to a host with no HTTP at all, and the tools run anywhere, so only the + * controller depends on a servlet web application and on the REST import. */ @Test fun `outside a web application there is a service and no controller`() { @@ -95,64 +109,61 @@ class GovernanceOperatorAutoConfigurationTest { } } - // ---- Absent when the loop is ---- + // ---- Import without a wired loop ---- /** - * The consumer smoke test's own scenario: a servlet web application that declared no schema and - * opted into nothing, holding the whole DICE classpath. It must resolve zero `/api/v1/metamodel` - * URLs, and hold none of the three operator beans. + * A host that imported the DICE REST surface and declared no schema. It must resolve zero + * `/api/v1/metamodel` URLs and start cleanly — the other DICE controllers it asked for are + * unaffected. * * The endpoint assertion is the load-bearing one. A `doesNotHaveBean(GovernanceController)` * check answers a narrower question — whether one bean type is on the context — and stays green * while a route reaches a handler by some path this test never named. */ @Test - fun `no DeclaredSchemaSource means no operator surface at all`() { + fun `importing DICE REST with no declared schema opens no route and starts cleanly`() { WebApplicationContextRunner() .withConfiguration(autoConfigurations) .withPropertyValues(GRAPH_BACKEND) - .withUserConfiguration(NoDeclaredSchemaConfig::class.java, EndpointMappingConfig::class.java) + .withUserConfiguration(RestImportingNoSchemaConfig::class.java, EndpointMappingConfig::class.java) .run { ctx -> assertThat(ctx).hasNotFailed() assertThat(metamodelEndpoints(ctx)).isEmpty() assertThat(ctx).doesNotHaveBean(GovernanceOperationsService::class.java) - assertThat(ctx).doesNotHaveBean(GovernanceTools::class.java) assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) } } /** - * The declared schema is required on its own account. A host that hand-wired a - * `GovernanceOperationsService` for its own code has said nothing about what it governs, so the - * HTTP surface stays closed until it declares a schema. + * A declared schema on its own is short of a service: the in-memory backend has no drift log and + * nothing to observe, so there is no runner and no operator service. The import is in place and + * the routes still have to stay shut, with a context that starts. * - * This is what the opt-in condition on [GovernanceHttpAutoConfiguration] buys. Take that - * condition away and the controller here comes back, because a service bean exists and the - * remaining conditions are all satisfied. + * This is the degradation that matters. The controller takes a `GovernanceOperationsService` in + * its constructor, so a condition that let it register here would fail the context outright. */ @Test - fun `a hand-wired operator service without a declared schema opens no HTTP surface`() { + fun `a declared schema with no service behind it opens no route and starts cleanly`() { WebApplicationContextRunner() .withConfiguration(autoConfigurations) - .withUserConfiguration(ServiceWithoutDeclaredSchemaConfig::class.java, EndpointMappingConfig::class.java) + .withUserConfiguration(RestImportingNoGraphConfig::class.java, EndpointMappingConfig::class.java) .run { ctx -> assertThat(ctx).hasNotFailed() - assertThat(ctx).doesNotHaveBean(DeclaredSchemaSource::class.java) - assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) + assertThat(ctx).hasSingleBean(DeclaredSchemaSource::class.java) + assertThat(ctx).doesNotHaveBean(GovernanceOperationsService::class.java) assertThat(metamodelEndpoints(ctx)).isEmpty() assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) } } @Test - fun `enabled=false removes the operator surface with the rest of the loop`() { + fun `enabled=false shuts the routes with the rest of the loop`() { webRunner .withPropertyValues("embabel.dice.metamodel.enabled=false") .run { ctx -> assertThat(ctx).hasNotFailed() assertThat(metamodelEndpoints(ctx)).isEmpty() assertThat(ctx).doesNotHaveBean(GovernanceOperationsService::class.java) - assertThat(ctx).doesNotHaveBean(GovernanceTools::class.java) assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) // The application's own DeclaredSchemaSource stays; only the metamodel beans go. assertThat(ctx).hasSingleBean(DeclaredSchemaSource::class.java) @@ -165,55 +176,150 @@ class GovernanceOperatorAutoConfigurationTest { * offering it. */ @Test - fun `drift mode off leaves no runner and therefore no operator surface`() { + fun `drift mode off leaves no runner and therefore no routes`() { webRunner .withPropertyValues("embabel.dice.metamodel.drift.mode=off") .run { ctx -> assertThat(ctx).hasNotFailed() assertThat(ctx).doesNotHaveBean(DriftCheckRunner::class.java) assertThat(ctx).doesNotHaveBean(GovernanceOperationsService::class.java) + assertThat(metamodelEndpoints(ctx)).isEmpty() assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) } } + // ---- A wired loop without the import ---- + /** - * The default in-memory backend has no drift log and no observed-schema source, so no runner and - * no operator surface. The rest of the loop still starts. + * The whole governance loop, in a servlet web application, with no `@Import` of the DICE REST + * surface. The service is there for the host's own code and zero URLs are published. + * + * This is the test the operator comment is about: HTTP used to switch itself on as soon as a + * schema existed, and now it takes the same explicit import every other DICE controller takes. */ @Test - fun `a host with no graph gets the loop without the operator surface`() { + fun `a wired loop publishes no route until the host imports DICE REST`() { WebApplicationContextRunner() .withConfiguration(autoConfigurations) - .withUserConfiguration(InMemoryBackendConfig::class.java) + .withUserConfiguration(InMemoryGovernanceConfig::class.java, EndpointMappingConfig::class.java) .run { ctx -> assertThat(ctx).hasNotFailed() - assertThat(ctx).doesNotHaveBean(GovernanceOperationsService::class.java) + assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) + assertThat(metamodelEndpoints(ctx)).isEmpty() assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) } } - // ---- A consumer's own bean wins ---- + /** + * The same with the graph backend selected and no schema declared: still no import, still no + * routes. The import is the only thing that publishes them. + */ + @Test + fun `no import means no route whatever the host declared`() { + WebApplicationContextRunner() + .withConfiguration(autoConfigurations) + .withPropertyValues(GRAPH_BACKEND) + .withUserConfiguration(NoDeclaredSchemaConfig::class.java, EndpointMappingConfig::class.java) + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(metamodelEndpoints(ctx)).isEmpty() + assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) + } + } + /** + * No entry in the auto-configuration imports file may register the controller. That file is the + * list Spring Boot applies to every application on the classpath, so a name in it is the one way + * governance HTTP could come back without a host asking for it. + */ @Test - fun `a consumer operator service wins whether it is registered before or after`() { - bothOrders(CustomOperationsConfig::class.java) { ctx -> - assertThat(ctx).hasNotFailed() + fun `no auto-configuration is registered for the governance HTTP surface`() { + val declared = PathMatchingResourcePatternResolver() + .getResources("classpath*:META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports") + .flatMap { it.inputStream.bufferedReader().readLines() } + .map { it.trim() } + .filter { it.startsWith("com.embabel.dice") } + + assertThat(declared).contains(MetamodelAutoConfiguration::class.java.name) + assertThat(declared).noneMatch { "Governance" in it } + } + + // ---- The agent tools are the host's to build ---- + + /** + * No autoconfigured context holds a `GovernanceTools` bean, in any of the shapes above. DICE + * registers no tool object as a bean — `DiscoveryTools`, `GraphQueryTools` and `Memory` are all + * constructed by the application that wants them, and governance follows that. + */ + @Test + fun `no autoconfigured context holds a GovernanceTools bean`() { + runner.run { ctx -> assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) - assertThat(ctx.getBean()) - .isSameAs(ctx.getBean(CustomOperationsConfig::class.java).singleton) + assertThat(ctx).doesNotHaveBean(GovernanceTools::class.java) } + webRunner.run { ctx -> + assertThat(ctx).hasSingleBean(GovernanceController::class.java) + assertThat(ctx).doesNotHaveBean(GovernanceTools::class.java) + } + WebApplicationContextRunner() + .withConfiguration(autoConfigurations) + .withPropertyValues(GRAPH_BACKEND) + .withUserConfiguration(NoDeclaredSchemaConfig::class.java) + .run { ctx -> + assertThat(ctx).doesNotHaveBean(GovernanceTools::class.java) + } } + /** + * And the host can build them in one line from the service the wiring did supply, which is what + * makes the missing bean a design choice a consumer can live with. `GovernanceToolsTest` pins + * which five tools come back. + */ @Test - fun `a consumer controller wins whether it is registered before or after`() { - bothOrders(CustomControllerConfig::class.java) { ctx -> - assertThat(ctx).hasNotFailed() - assertThat(ctx).hasSingleBean(GovernanceController::class.java) - assertThat(ctx.getBean()) - .isSameAs(ctx.getBean(CustomControllerConfig::class.java).singleton) + fun `a host builds the agent tools from the wired service`() { + runner.run { ctx -> + val tools = GovernanceTools.asTools(ctx.getBean()) + assertThat(tools).hasSize(5) } } + // ---- A consumer's own bean wins ---- + + @Test + fun `a consumer operator service wins whether it is registered before or after`() { + WebApplicationContextRunner() + .withUserConfiguration(InMemoryGovernanceConfig::class.java, CustomOperationsConfig::class.java) + .withConfiguration(autoConfigurations) + .run(::assertConsumerServiceWon) + + WebApplicationContextRunner() + .withConfiguration(autoConfigurations) + .withUserConfiguration(InMemoryGovernanceConfig::class.java, CustomOperationsConfig::class.java) + .run(::assertConsumerServiceWon) + } + + /** + * A host that wants these operations on a different path, or behind extra authorization, brings + * its own controller bean and still imports [DiceRestConfiguration] for the other DICE + * controllers. The shipped one backs off, so the two never compete for the same URLs. + */ + @Test + fun `a consumer controller wins and the shipped one backs off`() { + WebApplicationContextRunner() + .withConfiguration(autoConfigurations) + .withUserConfiguration( + RestImportingGovernanceConfig::class.java, + CustomControllerConfig::class.java, + EndpointMappingConfig::class.java, + ) + .run { ctx -> + assertThat(ctx).hasNotFailed() + assertThat(ctx).hasSingleBean(GovernanceController::class.java) + assertThat(ctx.getBean()) + .isSameAs(ctx.getBean(CustomControllerConfig::class.java).singleton) + } + } + // ---- Inertness ---- /** @@ -229,7 +335,7 @@ class GovernanceOperatorAutoConfigurationTest { assertThat(ctx.getBean().saved).isEmpty() assertThat(ctx.getBean().saved).isEmpty() - assertThat(ctx.getBean().findAll()) + assertThat(ctx.getBean().findAll()) .isNotEmpty .allMatch { it.status == PropositionStatus.ACTIVE } @@ -252,63 +358,18 @@ class GovernanceOperatorAutoConfigurationTest { assertThat(check.driftedEntityTypes).containsExactly("Ghost") assertThat(reports.saved).hasSize(1) // A check reports and moves nothing. - assertThat(ctx.getBean().findAll()) + assertThat(ctx.getBean().findAll()) .allMatch { it.status == PropositionStatus.ACTIVE } } } - // ---- Registration ---- - - /** - * The HTTP surface is registered as an auto-configuration of its own. That entry is what makes - * `spring.autoconfigure.exclude: com.embabel.dice.storage.autoconfigure.GovernanceHttpAutoConfiguration` - * a knob at all: Spring Boot's exclusion filter works on the names in this file, so a controller - * declared inside [MetamodelAutoConfiguration] could not have been switched off on its own. - */ - @Test - fun `the HTTP surface is its own auto-configuration, so a host can exclude it by name`() { - val declared = PathMatchingResourcePatternResolver() - .getResources("classpath*:META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports") - .flatMap { it.inputStream.bufferedReader().readLines() } - .map { it.trim() } - - assertThat(declared).contains(GovernanceHttpAutoConfiguration::class.java.name) - assertThat(declared).contains(MetamodelAutoConfiguration::class.java.name) - } - - /** - * What that exclusion leaves behind: the governance loop, the operator service and the agent - * tools, all in a servlet web application, with no controller. This is the same context an - * excluded [GovernanceHttpAutoConfiguration] produces, built by registering the other - * auto-configuration alone. - */ - @Test - fun `without the HTTP auto-configuration the service and the tools still stand`() { - WebApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(MetamodelAutoConfiguration::class.java)) - .withUserConfiguration(InMemoryGovernanceConfig::class.java) - .run { ctx -> - assertThat(ctx).hasNotFailed() - assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) - assertThat(ctx).hasSingleBean(GovernanceTools::class.java) - assertThat(ctx).doesNotHaveBean(GovernanceController::class.java) - } - } - - /** Runs [assertions] with [userConfiguration] registered before and after the auto-configurations. */ - private fun bothOrders( - userConfiguration: Class<*>, - assertions: (org.springframework.boot.test.context.assertj.AssertableWebApplicationContext) -> Unit, + private fun assertConsumerServiceWon( + ctx: org.springframework.boot.test.context.assertj.AssertableWebApplicationContext, ) { - WebApplicationContextRunner() - .withUserConfiguration(InMemoryGovernanceConfig::class.java, userConfiguration) - .withConfiguration(autoConfigurations) - .run(assertions) - - WebApplicationContextRunner() - .withConfiguration(autoConfigurations) - .withUserConfiguration(InMemoryGovernanceConfig::class.java, userConfiguration) - .run(assertions) + assertThat(ctx).hasNotFailed() + assertThat(ctx).hasSingleBean(GovernanceOperationsService::class.java) + assertThat(ctx.getBean()) + .isSameAs(ctx.getBean(CustomOperationsConfig::class.java).singleton) } /** @@ -359,14 +420,60 @@ internal open class EndpointMappingConfig { } /** - * An application that built a [GovernanceOperationsService] by hand for its own code and declared no - * schema. Everything the HTTP surface needs is here apart from the opt-in itself. + * A host that imported the DICE REST surface and wired the whole governance loop in memory. The + * observed schema holds a `Ghost` type the declaration never mentions, so a check has real drift to + * report. + * + * Its proposition store is an `InMemoryPropositionRepository`, which is what a host importing + * [DiceRestConfiguration] already has: `MemoryController` comes with that import and needs a + * `PropositionRepository` on the context. + */ +@Configuration(proxyBeanMethods = false) +@Import(DiceRestConfiguration::class) +internal open class RestImportingGovernanceConfig { + + @Bean + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun versionStore(): RecordingMetamodelVersionStore = RecordingMetamodelVersionStore() + + @Bean + open fun driftReportStore(): RecordingDriftReportStore = RecordingDriftReportStore() + + @Bean + open fun observedSchemaSource(): FixedObservedSchemaSource = FixedObservedSchemaSource() + + @Bean + open fun propositionStore(): InMemoryPropositionRepository = seededRepository() +} + +/** The same host with its declaration taken away, so governance never wires. */ +@Configuration(proxyBeanMethods = false) +@Import(DiceRestConfiguration::class) +internal open class RestImportingNoSchemaConfig { + + @Bean + open fun persistenceManager(): PersistenceManager = mock() + + @Bean + open fun propositionStore(): InMemoryPropositionRepository = seededRepository() +} + +/** + * A host that imported the DICE REST surface and declared a schema on the default in-memory backend. + * Governance wires as far as it can, and stops short of a drift log, an observed schema, a runner + * and therefore an operator service. */ @Configuration(proxyBeanMethods = false) -internal open class ServiceWithoutDeclaredSchemaConfig { +@Import(DiceRestConfiguration::class) +internal open class RestImportingNoGraphConfig { @Bean - open fun handWiredGovernanceOperations(): GovernanceOperationsService = CustomOperationsConfig().singleton + open fun declaredSchemaSource(): DeclaredSchemaSource = FixedDeclaredSchemaSource() + + @Bean + open fun propositionStore(): InMemoryPropositionRepository = seededRepository() } /** An application that brought its own operator service. */ @@ -401,3 +508,10 @@ internal object ThrowingDriftCheckRunner : DriftCheckRunner { override fun run(contextId: com.embabel.agent.core.ContextId?) = throw UnsupportedOperationException("never called; these tests only ask which bean won") } + +/** Two propositions, one of them mentioning the `Ghost` type the declaration never governs. */ +internal fun seededRepository(): InMemoryPropositionRepository = + InMemoryPropositionRepository().apply { + save(MetamodelTestFixtures.proposition("Ada haunts the archive", mentionType = "Ghost")) + save(MetamodelTestFixtures.proposition("Ada wrote the notes", mentionType = "Person")) + } diff --git a/dice/src/main/kotlin/com/embabel/dice/agent/GovernanceTools.kt b/dice/src/main/kotlin/com/embabel/dice/agent/GovernanceTools.kt index dcb1bc6d..955a257c 100644 --- a/dice/src/main/kotlin/com/embabel/dice/agent/GovernanceTools.kt +++ b/dice/src/main/kotlin/com/embabel/dice/agent/GovernanceTools.kt @@ -21,6 +21,7 @@ import com.embabel.dice.governance.GovernanceOperationsService import com.embabel.dice.governance.GovernanceRequestException import com.embabel.dice.governance.parseSince import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import org.jetbrains.annotations.ApiStatus import org.slf4j.LoggerFactory /** @@ -31,6 +32,20 @@ import org.slf4j.LoggerFactory * classpath, no MCP SDK and no servlet dependency. A consuming application calls [asTools] and * registers the returned `List` with its own MCP server or agent tool set. * + * ## The host builds these + * + * No DICE auto-configuration registers a `GovernanceTools` bean, which is how every other DICE tool + * object works — `DiscoveryTools`, `GraphQueryTools` and `Memory` are all constructed by the + * application that wants them. A tool object is only useful once it is registered with a particular + * agent or MCP server, and that registration is the host's call, so the host also decides when the + * object exists: + * + * ```kotlin + * @Bean + * fun governanceTools(operations: GovernanceOperationsService): List = + * GovernanceTools.asTools(operations) + * ``` + * * Every tool runs through one [GovernanceOperationsService], the same object `GovernanceController` * calls over HTTP, so an agent and an operator reading the same state get one answer. Results are * leak-free JSON via [Tool.Result.text]; a refused request — a limit outside its bounds, a blank @@ -50,6 +65,7 @@ import org.slf4j.LoggerFactory * * @param operations The single governance service every tool delegates to. */ +@ApiStatus.Experimental class GovernanceTools( private val operations: GovernanceOperationsService, ) { @@ -165,6 +181,9 @@ class GovernanceTools( /** * Create [Tool] instances exposing the governance operator surface. * + * The returned tools can be registered with an agent's tool set or an MCP server, e.g. + * alongside `DiscoveryTools` and `Memory`. + * * ```kotlin * val tools = GovernanceTools.asTools(governanceOperationsService) * ``` diff --git a/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceDtos.kt b/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceDtos.kt index a2065d22..022187c7 100644 --- a/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceDtos.kt +++ b/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceDtos.kt @@ -23,6 +23,7 @@ import com.embabel.dice.metamodel.MetamodelVersion import com.embabel.dice.metamodel.PropertySignature import com.embabel.dice.proposition.Proposition import com.embabel.dice.proposition.PropositionStatus +import org.jetbrains.annotations.ApiStatus import java.time.Instant /** @@ -45,6 +46,7 @@ import java.time.Instant * `Company LIST`. * @property after How it looks now, the same way. */ +@ApiStatus.Experimental data class PropertyChangeDto( val typeName: String, val propertyName: String, @@ -59,6 +61,7 @@ data class PropertyChangeDto( * @property before The name it went by. * @property after The name it goes by now. */ +@ApiStatus.Experimental data class TypeRenameDto( val before: String, val after: String, @@ -82,6 +85,7 @@ data class TypeRenameDto( * @property addedRelationships Rendered `From-[name]->To` descriptors the newer version adds. * @property removedRelationships Descriptors the newer version drops. */ +@ApiStatus.Experimental data class MetamodelDiffDto( val fromVersionHash: String, val toVersionHash: String, @@ -150,6 +154,7 @@ data class MetamodelDiffDto( * @property declaredDiff How the declaration itself moved since the last completed sweep, or `null` * when no baseline had been reconciled yet. */ +@ApiStatus.Experimental data class DriftReportDto( val schemaName: String, val versionHash: String, @@ -193,6 +198,7 @@ data class DriftReportDto( * @property sweptVersionHash The hash the last completed sweep reconciled against. `null` when no * sweep has completed, and also when the version store tracks no baseline at all. */ +@ApiStatus.Experimental data class DeclaredVersionDto( val schemaName: String, val contentHash: String, @@ -241,6 +247,7 @@ data class DeclaredVersionDto( * there was no baseline to compare against. * @property sweepImpact The merged comparison a sweep evaluates propositions against. */ +@ApiStatus.Experimental data class DriftCheckDto( val schemaName: String, val versionHash: String, @@ -287,6 +294,7 @@ data class DriftCheckDto( * @property quarantineReason The explanation still on it, `null` once the release cleared it. * @property metadataRevised When its metadata last moved, ISO-8601. */ +@ApiStatus.Experimental data class ReleasedPropositionDto( val propositionId: String, val contextId: String, @@ -318,6 +326,7 @@ data class ReleasedPropositionDto( * on the generic path. The message names the bound that was broken, since an operator reading a bare * `400` has nothing to correct. */ +@ApiStatus.Experimental class GovernanceRequestException(message: String) : RuntimeException(message) /** diff --git a/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceOperationsService.kt b/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceOperationsService.kt index 9c11b3e6..fcd1c532 100644 --- a/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceOperationsService.kt +++ b/dice/src/main/kotlin/com/embabel/dice/governance/GovernanceOperationsService.kt @@ -23,6 +23,7 @@ import com.embabel.dice.metamodel.MetamodelVersionStore import com.embabel.dice.metamodel.SweptBaselineStore import com.embabel.dice.proposition.PropositionStore import com.embabel.dice.spi.DriftSweepCapable +import org.jetbrains.annotations.ApiStatus import org.slf4j.LoggerFactory import java.time.Instant @@ -74,6 +75,7 @@ import java.time.Instant * is used here; nothing in this class sweeps. * @param propositions Read to confirm a proposition's context before a release touches it. */ +@ApiStatus.Experimental class GovernanceOperationsService( private val declaredSchemaSource: DeclaredSchemaSource, private val versionStore: MetamodelVersionStore, diff --git a/dice/src/main/kotlin/com/embabel/dice/web/rest/DiceRestConfiguration.kt b/dice/src/main/kotlin/com/embabel/dice/web/rest/DiceRestConfiguration.kt index df9065b5..84373fc9 100644 --- a/dice/src/main/kotlin/com/embabel/dice/web/rest/DiceRestConfiguration.kt +++ b/dice/src/main/kotlin/com/embabel/dice/web/rest/DiceRestConfiguration.kt @@ -16,15 +16,18 @@ package com.embabel.dice.web.rest import org.springframework.context.annotation.Configuration +import org.springframework.context.annotation.DeferredImportSelector import org.springframework.context.annotation.Import +import org.springframework.core.Ordered +import org.springframework.core.type.AnnotationMetadata /** * Opt-in Spring configuration that activates all DICE REST controllers. * - * Import this in your application config to expose the proposition extraction, memory, and - * discovery endpoints. Nothing is component-scanned — the controllers only activate when this - * class is imported AND the required beans (PropositionPipeline, PropositionStore, etc.) are - * present. + * Import this in your application config to expose the proposition extraction, memory, discovery, + * and schema-governance endpoints. Nothing is component-scanned — the controllers only activate when + * this class is imported AND the required beans (PropositionPipeline, PropositionStore, + * GovernanceOperationsService, etc.) are present. * * ```java * @Configuration @@ -37,5 +40,38 @@ import org.springframework.context.annotation.Import PropositionPipelineController::class, MemoryController::class, DiscoveryController::class, + GovernanceControllerImport::class, ) class DiceRestConfiguration + +/** + * Brings [GovernanceController] in last, once every other bean definition is on the registry. + * + * The other three controllers are named directly in the `@Import` above and carry their own + * `@ConditionalOnBean`. That condition is answered while the importing configuration class is being + * read, so it can see the host's own beans and it cannot see anything an auto-configuration will + * contribute later — Spring Boot registers auto-configuration bean definitions after every + * configuration class the application imported. + * + * `GovernanceController` needs a `GovernanceOperationsService`, and that service is exactly such a + * bean: `MetamodelAutoConfiguration` in `dice-storage-autoconfigure` builds it for a host that + * declared a schema. Naming the controller in the `@Import` list would therefore leave it switched + * off in every application that got its governance loop from the auto-configuration. + * + * A [DeferredImportSelector] is the fix, because Spring processes deferred imports at the end of the + * configuration-parsing round and this one declares the lowest precedence, so it runs behind Spring + * Boot's own auto-configuration selector. By the time the controller's condition is asked, the + * governance service is either on the registry or it never will be, and the answer is right in both + * directions: the controller appears for a host whose governance loop is wired, and stays away — + * with a clean context and no `/api/v1/metamodel` route — for a host that declared no schema, killed + * the loop with `embabel.dice.metamodel.enabled=false`, runs `drift.mode=off`, or uses the in-memory + * backend that has no drift log to read. + */ +internal class GovernanceControllerImport : DeferredImportSelector, Ordered { + + override fun selectImports(importingClassMetadata: AnnotationMetadata): Array = + arrayOf(GovernanceController::class.java.name) + + /** Behind Spring Boot's auto-configuration selector, which sits one step above this. */ + override fun getOrder(): Int = Ordered.LOWEST_PRECEDENCE +} diff --git a/dice/src/main/kotlin/com/embabel/dice/web/rest/GovernanceController.kt b/dice/src/main/kotlin/com/embabel/dice/web/rest/GovernanceController.kt index 341c7a4a..403d4caa 100644 --- a/dice/src/main/kotlin/com/embabel/dice/web/rest/GovernanceController.kt +++ b/dice/src/main/kotlin/com/embabel/dice/web/rest/GovernanceController.kt @@ -22,7 +22,10 @@ import com.embabel.dice.governance.GovernanceOperationsService import com.embabel.dice.governance.GovernanceRequestException import com.embabel.dice.governance.ReleasedPropositionDto import com.embabel.dice.governance.parseSince +import org.jetbrains.annotations.ApiStatus import org.slf4j.LoggerFactory +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean import org.springframework.http.HttpStatus import org.springframework.http.ResponseEntity import org.springframework.web.bind.annotation.ExceptionHandler @@ -45,16 +48,22 @@ import org.springframework.web.bind.annotation.RestController * settings use, and a per-context operation names its context in the path the way * [DiscoveryController] does. `GET` reads, `POST` runs a check or performs a release. * - * This controller is not component-scanned. `MetamodelAutoConfiguration` in - * `dice-storage-autoconfigure` registers it, under the same conditions as the rest of the governance - * loop plus a servlet web application and Spring MVC on the classpath. A host that wants the - * governance loop with no HTTP surface at all switches this one auto-configuration off: + * ## How it switches on * - * ```yaml - * spring: - * autoconfigure: - * exclude: com.embabel.dice.storage.autoconfigure.GovernanceHttpAutoConfiguration - * ``` + * This controller is not component-scanned, and it has no auto-configuration of its own. It goes on + * the context through [DiceRestConfiguration], the one import a host uses to open any DICE REST + * surface, and only when a [GovernanceOperationsService] bean is there to answer the routes. Two + * decisions, both the host's: import DICE REST, and wire the governance loop. + * + * So a host that imports [DiceRestConfiguration] and declared no schema resolves zero + * `/api/v1/metamodel` URLs and starts cleanly, and a host that wants the governance loop through + * agent tools or its own code with no endpoint open leaves the import out. See + * [GovernanceControllerImport] for why the condition is asked late enough to see a service the + * auto-configuration built. + * + * `@ConditionalOnMissingBean` lets a host put these operations somewhere else — a different path, + * extra authorization, a shape of its own — by declaring its own `GovernanceController` bean and + * still importing [DiceRestConfiguration] for the other controllers. This one then backs off. * * Every read is bounded. `limit` is clamped by the service, and a value outside its range answers * `400` with the bound named in the body, so an operator who asked for too much can see what to ask @@ -63,8 +72,11 @@ import org.springframework.web.bind.annotation.RestController * * @param operations The single governance service every route delegates to. */ +@ApiStatus.Experimental @RestController @RequestMapping("/api/v1/metamodel") +@ConditionalOnBean(GovernanceOperationsService::class) +@ConditionalOnMissingBean(GovernanceController::class) class GovernanceController( private val operations: GovernanceOperationsService, ) { diff --git a/docs/design/architecture.md b/docs/design/architecture.md index 308c705e..299239e2 100644 --- a/docs/design/architecture.md +++ b/docs/design/architecture.md @@ -13,7 +13,7 @@ DICE is a multi-module Maven build. Each module's intent, and what it's allowed |---|---| | `dice` | The core: proposition model, pipeline, gates, projection interfaces, query facades, agent tools, REST controllers. In-memory implementations only — no database driver. | | `dice-storage` | The durable Neo4j backend: `Drivine`-based repository, graph/Prolog/lineage projectors, schema and index bootstrap, and the governance persistence side: `MetamodelVersionStore`, the `DriftReportStore` drift log, and the `ObservedSchemaSource` that asks the live graph what it holds, excluding dice's own bookkeeping labels and edges so governance doesn't observe itself. Depends on `dice` and `dice-metamodel`. | -| `dice-storage-autoconfigure` | Spring Boot autoconfiguration that wires `dice-storage`'s beans (repository, projectors, trust scorer) into a host application, plus the schema-governance loop: version store, drift log, observed-schema source, differ, quarantine policy and drift runner. That loop is wired only when the application supplies a `DeclaredSchemaSource` bean, and its escalation tier is one property, `off` / `observe` / `quarantine`, defaulting to `observe`. See [metamodel-wiring.md](metamodel-wiring.md). Depends on `dice-storage`. | +| `dice-storage-autoconfigure` | Spring Boot autoconfiguration that wires `dice-storage`'s beans (repository, projectors, trust scorer) into a host application, plus the schema-governance loop: version store, drift log, observed-schema source, differ, quarantine policy and drift runner. That loop is wired only when the application supplies a `DeclaredSchemaSource` bean, and one property, `off` / `observe`, defaulting to `observe`, picks whether a drift runner bean is registered. No property quarantines anything; a host quarantines by calling `DriftSweepCapable.sweep`. See [metamodel-wiring.md](metamodel-wiring.md). Depends on `dice-storage`. | | `dice-ingestion` | Content-hash dedup ledger and source adapters that sit in front of `PropositionPipeline`, so the same artifact is never extracted twice concurrently. Depends on `dice`. | | `dice-report` | Rationale and structured report generation over propositions and their lineage. Depends on `dice`. | | `dice-metamodel` | Schema governance: content-hash stamps over the governed part of a `DataDictionary`, the declared-schema contract, the version and drift-report store contracts, diffing, drift checking, and non-destructive quarantine. A leaf over `embabel-agent-api`, with no dependency on `dice`; `dice-storage` implements its store contracts. | diff --git a/docs/design/metamodel-wiring.md b/docs/design/metamodel-wiring.md index 7afa7af4..edea8fb0 100644 --- a/docs/design/metamodel-wiring.md +++ b/docs/design/metamodel-wiring.md @@ -46,9 +46,13 @@ There is no scheduler and no startup hook. A check happens when the application stamps, observes, runs both comparisons and writes a `DriftReport`, and no path through it moves a proposition. -Acting on what a check found is a second, deliberate step. The application reads -`DriftCheckResult.quarantineDiff`, decides, and calls `DriftSweepCapable.sweep` once per context it -means to reconcile. The wiring supplies the object that performs a sweep and never calls it. +**Quarantine is applied by exactly one thing: a host calling `DriftSweepCapable.sweep`.** That is a +deliberate narrowing, and it is worth stating plainly because the obvious alternatives were all +available. Nothing quarantines on a schedule, nothing quarantines while the context starts, no +property switches quarantining on, and a drift check that found everything wrong still moves no +proposition. The application reads `DriftCheckResult.quarantineDiff`, decides, and calls +`DriftSweepCapable.sweep` once per context it means to reconcile. The wiring supplies the object +that performs a sweep and leaves the calling to the host. ```kotlin val result = runner.run() @@ -75,7 +79,7 @@ propositions by itself would be exactly the surprise this shape exists to remove ## Status transitions reach your listeners -A sweep moves a proposition to `STALE`, and things downstream need to hear about it. +A sweep moves a proposition to `QUARANTINED`, and things downstream need to hear about it. `ProjectionLineageStaleCascade` marks every projection record derived from that proposition stale, and it can only do so if the transition is announced. @@ -186,7 +190,9 @@ proposition. `GovernanceOperationsService` (in `dice`) is the whole surface. `GovernanceController` puts it on HTTP and `GovernanceTools` exposes it as agent tools, and both call the one service, so an operator -reading over HTTP and an agent reading through tools can never see different answers. +reading over HTTP and an agent reading through tools can never see different answers. The service is +the only one of the three the wiring supplies; the host decides about the other two, by importing +the DICE REST surface and by building the tools. | Operation | HTTP | Tool | | --- | --- | --- | @@ -216,48 +222,83 @@ proposition belonging to another context answers `404` and is left exactly as it release restores the status the proposition carried before quarantine, clears the quarantine metadata, and answers the state the proposition is in afterwards. -There is no operation that releases a whole report's worth of propositions. Nothing in the model ties -a quarantined proposition back to the report whose application quarantined it — a `DriftReport` has -no identity beyond its natural key, and the reason a sweep writes names the two schemas and nothing -about the check that produced them. +**Release works one proposition at a time, by design.** An operator who wants twenty propositions +back makes twenty calls. The narrowing follows from what the model records: a `DriftReport` carries +no identity a reason could name — its natural key is the schema name, the version hash, the context +and the capture instant — and the reason a sweep writes on a held proposition names the two schemas +and nothing about the check that produced them. So there is no honest way to ask for "everything +this report quarantined", and inventing a report id to make the bulk route possible would put a +promise in the API that the stored data cannot keep. -### Conditions +### What the wiring supplies, and what the host supplies -The service and the tools are registered by `MetamodelAutoConfiguration`, under the loop's own -conditions plus the three collaborators the surface needs: +`MetamodelAutoConfiguration` registers one thing here: the service. -| Bean | Registered when | +| Piece | Where it comes from | | --- | --- | -| `GovernanceOperationsService` | the governance loop is wired, and a `DriftReportStore`, a `DriftCheckRunner`, a `DriftSweepCapable` and a `PropositionStore` are all on the context | -| `GovernanceTools` | a `GovernanceOperationsService` is on the context | -| `GovernanceController` | a `GovernanceOperationsService` is on the context, the application is a servlet web application, and Spring MVC is on the classpath | +| `GovernanceOperationsService` | the auto-configuration, when the governance loop is wired and a `DriftReportStore`, a `DriftCheckRunner`, a `DriftSweepCapable` and a `PropositionStore` are all on the context | +| `GovernanceController` | `@Import(DiceRestConfiguration.class)` in the host's own configuration, when a `GovernanceOperationsService` is on the context | +| `GovernanceTools` | the host, calling `GovernanceTools.asTools(service)` | So `enabled=false`, no `DeclaredSchemaSource`, `drift.mode=off`, and the default in-memory backend -each remove the surface along with the rest of the loop. With no runner there is no check to run, and -a surface that could answer only half its own routes would be worse than none. +each remove the service along with the rest of the loop, and the two front ends go with it. With no +runner there is no check to run, and a surface that could answer only half its own routes would be +worse than none. -All three are `@ConditionalOnMissingBean`, so an application that defines its own service, tools or -controller keeps it — that is how a host puts the routes on a different path or behind extra -authorization. +The service is `@ConditionalOnMissingBean`, so an application that defines its own keeps it and both +front ends then run through that one. -Registering these beans runs nothing. Building the context stamps no version, writes no report and +Registering the service runs nothing. Building the context stamps no version, writes no report and moves no proposition; a check happens when somebody calls one. -### Turning the HTTP surface off +### HTTP switches on with the rest of the DICE REST surface -The surface is worth having through agent tools or a host's own code without opening an endpoint. -`GovernanceController` therefore has its own auto-configuration, so switching it off is one line and -leaves the service and the tools wired: +`GovernanceController` has no auto-configuration. It goes on the context through +`DiceRestConfiguration`, the single import that opens any DICE REST surface, which is how +`PropositionPipelineController`, `MemoryController` and `DiscoveryController` have always arrived: -```yaml -spring: - autoconfigure: - exclude: com.embabel.dice.storage.autoconfigure.GovernanceHttpAutoConfiguration +```java +@Configuration +@Import(DiceRestConfiguration.class) +public class MyAppConfiguration { } ``` +Two host decisions therefore have to line up before a `/api/v1/metamodel` URL resolves: import DICE +REST, and wire the governance loop. Leave the import out and the loop still works through the agent +tools and the host's own code, with no endpoint open. Import it with no schema declared and the +application starts clean, publishing its other DICE routes and none of these. + +There is one wrinkle worth knowing, because it decides where the controller can be named. +`@ConditionalOnBean` on an imported class is answered while Spring reads the configuration class +that imported it, and Spring Boot registers auto-configuration bean definitions after that point. A +plain entry in `DiceRestConfiguration`'s `@Import` list would therefore see no +`GovernanceOperationsService` in any application whose loop came from the auto-configuration — +which is every application following this note. `GovernanceControllerImport`, a +`DeferredImportSelector` with the lowest precedence, is what puts the question after Spring Boot's +own auto-configuration selector has answered. `GovernanceOperatorAutoConfigurationTest` pins all +four combinations of import and loop, reading the live handler mapping so an empty answer means a +client gets a 404. + +A host that wants these operations somewhere else — a different path, extra authorization, a shape +of its own — declares its own `GovernanceController` bean and keeps the import for the other +controllers; the shipped one is `@ConditionalOnMissingBean` and backs off. + The release route changes stored data, so put it behind whatever authorization the host uses for its administrative endpoints. +### The agent tools are yours to build + +Nothing registers a `GovernanceTools` bean. No DICE tool object is a bean — `DiscoveryTools`, +`GraphQueryTools` and `Memory` are all constructed by the application that wants them, because a +tool object is only useful once it has been registered with a particular agent or MCP server, and +that registration is the host's call: + +```kotlin +@Bean +fun governanceTools(operations: GovernanceOperationsService): List = + GovernanceTools.asTools(operations) +``` + ## Property reference | Property | Default | Meaning | From d05f1e7e417d746ad43b6e4f5758d73fb5ca8946 Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Sun, 6 Sep 2026 23:06:41 -0400 Subject: [PATCH 08/10] Manage the mockito-kotlin version from the root pom embabel-agent-dependencies manages neither mockito-kotlin nor the JetBrains annotations, so the root pom now carries both versions and the module poms declare the artifacts without one. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- dice-storage-autoconfigure/pom.xml | 4 +--- dice/pom.xml | 1 - pom.xml | 11 +++++++++++ 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/dice-storage-autoconfigure/pom.xml b/dice-storage-autoconfigure/pom.xml index 4d15e534..5527c141 100644 --- a/dice-storage-autoconfigure/pom.xml +++ b/dice-storage-autoconfigure/pom.xml @@ -82,14 +82,13 @@ org.jetbrains annotations - 26.0.1 @@ -124,7 +123,6 @@ org.mockito.kotlin mockito-kotlin - 5.4.0 test diff --git a/dice/pom.xml b/dice/pom.xml index 029c32a9..e02a04a7 100644 --- a/dice/pom.xml +++ b/dice/pom.xml @@ -132,7 +132,6 @@ org.mockito.kotlin mockito-kotlin - 5.4.0 test diff --git a/pom.xml b/pom.xml index 988fb0fc..663a238f 100644 --- a/pom.xml +++ b/pom.xml @@ -54,6 +54,11 @@ embabel-agent-dependencies, so the version is set here and every module inherits it. --> 26.0.2 + + 5.4.0 @@ -105,6 +110,12 @@ annotations ${jetbrains.annotations.version} + + + org.mockito.kotlin + mockito-kotlin + ${mockito-kotlin.version} + From eeb94ddb38d76883ca47aa0416ad42969c7e21bd Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Sun, 6 Sep 2026 23:06:41 -0400 Subject: [PATCH 09/10] Write the metamodel property prefixes once The metamodel auto-configuration and its properties read every prefix from DicePropertyPrefixes, so the property names live in one place. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- .../autoconfigure/DicePropertyPrefixes.kt | 26 +++++++++++++++++++ .../MetamodelAutoConfiguration.kt | 16 ++++++------ .../autoconfigure/MetamodelProperties.kt | 2 +- 3 files changed, 35 insertions(+), 9 deletions(-) create mode 100644 dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/DicePropertyPrefixes.kt diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/DicePropertyPrefixes.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/DicePropertyPrefixes.kt new file mode 100644 index 00000000..e1c2e071 --- /dev/null +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/DicePropertyPrefixes.kt @@ -0,0 +1,26 @@ +/* + * Copyright 2024-2026 Embabel Pty Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.embabel.dice.storage.autoconfigure + +/** + * Configuration prefixes the dice modules read, kept in one place so an annotation cannot misspell one. + */ +internal object DicePropertyPrefixes { + const val DICE = "embabel.dice" + const val STORE = "$DICE.store" + const val METAMODEL = "$DICE.metamodel" + const val METAMODEL_DRIFT = "$METAMODEL.drift" +} diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt index 77137606..205d24b9 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfiguration.kt @@ -124,7 +124,7 @@ import org.springframework.context.annotation.Bean @AutoConfiguration(after = [DiceStorageAutoConfiguration::class]) @ConditionalOnBean(DeclaredSchemaSource::class) @ConditionalOnProperty( - prefix = "embabel.dice.metamodel", + prefix = DicePropertyPrefixes.METAMODEL, name = ["enabled"], havingValue = "true", matchIfMissing = true, @@ -146,7 +146,7 @@ class MetamodelAutoConfiguration { * their own gets both, and these are still required. */ @Bean - @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") + @ConditionalOnProperty(prefix = DicePropertyPrefixes.STORE, name = ["type"], havingValue = "graph") fun metamodelSchema(): SchemaCatalog = SchemaCatalog.of(MetamodelSchema.specs()) /** @@ -155,7 +155,7 @@ class MetamodelAutoConfiguration { * this bean governance would observe its own report bookkeeping as drift. */ @Bean - @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") + @ConditionalOnProperty(prefix = DicePropertyPrefixes.STORE, name = ["type"], havingValue = "graph") fun metamodelStorageSchema(): DiceStorageSchema = MetamodelSchema /** @@ -168,7 +168,7 @@ class MetamodelAutoConfiguration { * capability at injection time and leave a `SweptBaselineStore` injection point unresolvable. */ @Bean - @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") + @ConditionalOnProperty(prefix = DicePropertyPrefixes.STORE, name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(MetamodelVersionStore::class) fun drivineMetamodelVersionStore(persistenceManager: PersistenceManager): SweptBaselineStore { logger.debug("Wiring graph MetamodelVersionStore: DrivineMetamodelVersionStore") @@ -176,7 +176,7 @@ class MetamodelAutoConfiguration { } @Bean - @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") + @ConditionalOnProperty(prefix = DicePropertyPrefixes.STORE, name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(DriftReportStore::class) fun drivineDriftReportStore(persistenceManager: PersistenceManager): DriftReportStore { logger.debug("Wiring graph DriftReportStore: DrivineDriftReportStore") @@ -185,13 +185,13 @@ class MetamodelAutoConfiguration { /** What dice owns in this application, derived from the registered storage schemas. */ @Bean - @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") + @ConditionalOnProperty(prefix = DicePropertyPrefixes.STORE, name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(DiceOwnedSchema::class) fun diceOwnedSchema(schemas: List): DiceOwnedSchema = DiceOwnedSchema.of(schemas) @Bean - @ConditionalOnProperty(prefix = "embabel.dice.store", name = ["type"], havingValue = "graph") + @ConditionalOnProperty(prefix = DicePropertyPrefixes.STORE, name = ["type"], havingValue = "graph") @ConditionalOnMissingBean(ObservedSchemaSource::class) fun drivineObservedSchemaSource( persistenceManager: PersistenceManager, @@ -310,7 +310,7 @@ class MetamodelAutoConfiguration { @ConditionalOnMissingBean(DriftCheckRunner::class) @ConditionalOnBean(value = [ObservedSchemaSource::class, DriftReportStore::class]) @ConditionalOnProperty( - prefix = "embabel.dice.metamodel.drift", + prefix = DicePropertyPrefixes.METAMODEL_DRIFT, name = ["mode"], havingValue = "observe", matchIfMissing = true, diff --git a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt index f6f8df9b..36fb48c9 100644 --- a/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt +++ b/dice-storage-autoconfigure/src/main/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelProperties.kt @@ -52,7 +52,7 @@ enum class DriftMode { * once one exists. */ @ApiStatus.Experimental -@ConfigurationProperties(prefix = "embabel.dice.metamodel") +@ConfigurationProperties(prefix = DicePropertyPrefixes.METAMODEL) data class MetamodelProperties( /** From 408705ffd6898c9e17840e20c6ee48dab40c5ec5 Mon Sep 17 00:00:00 2001 From: James Dunnam <7660553+jimador@users.noreply.github.com> Date: Sun, 6 Sep 2026 23:06:41 -0400 Subject: [PATCH 10/10] State that the Drivine schema label is excluded from drift DrivineObservedSchemaSource keeps _DrivineSchema out of the observed types through INFRASTRUCTURE_LABELS. The test comment said the opposite, and the test now asserts the exclusion. Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com> --- .../MetamodelAutoConfigurationIntegrationTest.kt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt index a2d82a21..82314beb 100644 --- a/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt +++ b/dice-storage-autoconfigure/src/test/kotlin/com/embabel/dice/storage/autoconfigure/MetamodelAutoConfigurationIntegrationTest.kt @@ -161,9 +161,8 @@ class MetamodelAutoConfigurationIntegrationTest { * auto-configured stores and observed-schema source really do line up with it. * * Drivine's `SchemaManager` writes a `_DrivineSchema` inventory node whenever it applies a - * `SchemaCatalog`, and that label is currently reported as drift, since `DiceOwnedSchema` - * describes dice's own labels and knows nothing about Drivine's. Fixing it is a `dice-storage` - * change and there is no assertion for it here. + * `SchemaCatalog`. `DrivineObservedSchemaSource` keeps that label out of the observed types + * through `INFRASTRUCTURE_LABELS`, and the second assertion below holds it to that. */ @Test fun `governance never reports its own bookkeeping as drift`() { @@ -172,6 +171,7 @@ class MetamodelAutoConfigurationIntegrationTest { assertThat(second.driftedEntityTypes).contains("Ghost") assertThat(second.driftedEntityTypes).doesNotContainAnyElementsOf(MetamodelSchema.LABELS) + assertThat(second.driftedEntityTypes).doesNotContain("_DrivineSchema") } }