Extraction runs: lifecycle machine and store contract

[jimador]

PR 7 of the extraction-integrity train (refs #67), stacked on #95. The run lifecycle and store contract, in dice core. The state machine lives in one place: save accepts only RUNNING runs, and ExtractionRunTransition.applyTo under store compare-and-set is the only way a run reaches COMPLETED/FAILED/CANCELLED — nothing can manufacture a terminal run without a store. Idempotency has teeth: replaying the same terminal payload succeeds; an incompatible rewrite is rejected by insert-or-compare on a SHA-256 fingerprint of the transition payload, deterministic and golden-literal-locked. ExtractionRunStore gives tenant-scoped bounded pages (scope before limit), bounded cycle-safe chain walks, whole-lineage lookup in one read off the denormalized root, and invocation persistence that survives header saves. The cross-backend contract suite carries the discriminating cases — interleaved replay, lineage disagreement, the terminal-by-terminal matrix, a transition race, invocation preservation — so the Drivine store inherits the semantics.

Changed in this review round:

• The terminal fingerprint hashes transition identity only — (status, finishedAt). Counts and failures travel as data on the transition and still participate in its equality, so Return typed product outcomes for batched and linked extraction passes #69's typed product outcomes change no persisted fingerprint format. TERMINAL_VERSION moved to xrun-terminal:v2 once, before any release, and the design doc states the choice.
• Applied transitions are announced. ExtractionRunTransitioned is a new DiceEvent carrying the terminal run; the in-memory store takes a DiceEventListener collaborator and emits after the write lands, gated on the result being applied. Nothing is wired automatically — the audit projection and host listeners have their subscription point.
• Header writes carry a version under compare-and-set. A stale save used to rebuild every header field, so it could erase revisions, counts, failures, profile or fingerprints another writer had stored; it is now rejected. Terminal invocation records are locked the same way: an identical replay succeeds, an incompatible rewrite is refused on the fingerprint.
• ProtectedContentRef is deferred out: DICE had no writer, reader, or retention behavior for it; it returns with the first runtime path that uses it. The doc states the reference as it now stands.

Breaking changes: none. New types only; the header-version parameter's binary consequences are enumerated in the CHANGELOG (which descriptors it preserves, which it moves, and what happens to a caller that never passes one).

Opt-in and status: EXPERIMENTAL, @ApiStatus.Experimental on every type. Nothing calls the store yet; the coordinator is the first real caller and is sequenced next.

stateDiagram-v2
    [*] --> RUNNING: save accepts RUNNING only
    RUNNING --> RUNNING: save under versioned compare-and-set
    RUNNING --> SUCCEEDED: transition
    RUNNING --> FAILED: transition
    RUNNING --> CANCELLED: transition
    SUCCEEDED --> SUCCEEDED: identical replay succeeds
    FAILED --> FAILED: identical replay succeeds
    CANCELLED --> CANCELLED: identical replay succeeds

Loading

Two write doors, and what each promises:

flowchart TD
    A[save: whole header] --> V{version matches stored?}
    V -- no --> R[rejected, stored row untouched]
    V -- yes --> W[header replaced, invocations merged by identity]
    B[recordInvocation: one child row] --> T{record already terminal?}
    T -- yes --> F{payload fingerprint identical?}
    F -- yes --> K[accepted as replay, nothing moves]
    F -- no --> R2[rejected]
    T -- no --> M[child row inserted or compared]
    W --> E[applied terminal transition emits ExtractionRunTransitioned]

Loading

Closed value sets are exercised by iterating their own entries, so a newly declared constant is covered the day it is added.

• ExtractionRunStore.save states the returned version for each kind of save. A first save returns version 0 (the version the caller named, now confirmed by the store), an accepted update the stored version plus one, and a replay the stored version unchanged. The KDoc used to say a first save returned the caller's version plus one. The contract test pins all three.

[igordayen]

from CODEX:

Findings:

• P2 InMemoryExtractionRunStore retains every run forever
  dice/src/main/kotlin/com/embabel/dice/proposition/extraction/InMemoryExtractionRunStore.kt:79-82 keeps all run headers and terminal
  fingerprints in maps, and ExtractionRunStore exposes no delete/prune/retention API. Bounded reads only bound the returned page, not
  stored memory.
  Example: a host using the shipped in-memory store for 50k extractions/day keeps every completed run and fingerprint until JVM
  restart. Either add explicit retention/pruning support, cap the in-memory implementation, or make it clearly dev/test-only so a
  long-running host cannot accidentally choose an unbounded audit store.

• P2 reads become global stop-the-world scans as the store grows
  InMemoryExtractionRunStore.page() filters and sorts runs.values while holding the single monitor (:260-267). runsInContext,
  childrenOf, and runsOfRoot all use that path. The API returns at most limit, but the work is still O(total stored runs), across all
  tenants, under the same lock used by save, recordInvocation, and transition.
  Example: after millions of historical runs, runsInContext("tenant-a", 10, since = null) scans/sorts the whole map and blocks all
  writers while it does so. Tenant-indexed maps or per-query indexes would keep the reference store’s thread-safety claim from
  degrading under retention.

• P3 listener failures can make a committed transition look failed to the caller
  transition() commits the terminal run, releases the lock, then calls listener.onEvent(...) without catching
  (InMemoryExtractionRunStore.kt:178-203). If the listener throws, the run is already terminal, but the caller sees an exception;
  retrying the same transition returns REPLAYED and emits no event.
  SafeDiceEventListener exists, but the store does not wrap the collaborator itself. If “transition succeeded” must not depend on
  listener quality, wrap internally or document this as a hard contract on callers.

[igordayen]

@jimador - would you please see few inquiries from myself and CODEX. Do we need to consider eviction policy to control memory growth?
Thanks

[jimador]

All three addressed.

• Retention: InMemoryExtractionRunStore takes maxRuns (default 10_000) and evicts the oldest ended runs past it, never a running one; the class doc says what a host running the reference store in production is accepting. e5be31b.
• Reads: each tenant's runs sit in their own index, so a page reads that tenant only; filter, order, limit is unchanged. b11703a.
• Listener: a throwing listener is logged at error and the transition stays committed and reports APPLIED. 84cbd60.

On your eviction question: yes for the reference store, and the cap above is it. For the Drivine store, retention past that is the durable store's own policy, kept for as long as an operator decides, and a sweep on finishedAt would be a separate slice; the design doc now says so.

[igordayen]

@jimador - looks good subject to rebasing, thanks