Skip to content

Extraction runs: the durable run model - #95

Merged
jimador merged 7 commits into
jimador/feat/extraction-profilesfrom
jimador/feat/extraction-run-model
Sep 10, 2026
Merged

jimador merged 7 commits into
jimador/feat/extraction-profilesfrom
jimador/feat/extraction-run-model

Conversation

@jimador

@jimador jimador commented Aug 31, 2026 •

Copy link
Copy Markdown
Collaborator

PR 6 of the extraction-integrity train (refs #67 — the model half), stacked on #94. The durable extraction-run record, as pure JVM value types. ExtractionRun is keyed by (ContextId, ExtractionRunRef) and carries profile, ordered source revisions, pass, parent/superseded refs plus a denormalized root-run ref (OpenLineage's ParentRunFacet pattern — a whole-lineage lookup becomes one indexed read), experiment/cohort refs, prompt/schema/metamodel fingerprints, runtime identity, status, timing, counts, and bounded failures. Requested and observed model facts are separate types with disjoint fields, so a guessed provider-effective value has nowhere to live. Invocation identity is the ordinal in a call plan fixed before any call fires; attempts number retries; completion order never mints. Opaque pseudonymous refs carry actor/request/session identity with serialized-row privacy tests.

Changed in this review round:

  • Failures speak a closed vocabulary. ExtractionFailure has no String parameter, property or field anywhere in its reachable shape and no factory taking a Throwable: code (11 values), stage, a validated providerStatus, one typed measure with its unit, at, and the invocation it names. detail, fromThrowable and the sanitize/cause-chain helpers are removed, so prompt content, PII, credentials and protected text have nothing shaped to hold them and cannot reach durable storage through this type.
  • Run recording aligns with the source-revision contract. The revision bound this type imposed is gone — a run accepts whatever SourceRevisionRef accepts, so extraction and revision queries can no longer succeed where run recording then fails.
  • The lineage root is derived. The constructor is private and childOf derives the root from the parent's own lineage, so a root that contradicts its parent has no public path in; the KDoc records the reflection residual honestly.
  • metamodelFingerprint keeps its field and its KDoc names its one writer: the extraction coordinator, resolving the host's DeclaredSchemaSource — per the decision that per-proposition schema attribution answers through run lineage.

Breaking changes: none. New types only. The failure-vocabulary rework reshapes types introduced earlier in this same unreleased train; no released surface moves.

Opt-in and status: EXPERIMENTAL — every type carries @ApiStatus.Experimental. Nothing here executes; the store contract, the Drivine store, and lineage stamping follow as #98, #99 and #101.

erDiagram
    ExtractionRun ||--o{ ExtractionInvocationRecord : "invocations, identity fixed before any call"
    ExtractionRun ||--o{ ExtractionFailure : "bounded, typed vocabulary, no free text"
    ExtractionRun ||--|| ExtractionRunLineage : "runRef, root, parent, pass"
    ExtractionRun }o--|| ExtractionContentProfileRef : "profile"
    ExtractionRun }o--o{ SourceRevisionRef : "ordered source revisions"
    ExtractionFailure }o--o| ExtractionInvocationRecord : "may name only an attempt the run records"
Loading

The denormalized root is what makes a whole-lineage lookup one indexed read.

Comment thread dice/src/main/kotlin/com/embabel/dice/proposition/extraction/ExtractionRun.kt Outdated
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from a0eb287 to 1288875 Compare September 1, 2026 04:20
@jimador
jimador marked this pull request as ready for review September 1, 2026 04:20
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from 1288875 to afab888 Compare September 2, 2026 10:51
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from afab888 to 7d8a8dd Compare September 2, 2026 14:11
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from 7d8a8dd to 81b6d63 Compare September 2, 2026 15:07
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from 81b6d63 to 074f22c Compare September 2, 2026 15:24
@jimador
jimador requested a review from igordayen September 2, 2026 20:41
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from 074f22c to c73ca0e Compare September 2, 2026 20:50
Comment thread docs/design/extraction-runs.md
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from c73ca0e to d6f1fc2 Compare September 7, 2026 04:14
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from d6f1fc2 to 395c8dd Compare September 8, 2026 20:30
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch 2 times, most recently from 4f380f2 to f4b1988 Compare September 8, 2026 21:07
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch 2 times, most recently from 2001718 to e46b2ab Compare September 9, 2026 00:02
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from e46b2ab to 80082ee Compare September 9, 2026 01:13

@igordayen igordayen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jimador - looks vey good, few comments to consider, thanks

Comment thread docs/design/extraction-runs.md
@jimador
jimador requested a review from igordayen September 9, 2026 21:58

@igordayen igordayen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jimador - looks good, subject to rebase. thanks

@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from 8af4867 to 2ac6915 Compare September 10, 2026 02:02
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from 2ac6915 to d08b783 Compare September 10, 2026 02:10
An ExtractionRun records what produced knowledge: tenant and run identity,
profile, ordered source revisions, lineage with a denormalized root ref so
whole-lineage reads never walk a chain, experiment and cohort refs,
prompt, schema, and metamodel fingerprints, runtime identity, status,
timing, counts, and bounded sanitized failures that can never carry source
text. Requested and observed model facts are separate types with disjoint
fields, so a guessed provider-effective value has nowhere to live.
Invocation identity is the ordinal in a plan fixed before any call fires,
and a failure can only reference an attempt the run has a record of. Every
stored string is bounded and over-long values are rejected, not truncated;
replay fidelity tops out at approximate, on purpose.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Run recording no longer caps sourceKey or sourceRevision length: the
revision contract is opaque and non-blank, so a value a query accepts
can never fail run recording. A real store constraint brings the bound
back if one ever exists. ExtractionRunLineage's constructor goes
private with copy() following it, leaving root() and childOf() as the
only public mints — both derive the root from the parent's own record,
so a root that contradicts the parent chain has no public way in.
Reflection and a future deserializer still reach the constructor; the
docs and a test state that residual until a store slice closes it.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
ExtractionFailure carried free text into durable storage: truncation
and line-break flattening bounded its size while prompt content, PII
and credentials rode through untouched. The type now holds a code, an
optional stage, a provider status in the HTTP range, and a unit-named
measure paired with a number — no String or Throwable parameter exists
anywhere in its reachable shape, so raw source text cannot be
represented. Reflection canaries pin the absence of any free-text
route. ProtectedContentRef returns as specification only: the host owns
writer, reader, retention and expiry, and DICE stores the handle's
contract, never its content. The envelope's metamodelFingerprint names
the extraction coordinator as its producer, and per-proposition schema
attribution answers through run lineage.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The decision lived in envelope KDoc only. The design doc now says what
the versioning doc says in the same words: the run record carries the
declared schema's content hash, the coordinator resolves it from the
host's DeclaredSchemaSource, and per-proposition attribution answers
through run lineage.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
ExtractionRequestedModelConfig already carried the same six hyperparameters
the framework's LlmOptions does, under the same names. It now implements
LlmHyperparameters directly, so a caller holding the record already holds
something that reads as the interface, not a lookalike copy.

from(options, modelRole, thinkingFingerprint, selectionFingerprint) builds
a config straight off an LlmOptions a host handed the model, taking
modelRole from options.role by default. No field is renamed and the
constructor descriptor is unchanged; the six fields just gain override.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
ExtractionModelUsage gains a companion factory, from(usage), that reads
a call's counts straight off the framework's Usage: promptTokens becomes
inputTokens, completionTokens becomes outputTokens, and totalTokens
carries across unchanged. cachedInputTokens and reasoningTokens stay
null, because Usage does not report either one.

The record stays its own type. Usage is a final class carrying a native
SDK usage object this record deliberately never stores, so extending it
is not an option and converting from it is.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
@jimador
jimador force-pushed the jimador/feat/extraction-run-model branch from d08b783 to 687e61e Compare September 10, 2026 02:17
@jimador
jimador merged commit 0dc362d into main Sep 10, 2026
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants