Skip to content

Add versioned experiment records and explicit evidence collation - #35

Draft
PaulHax wants to merge 1 commit into
portable-response-reportfrom
experiment-records
Draft

PaulHax wants to merge 1 commit into
portable-response-reportfrom
experiment-records

Conversation

@PaulHax

@PaulHax PaulHax commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Comparison membership needs explicit scientific identities

A recipe or run name cannot identify a replicate, establish a retry relationship or select comparable evidence.

Save validated records and explicit selections

Add versioned protocols, attempts, artifacts and evaluations with relationship checks, sourced unknowns, budgets and metric definitions. Collation selects named attempts and evaluations and retains included, excluded and missing cells in an immutable content-identified snapshot. Read-only frozen-defender and co-training adapters preserve unresolved historical membership; study-specific rules establish comparability.

The frozen analyzer verifies the saved configuration and analyzer/helper identity and retains its source with the report.

Part of #29, #31 and #33. Based on #34.

@PaulHax

PaulHax commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

The review dataset exercises the adapters with a real historical co-training SQLite store and retained frozen-Blue evaluations.

Both co-training executions remain distinct, with unknown trial and retry relationships. The frozen-study inventory records 9 expected trial cells, 0 missing cells and 4 unresolved exclusions. Included attempts and evaluations are not counted as additional replicates.

The archive includes the historical store, raw evaluations and verification commands. Its frozen-study inventory is derived; excluded original source archives prevent replaying the full original source-validated collation.

@PaulHax
PaulHax force-pushed the experiment-records branch from 854c068 to 3dba0fb Compare October 29, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant