Snapshot format v3: additive read-floor bump for recovery provenance markers (v2 resumes without migration)

[S2thend]

Summary

Bump the snapshot format from 2 to 3 to add two per-node recovery markers that format 2 cannot hold, using an additive read-floor design: the engine reads versions {2, 3} and writes 3. Existing v2 snapshots (including all {parent}/{node}@{visit} child siblings) resume unmodified and upgrade lazily on their next persist. No migration tool, no migration step.

Scope: 0.6 — this amends contracts C4.1/C4.4 ("format stays 2, no new keys"), which are pinned for feature 004 and must not change on the 004 branch.

Motivation

Two recovery limitations were confirmed during the 0.5.0 hardening review and are documented as frozen-format trade-offs; both need a durable per-node marker that format 2 forbids:

1. Launch-provenance marker (fixes the O(K) empty-log scans on resume). A recovered parent currently grants a conservative recovery hint to every SUBGRAPH node's first dispatch, so K−1 never-launched nodes each pay a full event-log scan that can only return empty. Filtering by persisted node status was implemented, then refuted and reverted: the resume path resets RUNNING→PENDING, any sibling persist makes that erasure durable, and a crash during recovery then denies the hint to a child that still holds retained log history — re-issuing duplicate child (graph_id, event_id) pairs. See the rejected alternative recorded in research.md R16 and the pinning test test_crash_during_recovery_still_seeds_child_event_ids. A durable launched marker that survives the reset is the sound fix.

2. Failure-origin marker (unblocks recovery from dispatch failures). A propagated NODE_SCHEDULED listener/log failure persists the node as FAILED even though its handler never ran (FR-016), and _raise_if_persisted_failure then blocks hierarchical resume permanently — the only escape hatch is operator deletion of the child snapshot. FR-012's frozen format is the stated reason resume cannot distinguish "handler failed" from "dispatch failed, handler never ran". A failure_origin marker would let resume safely re-execute the latter without reopening the no-retry policy for genuine handler failures (FR-006).

Bundling both markers into one bump matters because each format bump imposes the downgrade cost once (see Costs).

Constitutional basis

Constitution Principle VI permits exactly this: producers/consumers MUST either preserve snapshot()/restore() compatibility for supported versions (plural) or document migration/rejection behavior. Declaring {2, 3} supported with v2 restore compatibility preserved satisfies the first path with zero migration.

Design

• Validator: Scheduler._validate_snapshot_format currently exact-matches SUPPORTED_SNAPSHOT_FORMAT_VERSION == 2. Change to a membership check against a supported set {2, 3}; keep the fail-fast ValueError naming the actual version, the supported versions, and discard-or-migrate guidance for anything else.
• Reader: new keys are optional-on-read (.get(...)) in ExecutionState.restore/NodeRuntimeState.from_dict; existing required keys unchanged, so v2 payloads restore byte-for-byte semantically.
• Writer: snapshot() always emits snapshot_format_version: 3 plus both markers. Every persist writes complete state, so any v2 record is rewritten as v3 the first time it is touched (lazy upgrade-on-write, per record).
• Tri-state semantics (critical): an absent marker means unknown, never a default value.
  • launched absent → conservative recovery hint (today's behavior). Reading absence as "never launched" reintroduces the double-crash duplicate-id bug for every v2-written snapshot; test_crash_during_recovery_still_seeds_child_event_ids uses a markerless snapshot and must keep passing unchanged.
  • failure_origin absent → treat as terminal FAILED (today's behavior); only an explicit dispatch origin permits re-execution.
• Mixed trees: each record is validated/restored independently, so a v3 parent over v2 child siblings (and vice versa) resumes without coordination.
• Stores: SnapshotStore implementations are unaffected — payloads remain opaque; the protocol gains nothing.

Costs / compatibility

• Downgrade is the one-way door: a 0.5 engine reading a v3 snapshot rejects it (correct, guidance-bearing error). Once any v3 record is written, rolling the library back requires discarding or hand-migrating those records. This is the argument for batching both markers into a single bump.
• Forward incompatibility is already the project's established pattern (0.4.x rejects subgraph payloads at NodeKind parse).

Tasks

• Spec amendments (0.6 feature dir): C4.1/C4.4 supported-set wording, FR-012 marker semantics, FR-006/FR-016 interaction for failure_origin, R16 follow-up noting the marker supersedes the rejected status filter
• _validate_snapshot_format: membership check + updated error message listing supported versions
• ExecutionState.snapshot()/restore(): emit v3 + both markers; optional-on-read with tri-state handling
• Recovery-candidate loop: use launched marker when present; absent → conservative hint (unchanged path)
• _raise_if_persisted_failure: permit re-execution only for explicit failure_origin == "dispatch"
• Tests: v2-snapshot resume (markerless → conservative behavior, byte-identical results), v3 round-trip, mixed-version tree resume, upgrade-on-write, downgrade rejection message, double-crash pin unchanged
• Docs: README recovery boundaries, CHANGELOG migration note ("no action required; rollback after first v3 write requires discarding v3 records")

Acceptance criteria

• A store populated entirely by 0.5 (v2) resumes under 0.6 with no operator action and identical results; records upgrade to v3 as they are re-persisted.
• test_crash_during_recovery_still_seeds_child_event_ids passes unmodified.
• Resume of a recovered parent with K subgraph nodes and one in-flight (v3 snapshots) performs exactly 1 child log read, not K.
• A child whose v3 snapshot records failure_origin: dispatch re-executes on resume; a handler-failure snapshot still refuses retry with the existing RuntimeError.
• 0.5 engine rejects a v3 snapshot with the version boundary and operator guidance named.

Non-goals

• No SnapshotStore/EventLog protocol changes.
• No migration tooling (lazy upgrade-on-write covers it).
• No change to the composed log-only-recovery caveat (root ids only without a parent snapshot) — that is inherent to snapshotless recovery, not the format.

References

• Review + fix commits: 6435ff8 (hardening), 5e6b95d (review follow-ups incl. the reverted status filter and the double-crash pin)
• specs/004-subgraph-node/research.md R16 "Alternatives considered" (rejected status-filtered hints)
• specs/004-subgraph-node/contracts/subgraph-behavior.md C4.1/C4.4, C5.5, C5.7
• Constitution Principle VI (.specify/memory/constitution.md)