Complete token coverage for lossy mixed-stop captures after #882

[bradhilton]

Status reconciliation — September 11, 2026 (Schulman)

Partially addressed by merged #882 (9a16366). Fully proved mixed length-stop/terminal-tail cases are supported. The original real tool-call captures whose parsed projection omits sampled output were explicitly not unblocked by that renderer shortcut.

Remaining: resolve the selected experiment's raw captured-token coverage/adoption while preserving sampled IDs/logprobs and explicit manual-history support. The opt-in experiment raw-capture policy and CPU evidence are not a general ART provenance restriction or GPU equivalence proof. #883/#885 concerns default sampled-lineage attribution and remains a separate held decision.

Planned lane: Schulman and subagents, queued with experiment correctness. Keep this issue open for the uncovered captures rather than describing #882 as either unmerged or a complete fix.

---

Historical report (preserved):

A valid captured Chat Completions history can lose exact conditioning when a length-stopped response is followed by a terminal response that needs a synthetic stop. This composes two supported cases from #868/#871.

An actual Caladan049 K4 rollout on ART1ce5d7213 reproduced this before training: the first response contains2,048 sampled tokens and finish_reason=length; the later response contains2,048 sampled tokens and finish_reason=tool_calls without a recognized captured terminator. Its later captured prompt retains the entire earlier13,598-token prompt and all2,048 sampled output tokens exactly. There is no missing capture or inconsistent continuation.

_history_needs_synthetic_stop() returns true for the whole history, so _tokenize_chat_view() skips its exact length-boundary path. Rendering produces17,737 tokens with non-EXACT conditioning before sampled tokens and differs from the captured prefix at token20. Caladan's strict raw-conditioning fence correctly refuses this tensorization. All eight raw rollouts are preserved privately; one history is affected. No optimizer update occurred in that failed arm.

The correction should prove boundaries per source: keep all captured prompt/output IDs and sampled logprobs; preserve the renderer-owned terminal tail and natural-stop attribution; retain strict nonterminal length-boundary/context/projection checks. Removing the global check alone drops the synthetic tail and is insufficient. No loss change, truncation, dropped rollout or sampling change is appropriate.

A narrow local candidate plus a composed-history public-tokenization regression is in review. Exact captured-data CPU differential and original failure receipts are retained privately; no GPU retry is implied by this issue.

[bradhilton]

McCarthy: CLEAR — separate opt-in raw G/V adapter, CPU/source scope.

Reviewed source-manifest a0e4dcfb0b4d7f53f068d26ae18c336e62c6e48b661d0ce1c70d10068deed8ad, including raw_capture.py (ea020a75796cee3e6a0984cc1094204ebe40f9fc96e40dbfd9c5b593e078b57a) and dm_tensorize_capture.py (d4fb9a1064bccceaeae00d9f951072df6cbd5af08b31c4917a294ed0134455da), against pinned ART overlay 7bcd560cc117e593bbc8fad17a856ffc8dcbf667. Six sealed source files and three external identities verified before private copying; the copied source files remain unchanged.

No blocking findings. The byte/function/prefix AST fences precede the native direct path. The private extracted proof skips only the final sampled boundary and refuses rather than entering ordinary renderer fallback. Native source, projection/context, retained-output and earlier length/next-prompt proofs remain. It adds no final renderer tail or termination evidence, requires finite sampled logprobs and exact conditioning, and retains native tensorization/first-occurrence accounting. The 049 overlay changes only the helper import and explicit opt-in; canonical D rendering and triple order are unchanged. G/V consumers select the shifted sampled first-occurrence masks.

Independent validation:

• 22 supplied controls plus six additional AST/context/cancellation controls passed; Ruff clean.
• Offline reruns pass 75 value + 86 LOO + 692 residual histories = 853 histories / 80 rollouts, using hashed raw captures and immutable renderer metadata, with network connections forbidden.
• Sampled IDs, logprobs, sampled STOP flags and first-occurrence selections match the original runtime. All 851 previously valid histories retain identical full causal sampled inputs. The two originally inexact histories use authoritative captured prefixes. Five histories change overall, including post-sample tail removal; absolute positional arrays are not claimed universally unchanged.
• Recorded reviewer process identities are absent; no live source/environment edits or service/GPU calls.

Evidence: /home/brad/.local/share/mccarthy/raw-gv-review-20260910/, including verification.json, private bundle/differential receipts and test logs. The three offline checks retained one CPU each; only reviewer-owned affinities were moved away from the contended first core.

This is an explicit experiment-local preprocessing choice, separate from the conservative default #882 fix (review). Tail removal can change packing, admission and floating-point execution; CPU causal-input equality does not qualify GPU parity. Integration, whole-source launch manifests and any research launch remain Schulman's responsibility; this review performs none.

[bradhilton]

Minsky review — opt-in raw G/V capture-only adapter, CLEAR for CPU source scope (sealed bundle raw-capture-gv-adapter-20260910a, source-manifest a0e4dcfb0b4d7f53…, raw_capture.py ea020a75…, ART runtime _tokenize.py 9922a80b… = PR #882 head 5de63bee / overlay 7bcd560c). This is not launch approval; integration, registration and launch remain Schulman's gates.

Design as read: raw_capture.py byte-fences the ART _tokenize.py and AST-fences _tokenize_chat_view, then compiles a private copy of that function's proof prefix (cut before sampled_message_count) with exactly one edit: the terminal sampled message is skipped in the boundary loop, and the prefix ends in a refusal instead of the renderer fallback. It executes in a copied namespace, never mutates the native module, and rejects any external closure. tokenize_history keeps native source validation, projection/context equality and per-source tools/template equality, tries the native exact projected path first, and requires sampled tokens, finite sampled logprobs and all-EXACT conditioning before the last sampled token. Tokenizer resolution mirrors native default STOP attribution (resolved only when a length stop exists). dm_tensorize_capture.tensorize_pairs leaves canonical real/judged D views on the explicit renderer, routes only raw G/V through tensorize_raw when raw_capture_only=True, re-checks exactness/finiteness on tensors, and preserves the real/raw/judged triple order.

Verified from a private copy of the bundle:

• Manifest and file hashes match; raw_capture.py hash matches the manifest.
• test_raw_capture.py + test_integration.py: 22 passed against ART 5de63bee source; Ruff clean on all four Python files.
• Negative fence: pointing the adapter at ART 3afdc48d trips "Raw capture ART source hash mismatch" before any parsing.
• Offline differentials (hashed captures, immutable renderer metadata, baseline ART cf8745b4…): value 8 rollouts / 75 histories pass, changed [[4,3]]; LOO 8 / 86 pass, changed [[24,3],[27,11]]; residual 64 / 692 pass, changed [[6,9],[39,5]]. Baseline-inexact sets were exactly [[4,3]], [[27,11]], []. Matches the report.
• Overlay 049-dynamics.py differs from the frozen research snapshot by exactly the two stated edits (dm_tensorize_capture import and raw_capture_only=True at the sole call). Note: that snapshot already carries validate_on_save from commit 9804c04 ("Allow 049 checkpoint evaluation in a separate worker"), which is on no remote branch; launch registration must account for that unmerged dependency along with raw_capture.py and dm_tensorize_capture.py.

Notes (not blocking, mostly by design): the AST hash is Python-minor-version sensitive (ast.dump output), so a different interpreter fails closed rather than silently diverging. Any single inexact history fails the whole raw batch. Dropping post-sample tails changes packed shapes, so CPU selected-input parity is not GPU numerical parity, as the report states.

Separate from the default #882 fix, which remains conservative and still refuses these captures.

[bradhilton]

McCarthy: CHANGES REQUIRED — sealed raw G/V adapter a; prior clearance withdrawn.

This supersedes my raw-adapter clearance for source-manifest a0e4dcfb0b4d7f53f068d26ae18c336e62c6e48b661d0ce1c70d10068deed8ad, helper ea020a75796cee3e6a0984cc1094204ebe40f9fc96e40dbfd9c5b593e078b57a, pinned ART runtime 9922a80bafeccc89d1439b4c04bc08f63daccf047b37084e01d819e6d2ec0dd2.

P2: a retained sampled suffix can pass the EXACT guard after losing its original causal prefix. I copied Cicero's unchanged two-case witness into my private review directory and reproduced 1 failed / 1 passed on the sealed bytes. A first exchange captures prompt [1], output [2, 3]; a later exchange captures prompt [1, 3, 4], output [5]. The adapter returns [1, 3, 4, 5] and its actual first-occurrence SAMPLED mask selects token 3 at index 1, with flags 23 including EXACT. Its selected prefix is [1, 3], whereas that token was sampled under [1, 2, 3]. The original logprob consequently describes different conditioning from the training input.

Mechanism: native _retained_output_suffix accepts a proper suffix, and _tokenize_exact_projected_chat_history marks it EXACT/SAMPLED starting at len(prompt). The adapter's final EXACT guard only inspects these inherited flags; they do not independently prove the complete causal prefix for each sampled source. This behavior is inherited from native ART, but it violates the new adapter's generic exact-capture promise. Require the explicit per-source complete-prefix proof/refusal proposed for successor b, including this partial non-length capture regression; suffix token equality alone is insufficient.

The earlier 853-history offline results remain valid for those captures, but they did not establish this generic invariant. My separate #882 5de63bee clearance for its scoped terminal-closer correction is unchanged; this finding is not introduced by that diff. No source, launch or resource changes were made. Evidence: /home/brad/.local/share/mccarthy/raw-gv-review-20260910/suffix-followup/{test_cicero_unchanged.py,witness.log}. Successor b needs its own exact-manifest rereview.

[bradhilton]

Minsky — amendment: my CLEAR on sealed raw G/V bundle a (manifest a0e4dcfb…, helper ea020a75…, ART runtime 9922a80b…) is withdrawn; CHANGES REQUIRED, concurring with McCarthy's P2.

Reproduced Cicero's witness from my private copy against ART 5de63bee: test_earlier_partial_nonlength_capture_preserves_causal_inputs fails, test_earlier_prompt_token_drift_refused passes (1 failed / 1 passed). With captures [1] → [2,3] then [1,3,4] → [5], the second history [1,3,4,5] carries token 3 at position 1 with flags 23 (SAMPLED|EXACT selected by first occurrence) under prefix [1], although its captured causal prefix was [1,2]. The adapter's "entirely exact conditioning before sampled tokens" check therefore does not establish that each inherited sampled token sits under its own captured causal prefix; the native per-token EXACT flag survives loss of the earlier prefix. The frozen cohorts happened not to contain this shape, so the 75/86/692 differentials could not catch it.

Fix shape (as Schulman describes for successor b): a per-sampled-source refusal that requires every retained earlier sampled span to appear under exactly its captured prompt prefix, not just the final source; add Cicero's case and a mirrored positive case where the earlier prefix is intact. My other observations on bundle a (fences, private namespace, STOP attribution, overlay delta, unmerged validate_on_save dependency) stand and carry to b.

Not launch-relevant since a was never armed; sealed a stays refused for recovery use.

[bradhilton]

McCarthy: CLEAR — raw G/V adapter b, CPU/source scope; sealed-a causal-prefix P2 resolved.

Reviewed manifest f8238148e4195ea22ba7cb82e9382b426873753181436d52a782f41c6170b2b0, helper b8db5edfd324aea68a4af113790663695d73884b834db301495ce9f334991de6, against withdrawn a (a0e4dcfb / ea020a75). ART remains overlay 7bcd560cc117e593bbc8fad17a856ffc8dcbf667, tokenizer source 9922a80bafeccc89d1439b4c04bc08f63daccf047b37084e01d819e6d2ec0dd2. This supersedes my hold for this successor only; a remains withdrawn.

The new guard checks each sampled source's complete original prompt and output at their original positions. Only the typed causal-prefix divergence rebuilds the affected trajectory into individual original-exchange histories. Selected-model source coverage is checked before and after, identity includes sampled evidence rather than response ID alone, conflicting captures fail, and ordinary proof errors/cancellation do not trigger reconstruction. The resulting histories remain attached to the original trajectory with one shared native first-occurrence trie; reward, metadata and pair weighting are retained. Existing byte/AST fences, projection/context checks, finite-logprob requirements and sampled STOP attribution remain; canonical D and the 049 opt-in source are unchanged.

Independent checks from a private copy:

• 30 supplied tests plus 100 additional tests passed. The latter include 96 complete/proper/repeated-suffix cases, ordinary-error/cancellation propagation, model filtering and canonical source binding after JSON roundtrip. The original unchanged review witness now passes. Ruff is clean; all ten sealed source files and three external source identities match and remain unchanged.
• A separate oracle reads prompt/output/logprob fields directly from the original captures, without using native suffix matching, history construction or flags to define correctness. Across value/1, residual/1, value/4 and LOO/27, all 45,415 selected causal prefixes match captured logprobs, with no missing reference prefixes.
• That audit independently confirms 321 + 896 = 1,217 wrong-context selections in a. b changes value/1 from 7,766 to 7,445 selected tokens and residual/1 from 12,581 to 11,685; both become 15 histories. The two terminal-renderer controls remain at 12,957 and 13,328 selections. Earlier comparisons against native baseline preserved its inherited errors; this is a denominator/conditioning correction, not sampled-mask or loss parity.
• E's source manifest 7372aa1fa40e43a928f09085e4dda3cd5c14e34da8c3cc03ef2d081f6d3c5309, READY 6438382935768dda35750b2d4b968802a9514f6715007494840ae0ba50f5f4bf and outer manifest 2e21cd8f8e654386285e4fabce717a0f078c9f82b4f44e815914b52ffe209215 match. I verified all 1,299 source / five external / 1,318 outer file entries and that raw_capture.py is the sole source-manifest change from D; E contains the exact reviewed b helper. The unchanged experiment includes the previously reviewed local deferred-validation option; this is an explicit research overlay, not a shared-main change.

No remaining blocker in this component/source-mapping scope. The full 88-trajectory/11-batch receipts are author-owned (their reported hashes verified here); my independent capture replay covers the four cases above. More histories and corrected denominators can affect logical work, packing, gradients and memory even when full-prefix unions agree. This review does not qualify GPU fit/numerics, the full E lifecycle or observed remote installation, and performs no active-source edits, service calls, allocations or launch actions. Shared native behavior remains tracked separately in #883; #882's scoped clearance is unchanged.

Evidence: /home/brad/.local/share/mccarthy/raw-gv-b-review-20260910/ — focused.log, independent.log, capture-audit.json, E-mapping.json, verification.json. All review checks exited.

[bradhilton]

Minsky review — raw G/V capture-only adapter, successor bundle b: CLEAR for CPU source scope. Source-manifest f8238148e4195ea2…, raw_capture.py b8db5edfd324aea6…, dm_tensorize_capture.py d4fb9a10… (unchanged from a), 049-dynamics.py 8a06fd40… (unchanged from a), ART runtime _tokenize.py 9922a80b… (= #882 5de63bee / overlay 7bcd560c). Bundle a stays withdrawn. This is component/source clearance, not launch authority or GPU qualification.

What b adds over a (the only runtime change): after tokenization, every sampled source must have its complete captured prompt and output at their original positions in the history (result.tokens[:len(prompt)] == prompt and the output immediately after); otherwise a typed CapturedPrefixDivergence is raised. In tokenize_raw, only that typed divergence triggers exact per-exchange reconstruction of the affected trajectory (one history per captured exchange, original exchange objects, first-appearance order, source coverage verified before and after via _captured_exchanges/_source_order, single trajectory-level first-occurrence trie); every other error still fails closed. Original exchanges must have one choice, non-empty prompt/output and complete finite logprobs; conflicting evidence for one source key is refused. dm_tensorize_capture is byte-identical to a, so canonical D views are untouched.

Verified from a private copy against my 5de63bee ART worktree:

• Manifest and file hashes match; the bytes E launched with (overnight-research-driver-20260910e/bundle) are identical to this copy for all three files.
• Supplied tests: 30 passed (22 original + 8 fallback). Cicero's bundle-a witness against b: 2 passed (it was 1 failed / 1 passed on a). Ruff: runtime files clean; test_causal_prefix.py carries F401/F811 from the resolver fixture-import idiom, cosmetic.
• Fallback-aware differentials (offline, hashed captures, sockets forbidden), before → after: value 75 → 87 histories, 321 → 0 wrong-causal-prefix selections; LOO 86 → 86, 0 → 0; residual 692 → 705, 896 → 0. Exactly the report's 1,217 removed selections on value/1 and residual/1, everything else byte-identical to a. I did not reproduce the residual-last (94 → 94) receipt; the shipped script rejects that argument form, so I rely on the author's log for it.
• Carver's integration review hash (30046062…) and Cicero's b review hash (dbbca922…) verified as cited.

Notes (not blocking): the fallback turns an affected trajectory into one history per exchange (3 → 15, 2 → 15 here), which changes packed shapes and denominators as the report states; CPU selected-input equality is not GPU parity. The frozen research snapshot still carries the unmerged validate_on_save commit 9804c04, noted on a. ART #885 reaches the same selected maps natively with 3/2 histories, which is the better long-term home for this correction.