Repository navigation
feat(harness): distinguish recorded context from Assistant tasks [SAP-3292] #1020
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@sapiom/harness": patch | ||
| --- | ||
|
|
||
| Recognize verified continuation context without inventing an unfinished human task. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,6 +17,7 @@ import { | |
| openCodeVisibleParts, | ||
| } from "../shared/opencode-completion.js"; | ||
| import { | ||
| assistantTaskMessages, | ||
| openCodeTurn, | ||
| type OpenCodeTurnMessage, | ||
| } from "../shared/opencode-turn.js"; | ||
|
|
@@ -229,6 +230,9 @@ export function projectAssistantRecord( | |
| binding: AssistantRecordBinding, | ||
| revision: number, | ||
| capturedAt = new Date().toISOString(), | ||
| continuation?: | ||
| | import("../shared/assistant-continuation.js").AssistantContinuationView | ||
| | null, | ||
| ): AssistantRecord { | ||
| try { | ||
| binding = validateAssistantRecordBinding(binding); | ||
|
|
@@ -243,7 +247,7 @@ export function projectAssistantRecord( | |
| }; | ||
| const seen = new Set<string>(), | ||
| partIds = new Set<string>(); | ||
| const native = nativeMessages | ||
| const validated = nativeMessages | ||
| .map((value) => { | ||
| const envelope = object(value), | ||
| info = object(envelope.info), | ||
|
|
@@ -280,6 +284,10 @@ export function projectAssistantRecord( | |
| Number(object(a.info.time).created) - | ||
| Number(object(b.info.time).created), | ||
| ); | ||
| const native = assistantTaskMessages( | ||
| validated as unknown as OpenCodeTurnMessage[], | ||
| continuation, | ||
| ) as unknown as typeof validated; | ||
|
Comment on lines
+287
to
+290
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Existing seed counts never disappear When a retained record contains the seed, Learn moreCapture reconciliation treats every missing turn as native compaction and restores it from the previous record. The new seed filter also expresses removal as a missing turn, so reconciliation cannot distinguish deliberate exclusion from compaction. This affects records created before this behavior ships and records written by any checkpoint that lacked attestation. Example: Revision 4 contains only Recommended fix: Remove an exact attested seed from the previous record during reconciliation, or pass enough provenance into reconciliation to exclude that turn before unioning. Recompute counts so the deliberate removal is not preserved by the monotonic Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| const messages = native as unknown as OpenCodeTurnMessage[]; | ||
| const tokens = openCodeCompletionTokens(messages); | ||
| const turns = new Map<string, AssistantRecord["turns"][number]>(); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,224 @@ | ||
| import { createHash } from "node:crypto"; | ||
| import { expect, it } from "vitest"; | ||
| import { | ||
| openCodeTurn, | ||
| isAssistantContinuationSeed, | ||
| type OpenCodeTurnMessage, | ||
| } from "./opencode-turn.js"; | ||
| import { | ||
| verifyAssistantContinuationView, | ||
| type AssistantContinuationView, | ||
| } from "./assistant-continuation.js"; | ||
| import { projectAssistantRecord } from "../core/assistant-record.js"; | ||
|
|
||
| const text = "Recorded context: the previous draft was completed."; | ||
| const view: AssistantContinuationView = { | ||
| operationId: "11111111-1111-4111-8111-111111111111", | ||
| sourceSessionId: "source", | ||
| sourceRecordRevision: 7, | ||
| capturedAt: "2026-09-14T00:00:00.000Z", | ||
| retainedTurns: 1, | ||
| omittedTurns: 0, | ||
| seed: { | ||
| conversationId: "ses_child", | ||
| messageId: "msg_seed", | ||
| partId: "prt_seed", | ||
| text, | ||
| sha256: createHash("sha256").update(text).digest("hex"), | ||
| }, | ||
| }; | ||
| const seed = (): OpenCodeTurnMessage => ({ | ||
| info: { | ||
| id: "msg_seed", | ||
| sessionID: "ses_child", | ||
| role: "user", | ||
| time: { created: 1 }, | ||
| }, | ||
| parts: [ | ||
| { | ||
| id: "prt_seed", | ||
| messageID: "msg_seed", | ||
| sessionID: "ses_child", | ||
| type: "text", | ||
| synthetic: true, | ||
| ignored: false, | ||
| text, | ||
| metadata: { | ||
| sapiomContinuation: { | ||
| operationId: view.operationId, | ||
| briefHash: view.seed.sha256, | ||
| }, | ||
| }, | ||
| }, | ||
| ], | ||
| }); | ||
| const binding = { | ||
| harnessSessionId: "child", | ||
| cwd: "/workspace", | ||
| conversationId: "ses_child", | ||
| contextAuthorityScope: "a".repeat(64), | ||
| }; | ||
|
|
||
| it("keeps an exact prepared seed ready with no recovery or archived human task", async () => { | ||
| expect(await verifyAssistantContinuationView(view, "ses_child")).toEqual( | ||
| view, | ||
| ); | ||
| expect(openCodeTurn([seed()], "idle", undefined, view)).toEqual({ | ||
| status: "ready", | ||
| }); | ||
| const record = projectAssistantRecord([seed()], binding, 1, undefined, view); | ||
| expect(record).toMatchObject({ | ||
| turns: [], | ||
| turnCount: 0, | ||
| messageCount: 0, | ||
| limitations: [], | ||
| }); | ||
| expect(openCodeTurn([seed()], "busy", undefined, view)).toEqual({ | ||
| status: "working", | ||
| }); | ||
| }); | ||
|
|
||
| it.each([ | ||
| [ | ||
| "message identity", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.info!.id = "msg_other"; | ||
| }, | ||
| ], | ||
| [ | ||
| "native identity", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.info!.sessionID = "ses_other"; | ||
| }, | ||
| ], | ||
| [ | ||
| "part identity", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.id = "prt_other"; | ||
| }, | ||
| ], | ||
| [ | ||
| "part parent", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.messageID = "msg_other"; | ||
| }, | ||
| ], | ||
| [ | ||
| "part session", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.sessionID = "ses_other"; | ||
| }, | ||
| ], | ||
| [ | ||
| "ordinary text", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.synthetic = false; | ||
| }, | ||
| ], | ||
| [ | ||
| "ignored input", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.ignored = true; | ||
| }, | ||
| ], | ||
| [ | ||
| "changed brief", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.text = "new human request"; | ||
| }, | ||
| ], | ||
| [ | ||
| "missing marker", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.metadata = {}; | ||
| }, | ||
| ], | ||
| [ | ||
| "wrong operation", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.metadata!.sapiomContinuation = { | ||
| operationId: "other", | ||
| briefHash: view.seed.sha256, | ||
| }; | ||
| }, | ||
| ], | ||
| [ | ||
| "wrong hash", | ||
| (m: OpenCodeTurnMessage) => { | ||
| m.parts[0]!.metadata!.sapiomContinuation = { | ||
| operationId: view.operationId, | ||
| briefHash: "b".repeat(64), | ||
| }; | ||
| }, | ||
| ], | ||
| [ | ||
| "extra part", | ||
| (m: OpenCodeTurnMessage) => { | ||
| (m.parts as unknown[]).push({ type: "text", text: "user request" }); | ||
| }, | ||
| ], | ||
| ] as const)( | ||
| "does not exempt a %s from normal task status", | ||
| (_label, mutate) => { | ||
| const message = seed(); | ||
| mutate(message); | ||
| expect(isAssistantContinuationSeed(message, view)).toBe(false); | ||
| expect(openCodeTurn([message], "idle", undefined, view)).toEqual({ | ||
| status: "failed", | ||
| }); | ||
| }, | ||
| ); | ||
|
|
||
| it("does not trust a synthetic marker without server attestation or a changed attested brief", async () => { | ||
| expect(openCodeTurn([seed()], "idle")).toEqual({ status: "failed" }); | ||
| expect(projectAssistantRecord([seed()], binding, 1).turnCount).toBe(1); | ||
| expect( | ||
| await verifyAssistantContinuationView({ | ||
| ...view, | ||
| seed: { ...view.seed, text: "changed" }, | ||
| }), | ||
| ).toBeNull(); | ||
| expect(await verifyAssistantContinuationView(view, "ses_other")).toBeNull(); | ||
| }); | ||
|
|
||
| it("counts a later human task and preserves unexpected work attached to the seed", () => { | ||
| const user = seed(); | ||
| user.info!.id = "msg_human"; | ||
| Object.assign(user.parts[0]!, { | ||
| id: "prt_human", | ||
| messageID: "msg_human", | ||
| synthetic: false, | ||
| text: "Make the next change.", | ||
| }); | ||
| expect(openCodeTurn([seed(), user], "idle", undefined, view)).toEqual({ | ||
| status: "failed", | ||
| }); | ||
| expect( | ||
| projectAssistantRecord([seed(), user], binding, 1, undefined, view), | ||
| ).toMatchObject({ turnCount: 1, messageCount: 1 }); | ||
| const unexpected: OpenCodeTurnMessage = { | ||
| info: { | ||
| id: "msg_answer", | ||
| sessionID: "ses_child", | ||
| role: "assistant", | ||
| parentID: "msg_seed", | ||
| finish: "stop", | ||
| time: { created: 2, completed: 3 }, | ||
| }, | ||
| parts: [ | ||
| { | ||
| id: "prt_answer", | ||
| sessionID: "ses_child", | ||
| messageID: "msg_answer", | ||
| type: "text", | ||
| text: "Unexpected answer", | ||
| }, | ||
| ], | ||
| }; | ||
| expect(openCodeTurn([seed(), unexpected], "idle", undefined, view)).toEqual({ | ||
| status: "finished", | ||
| }); | ||
| expect( | ||
| projectAssistantRecord([seed(), unexpected], binding, 1, undefined, view), | ||
| ).toMatchObject({ turnCount: 1, messageCount: 2 }); | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Captured seeds remain human tasks
For prepared children,
continuationstays undefined because the production factory omits the resolver. Captures retain the seed and inflate task counts.Learn more
The new archive filtering runs only when
AssistantRecordCapturereceives anAssistantContinuationView. The sole production construction site still passes three arguments, so the optional resolver is always absent outside tests. Every capture therefore callsprojectAssistantRecordwithnull, preserving the old behavior for the exact scenario this change targets.Example: A prepared child contains only its no-reply seed. Its first observer checkpoint projects one turn and one message instead of zero because no continuation attestation reaches the capture.
Recommended fix: Construct the continuation store before the observer factory and pass a resolver that reads the hosted child receipt, validates the prepared phase and exact binding, then projects an
AssistantContinuationView. Keep the lookup under the capture signal and fail the checkpoint if a linked child cannot be verified.Was this helpful? React with 👍 or 👎 to provide feedback.