diff --git a/docs/spec/instrument/hooks.md b/docs/spec/instrument/hooks.md index 9187081f..82f7cd29 100644 --- a/docs/spec/instrument/hooks.md +++ b/docs/spec/instrument/hooks.md @@ -201,7 +201,9 @@ Schema: [`hooks/tool-call-result.json`](https://genai-security-project.github.io Fires after tool execution, before the result is ingested into the agent's reasoning context. Provenance: `origin: tool_output`, with `derived_from` set to the originating `toolCallRequest`'s provenance ids when the tool's output is data-derived. -**Payload:** `execution_id` (correlated to the request), outputs, exit status, duration. +**Payload:** `request_id_ref` (the originating request), outputs, exit status, duration. + +`request_id_ref`, when present, is the `request_id` of the originating `toolCallRequest`. A Guardian SHOULD resolve it against the `toolCallRequest` records it holds for the session, and MUST record the reference in its audit record when it cannot resolve it. Whether an unresolvable reference is grounds to deny is the deployment's policy. The record is required because an unresolvable reference is evidence, so it has to survive even when the result is allowed. **Decision:** ALLOW / DENY / MODIFY. diff --git a/docs/spec/instrument/specification.md b/docs/spec/instrument/specification.md index 104aa4c8..9cd832f2 100644 --- a/docs/spec/instrument/specification.md +++ b/docs/spec/instrument/specification.md @@ -310,7 +310,7 @@ For every step where the Guardian writes a ContextEntry (the content-bearing ste A `subagentStop` payload MAY omit `final_chain_hash`: a framework that maintains no session-chain omits the field rather than fabricate a value, since fabrication corrupts the artifact the field exists to produce. A Guardian MUST treat the omission as "chain not maintained by this framework," not as an integrity failure; integrity claims about that subagent's chain are simply unavailable. -Tamper-evidence here is bounded by the signature in use. Under the HMAC baseline the published, signed head gives integrity against a network tamperer and lets a key-holder verify the chain, but it does not bind the Guardian itself: the Guardian holds the symmetric key and can re-sign a rewritten head. Non-repudiation, proving to a third party that a specific Guardian issued a specific head, requires the asymmetric ACS-Crypto profile. The baseline detects accidental divergence and cross-Guardian disagreement; defeating a determined, compromised Guardian is a profile-level guarantee, not a Core one. +Tamper-evidence here is bounded by the signature in use. Under the HMAC baseline the published, signed head gives integrity against a network tamperer and lets a key-holder verify the chain, but it binds neither party. The Guardian and the Observed Agent derive the same per-session key (§10), so the Guardian can re-sign a rewritten head, and the Observed Agent can produce a validly signed response the Guardian never sent, an `allow` or a `chain_hash` included. A valid HMAC shows that a holder of the session key produced the message, not which holder, so under HMAC a Guardian decision is not attributable to the Guardian against the Observed Agent. Non-repudiation, proving to a third party that a specific Guardian issued a specific head or decision, requires the asymmetric ACS-Crypto profile. The baseline detects accidental divergence and cross-Guardian disagreement; defeating a determined, compromised Guardian is a profile-level guarantee, not a Core one. ## 9. Approver Model @@ -349,6 +349,10 @@ This rule preserves the security guarantee (actions that would have been `ASK`'d A signature over the §10 canonical input is REQUIRED in ACS-Core. The signature envelope is `{ algorithm, value, key_id }`, the algorithm drawn from the registry below and the per-direction supported set declared in the handshake. `HMAC-SHA256` is the baseline that satisfies the requirement; ACS-Core does not require asymmetric or post-quantum signatures, which are the ACS-Crypto profile (§10.1). The per-session HMAC key is `HKDF`-derived from deployment-provided input keying material (a pre-shared secret, or a transport channel binding such as a TLS exporter) together with the `session_id`. ACS v0.1 defines no in-band key-exchange: a deployment with neither a shared secret nor a secure transport cannot use the HMAC baseline and satisfies the requirement with an ACS-Crypto algorithm whose key distribution it manages out of band. +**HMAC key derivation.** For `HMAC-SHA256`, the per-session key is HKDF ([RFC 5869](https://www.rfc-editor.org/rfc/rfc5869)) with SHA-256, no salt (RFC 5869's default, a string of 32 zero bytes, which gives the same key as an empty salt), `info` set to the UTF-8 bytes of `metadata.session_id`, and a 32-byte output, applied to the secret that `key_id` names. Both parties MUST derive the key with exactly these parameters, so independent implementations holding the same secret derive the same key. `info` is the `session_id` string exactly as the request carries it, not parsed or normalized first: an uppercase or `urn:uuid:` spelling of the same UUID derives a different key. A response carries no `session_id`, so it is signed and verified with the key of the request it answers. The secret MUST be at least 32 bytes from a cryptographically secure random source. HKDF does no key stretching, so a passphrase used as the secret can be recovered offline from one captured signature. This derivation is defined for a pre-shared secret. v0.1 defines no exporter label or context for a channel-binding input, so that input is not yet interoperable between implementations. For example, the 32-byte secret `000102…1e1f` (the bytes 0 through 31) and the `session_id` `6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36` give the key `80f0dc8c611279bfc2af2d5f374c5cb685888192ce1af5c3e0673d8fa748e8a4`. + +**`key_id` for HMAC.** For `HMAC-SHA256`, `key_id` is an opaque label, chosen by the deployment, that names one configured secret. This specification reserves no values, and an implementation takes the label from its configuration. It does not hardcode one. A verifier MUST reject an unknown `key_id` (`SIGNATURE_INVALID`, §17.1) and MUST NOT fall back to another secret. `key_id` MUST NOT be derived from the secret. A Guardian that signs its response with `HMAC-SHA256` uses the secret, and the `key_id`, that the request named. The `SIGNATURE_INVALID` error for an unknown `key_id` is unsigned, because no configured secret matches it. A deployment rotating its secret configures the new secret under a new label, such as `"2026-10"`, so a verifier can hold both secrets while the rotation is in progress. That is what the §17.1 recovery for `SIGNATURE_INVALID`, re-resolving `key_id`, relies on. For the asymmetric algorithms of ACS-Crypto, `key_id` identifies the public key or hybrid key descriptor (§10.2). + **Signed input.** The signed input for every algorithm is the RFC 8785 (JCS) canonicalization of the request or response envelope with the `signature` field removed, encoded as UTF-8. This binds the signature to the whole envelope, including `method`, `metadata.session_id`, `request_id`, and `timestamp`, so a captured signature cannot be lifted into a different envelope (for example, re-wrapping a signed `toolCallResult` under a different `session_id`). A verifier MUST recompute this canonical form and MUST reject a signature that does not cover it (`SIGNATURE_INVALID`, §17.1). The input is fixed by this rule, not declared per message: a discoverable `signed_fields` would let a producer advertise partial coverage that omits `session_id`, reopening the gap this closes. Alternative canonicalization is not permitted in v0.1, matching the chain-hash rule in §8. ### 10.1 Algorithm registry @@ -357,7 +361,7 @@ The registry is crypto-agile: the `{ algorithm, value, key_id }` envelope suppor | Algorithm | Class | v0.1.0 status | |---|---|---| -| `HMAC-SHA256` | Symmetric MAC | RECOMMENDED. Shared-secret integrity; simplest deployment path. Sufficient for same-host and trusted-network topologies where the threat is accidental tampering or replay. | +| `HMAC-SHA256` | Symmetric MAC | RECOMMENDED. Shared-secret integrity; simplest deployment path. Sufficient for same-host and trusted-network topologies where the threat is accidental tampering or replay. Both parties hold the key, so it attributes a message to neither (§8.6). | | `ECDSA-P256` | Classical asymmetric | OPTIONAL. Strongest current ecosystem support across Java, Node, .NET, HSMs, and major cloud KMS providers. | | `RSA-PSS-SHA256` | Classical asymmetric | OPTIONAL. Legacy interop; deployments with existing RSA PKI. | | `ML-DSA-65` | PQC, lattice (FIPS 204) | OPTIONAL. ~128-bit post-quantum security; ~3.3 KB signatures. Recommended for deployments shipping PQC libraries today. | diff --git a/reference-implementations/agt/README.md b/reference-implementations/agt/README.md index cdc6a862..c50cd580 100644 --- a/reference-implementations/agt/README.md +++ b/reference-implementations/agt/README.md @@ -312,7 +312,7 @@ Verified on macOS (arm64): natively with otelcol-contrib 0.162.0, and with the D **Run it again.** Every output here except the converter's appends, so a second run on the same `.acs` doubles every count, and the converter refuses to run at all. Stop the Guardian and the collector first, with Ctrl-C or `docker stop acs-otelcol`. A process that is still running keeps writing into the deleted file, and its spans are lost without an error. Then delete it: `rm -rf .acs && mkdir -p .acs && chmod 700 .acs`. -**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability` and `acs.request_id_ref`, because the shim doesn't send them yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). The committed sample drives the Guardian directly, so it has both. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229). +**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229). The trace files, the audit log and the collector's port can be written by any process running as you, the observed agent included, and in a container by other containers on the same Docker network, so the output isn't tamper-evident. `service.name` (`acs-reference-guardian`, `acs-reference-audit-converter`) is a label each writer sets for itself, not authentication, and loopback isn't authentication either. These labels belong to the reference implementation, not to the ACS specification. @@ -420,7 +420,7 @@ The `.acs/` directory holds four kinds of local file. None is ever committed. | File | Written by | Holds | |---|---|---| -| `envelopes.jsonl` | Guardian | Every envelope, both directions, before validation | +| `envelopes.jsonl` | Guardian | Every envelope, both directions, before validation, plus a `finding` line for a fact the spec requires the Guardian to record, such as a `request_id_ref` it cannot resolve | | `session-context.jsonl` | Guardian | One hash-chain entry per governed step | | `audit.jsonl` | Host shim | Every fail-open proceed and every posture-driven block | | `sessions/.json` | Host shim, Claude Code only | The negotiated ServerHello, so the next hook subprocess finds it | @@ -492,7 +492,7 @@ The suite has 68 test files. Most of them drive a real Guardian, the real host s These are known limits of the tree as it stands. None is fixed. -- **Memory and disk grow without bound.** The Guardian's session store never evicts a session. The envelope log, the audit log and the session-context log have no rotation and no size cap. A long-lived Guardian grows until something else stops it. +- **Memory and disk grow without bound.** The Guardian's session store keeps at most 1024 sessions and evicts the least recently written one, but it bounds sessions, not the steps within one. The envelope log, the audit log and the session-context log have no rotation and no size cap. A long-lived Guardian grows until something else stops it. - **Log writes sit on the decision path.** The Guardian writes two synchronous file appends per request in a single-threaded server. A slow disk blocks every in-flight decision. - **The wire is unauthenticated.** See [What this project is, and is not](#what-this-project-is-and-is-not). Loopback by default is the only protection. - **The egress gate has three measured soft spots.** A shell command whose destination the extractor cannot parse falls through to `allow`, not `deny`. With the shipped allowlist, any command whose text contains an off-list URL is denied, whether or not it reaches that URL. A `WebFetch` URL reaches AGT's gate with no parsing, so four userinfo-style URL shapes pass on the fetch route and fail on the shell route. Closing the last one means editing a `.rego` file, which this project does not do. @@ -522,6 +522,11 @@ These findings are open on this tree. They are recorded here so a reader does no - A `steps/skillLoad` hook that drives AGT's `content_hash` gate is planned, not built. No code for it exists in the tree, and the ServerHello does not name it. - Two AGT intervention points, `pre_model_call` and `post_model_call`, have no ACS v0.1.0 method. Whether this project proposes `steps/modelCall` for ACS v0.2 is open. - Whether to report a fail-open to AGT upstream is open: a `./`-prefixed `bundle:` path silently disables all policy and returns `allow`. This tree throws on that path as a backstop. +- A finding is recorded on the step's own entry in the hash chain (`session-context.jsonl`), as `findings` plus the unresolved `request_id_ref`, and the entry's hash covers both. Entries with no findings hash exactly as before. A readable copy goes to the envelope log, or, when there is no envelope log or it has disabled itself, to stderr as `acs-finding ` followed by one JSON object with the same fields plus `recorded_at`, `method` and `rpc_id`. Findings don't reach the OTel export. The chain is tamper-evident only as far as §8.6 allows, since this Guardian publishes no chain head. +- A reference fails to resolve for three ordinary reasons as well as a suspicious one: the request was recorded before a Guardian restart, its session was evicted, or the request never reached this Guardian and the host proceeded under its posture. The finding doesn't say which. Run one Guardian per session, or route by session, or every result that lands on a different Guardian is a finding. +- A reference to a request the Guardian denied counts as resolved, so a result for a denied tool call gets no finding. +- Specification §10.3 says a Guardian MUST reject a duplicate `request_id` in a session. This Guardian records a `request_id_duplicate` finding instead, for a `steps/toolCallRequest` or `steps/toolCallResult` that reuses an id any earlier step in the session carried, and still decides the step. It sees only the two methods it dispatches. +- The shim derives a request's `request_id` from the host's call id, so asking twice about one call repeats the id. If a Guardian answers that with `REPLAY_DETECTED`, the shim retries that request once with a random id. The result hook runs on its own and still names the derived id, so the result resolves against the first request the host sent for that call. The JSON-RPC `id` stays random either way, so the envelope log never pairs two requests under one id, and the host audit log records the `request_id` separately as `request_id`. - Six required Trace-pillar attributes exist on the wire but are optional there. Requiring them is an ask for v0.2. The Guardian also leaves `AcsResult.metadata` empty, which it could fill today. ## Verify diff --git a/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml b/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml index 0f227b7c..e77ea83c 100644 --- a/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml +++ b/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml @@ -19,6 +19,9 @@ hooks: PreToolUse: acs_method: steps/toolCallRequest tool_name: $.tool_name + # Claude Code's id for this one tool call, the same on PreToolUse and + # PostToolUse, so the result can name the request it answers. + call_id: $.tool_use_id arguments: $.tool_input # ACS's own raw_command: the command line this step IS, verbatim, which is # a different thing from the argument bag above and is why it gets its own @@ -128,6 +131,9 @@ hooks: PostToolUse: acs_method: steps/toolCallResult tool_name: $.tool_name + # Claude Code's id for this one tool call, the same on PreToolUse and + # PostToolUse, so the result can name the request it answers. + call_id: $.tool_use_id # THIS GATE'S MATCHER STAYS SINGLE-TOOL WHILE `outputs.from` IS A SINGLE # PATH, and that is the fact to read before widening `^Bash$` in # settings.json to match the request gate's `^(Bash|WebFetch)$`. JSON diff --git a/reference-implementations/agt/hosts/claude-code/test/pairing.test.ts b/reference-implementations/agt/hosts/claude-code/test/pairing.test.ts new file mode 100644 index 00000000..50ed04df --- /dev/null +++ b/reference-implementations/agt/hosts/claude-code/test/pairing.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +// Test-only import, the same precedent as hook.test.ts: a real Guardian, so +// this proves the pairing on the wire rather than against a copied shape. +import { startGuardian } from "guardian"; +import { + createGuardianClient, + createMemorySessionConfigStore, + DEFAULT_TIMEOUT_MS, + governStep, + loadHookmap, + NULL_AUDIT_SINK, + resolveSessionConfig, + toSessionUuid, +} from "host-adapter"; + +const HOOKMAP_PATH = fileURLToPath(new URL("../claude-code.hookmap.yaml", import.meta.url)); + +// The shipped hookmap names `tool_use_id` as the call id. Claude Code's hooks +// reference shows the field on both PreToolUse and PostToolUse, and a +// transcript check found the same value on both hooks of one call. +function findingLines(path: string): unknown[] { + if (!existsSync(path)) { + return []; + } + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as { direction: string; envelope: unknown }) + .filter((entry) => entry.direction === "finding") + .map((entry) => entry.envelope); +} + +async function runCall(sessionId: string, beforeId: string, afterId: string): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-pairing-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath }); + try { + const hookmap = loadHookmap(HOOKMAP_PATH); + const client = createGuardianClient(guardian.url); + const session = await resolveSessionConfig( + { guardian: client, agentId: hookmap.host, sessionId: toSessionUuid(sessionId), timeoutMs: DEFAULT_TIMEOUT_MS }, + createMemorySessionConfigStore(), + ); + const common = { session_id: sessionId, tool_name: "Bash", tool_input: { command: "ls" } }; + for (const [hookEventName, payload] of [ + ["PreToolUse", { ...common, hook_event_name: "PreToolUse", tool_use_id: beforeId }], + ["PostToolUse", { ...common, hook_event_name: "PostToolUse", tool_use_id: afterId, tool_response: { stdout: "a.txt" } }], + ] as const) { + await governStep({ hookEventName, payload, hookmap, guardian: client, session, sessionId, audit: NULL_AUDIT_SINK }); + } + return findingLines(logPath); + } finally { + await guardian.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +describe("Claude Code: a result names the request it answers", () => { + it("records no finding when PostToolUse carries the same tool_use_id as PreToolUse", async () => { + expect(await runCall("sess-raw-not-a-uuid", "toolu_01", "toolu_01")).toHaveLength(0); + }); + + it("records an unresolved reference when the result's call never reached the Guardian", async () => { + const findings = await runCall("sess-raw-not-a-uuid", "toolu_01", "toolu_02"); + expect(findings).toHaveLength(1); + expect(findings[0]).toMatchObject({ finding: "request_id_ref_unresolved" }); + }); +}); diff --git a/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml b/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml index aeba2105..8d556d1e 100644 --- a/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml +++ b/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml @@ -129,6 +129,9 @@ hooks: tool.execute.before: acs_method: steps/toolCallRequest tool_name: $.tool + # OpenCode's id for this one tool call, the same before and after it, + # so the result can name the request it answers. + call_id: $.callID arguments: $.args # The same declaration Claude Code's hookmap makes, at this host's own # field: OpenCode hands the plugin a mutable `{args}`, so the shell tool's @@ -209,6 +212,9 @@ hooks: tool.execute.after: acs_method: steps/toolCallResult tool_name: $.tool + # OpenCode's id for this one tool call, the same before and after it, + # so the result can name the request it answers. + call_id: $.callID # SCOPED TO bash ONLY, and this is data because OpenCode has no matcher to # get it from. OpenCode fires this hook for EVERY tool, and `metadata` is # PER-TOOL -- measured on 1.18.15 across four tools: `bash`'s metadata is diff --git a/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts b/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts index 26ad1873..2ecdf356 100644 --- a/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts +++ b/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts @@ -11,7 +11,7 @@ */ import { describe, expect, it } from "bun:test"; import { fileURLToPath } from "node:url"; -import { buildEnvelope, loadHookmap } from "host-adapter"; +import { buildEnvelope, loadHookmap, toInvocationRequestId } from "host-adapter"; import { validateEnvelope } from "guardian"; const HOOKMAP = fileURLToPath(new URL("../opencode.hookmap.yaml", import.meta.url)); @@ -138,7 +138,7 @@ describe("opencode.hookmap.yaml", () => { expect(hooks["tool.execute.before"]?.tools).toEqual(["bash", "webfetch"]); }); - it("builds a wire payload of exactly {tool, exit_status, outputs:[{value}]}, with no mirrors anywhere in it", () => { + it("builds a wire payload of exactly {tool, request_id_ref, exit_status, outputs:[{value}]}, with no mirrors anywhere in it", () => { // This is the first hookmap that actually declares `outputs.mirrors`, so // this is the first place a mirror leaking onto the wire could be caught. // `buildPayload` never reads `mirrors` -- this asserts the envelope it @@ -167,6 +167,8 @@ describe("opencode.hookmap.yaml", () => { expect(() => validateEnvelope(envelope)).not.toThrow(); expect(envelope.params.payload).toEqual({ tool: { name: "bash" }, + // From the hookmap's call_id: the request this result answers. + request_id_ref: toInvocationRequestId("6c616a11-495a-4f05-878a-c1bfaa29f0e5", "call-1"), exit_status: "success", outputs: [{ value: "TOKEN=ghp_ABCDEF123456" }], }); diff --git a/reference-implementations/agt/packages/guardian/README.md b/reference-implementations/agt/packages/guardian/README.md index 4c604f7f..7ae1cd96 100644 --- a/reference-implementations/agt/packages/guardian/README.md +++ b/reference-implementations/agt/packages/guardian/README.md @@ -38,11 +38,13 @@ To see a decision without an agent client, pipe a hook payload into the Claude C 1. The raw envelope is appended to the envelope log, before anything else. 2. The envelope is validated against `request-envelope.json`. A `steps/toolCallRequest` or `steps/toolCallResult` is also validated against its own payload schema. An invalid envelope on a `steps/*` method is answered with an honoured `deny`, not a bare error, so the host has a decision to act on. -3. The AGT intervention point for the method is resolved from `mapping.yaml`, and so is the argument the policy target is read from for the tool the payload names. -4. The AGT snapshot is assembled, one shape per gate, and evaluated through the bridge. -5. The verdict is mapped to an ACS decision with the tables in `mapping.yaml`. A `transform` becomes a `modify` whose modification differs by gate: a parameter override at the request gate, a redaction on the result payload at the result gate. -6. One entry is appended to the session's hash chain. Information-flow labels AGT emitted are stored, so the next step of the session sends them back as input. -7. The outgoing response is checked against `response-envelope.json`. The result is logged on stderr and never thrown. The response is then recorded and sent. +3. A `steps/toolCallResult` whose `request_id_ref` names no `toolCallRequest` in the session gets the finding `request_id_ref_unresolved` on its own hash-chain entry (step 8), because `hooks.md` requires the Guardian to record a reference it cannot resolve. A readable copy goes to the envelope log as a `finding` line, or to stderr as an `acs-finding` line when there is no envelope log or it has disabled itself. The decision is left to policy. Both shipped hookmaps name the host's per-call id as `call_id` (`tool_use_id` for Claude Code, `callID` for OpenCode). The shim derives the request's `request_id` from it and sends the same value as the result's `request_id_ref`. A host that leaves the id out gets a random `request_id` and no reference. Results are governed only for the tools the hookmaps name, and Claude Code reports a failed call through `PostToolUseFailure`, which isn't mapped, so a failed call sends no result. +4. A step whose `request_id` an earlier step in the session already carried gets a `request_id_duplicate` finding the same way, and is still decided. +5. The AGT intervention point for the method is resolved from `mapping.yaml`, and so is the argument the policy target is read from for the tool the payload names. +6. The AGT snapshot is assembled, one shape per gate, and evaluated through the bridge. +7. The verdict is mapped to an ACS decision with the tables in `mapping.yaml`. A `transform` becomes a `modify` whose modification differs by gate: a parameter override at the request gate, a redaction on the result payload at the result gate. +8. One entry is appended to the session's hash chain. Information-flow labels AGT emitted are stored, so the next step of the session sends them back as input. +9. The outgoing response is checked against `response-envelope.json`. The result is logged on stderr and never thrown. The response is then recorded and sent. `handshake/hello` is answered with a constant ServerHello that names exactly the methods this Guardian dispatches: `steps/toolCallRequest` and `steps/toolCallResult`. The Guardian does not read the ClientHello. Any other method is answered with a JSON-RPC error carrying `METHOD_NOT_DISPATCHED_CODE`, which the conformance harness reads back to confirm the ServerHello tells the truth. diff --git a/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts b/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts index 7dd92b10..9cf327bb 100644 --- a/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts +++ b/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts @@ -290,10 +290,11 @@ export async function convertAuditLog(auditPath: string, otelPath: string, mappi if (sessionId !== undefined) { attributes["acs.session.id"] = sessionUuid(sessionId); } - // The host builds every envelope with its JSON-RPC id equal to - // params.request_id (build-envelope.ts), so the entry's rpc_id is the - // request id the Guardian's span for the same request carries. - const requestId = readString(entry.rpc_id); + // The entry's request_id is the id the Guardian's span for the same + // request carries. Entries written before that field existed have only + // rpc_id, which equalled params.request_id whenever the host did not + // derive the request id from its own call id. + const requestId = readString(entry.request_id) ?? readString(entry.rpc_id); if (requestId !== undefined) { attributes["acs.request_id"] = requestId; } diff --git a/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts b/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts index e83d80f2..ed217338 100644 --- a/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts +++ b/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts @@ -28,8 +28,13 @@ import { appendFileSync, mkdirSync } from "node:fs"; import { dirname } from "node:path"; -/** Which side of the exchange one envelope-log line recorded. */ -export type EnvelopeLogDirection = "request" | "response"; +/** + * Which side of the exchange one envelope-log line recorded, or "finding" for + * a fact the Guardian established about a request that the spec requires it + * to record. A finding's `envelope` is the Guardian's own record, not a + * JSON-RPC object, and its `rpc_id` names the request it concerns. + */ +export type EnvelopeLogDirection = "request" | "response" | "finding"; /** * One line of the envelope log, named for what was recorded rather than for @@ -49,9 +54,14 @@ export type EnvelopeLogEntry = { envelope: unknown; }; -/** Where envelope-log lines go: the Guardian-side writer role. */ +/** + * Where envelope-log lines go: the Guardian-side writer role. `write` says + * whether the line landed. Most callers ignore that, because the log is + * observability. A caller recording a fact the spec requires it to keep + * reads it and puts the fact somewhere else when the log did not take it. + */ export type EnvelopeLogSink = { - write(direction: EnvelopeLogDirection, envelope: unknown, method: string | null): void; + write(direction: EnvelopeLogDirection, envelope: unknown, method: string | null, rpcId?: string | number | null): boolean; readonly path: string | null; }; @@ -68,7 +78,9 @@ export type CreateEnvelopeLogSinkOptions = { * `EnvelopeLogSink` type, not a special case. */ export const NULL_ENVELOPE_LOG_SINK: EnvelopeLogSink = { path: null, - write(): void {}, + write(): boolean { + return false; + }, }; /** The JSON-RPC id, when it is a scalar. Both request and response envelopes @@ -114,9 +126,9 @@ export function createEnvelopeLogSink({ return { path, - write(direction, envelope, method): void { + write(direction, envelope, method, rpcId): boolean { if (disabled) { - return; + return false; } try { const entry: EnvelopeLogEntry = { @@ -124,14 +136,18 @@ export function createEnvelopeLogSink({ recorded_at: now().toISOString(), direction, method, - rpc_id: extractRpcId(envelope), + // A finding carries no JSON-RPC id of its own, so the caller names + // the request it concerns. + rpc_id: rpcId === undefined ? extractRpcId(envelope) : rpcId, envelope, }; const line = `${JSON.stringify(entry)}\n`; appendFileSync(path, line); seq += 1; + return true; } catch (error) { fail(error); + return false; } }, }; diff --git a/reference-implementations/agt/packages/guardian/src/server.ts b/reference-implementations/agt/packages/guardian/src/server.ts index 922c11bd..ba5e90c9 100644 --- a/reference-implementations/agt/packages/guardian/src/server.ts +++ b/reference-implementations/agt/packages/guardian/src/server.ts @@ -89,6 +89,8 @@ import { isToolCallResult, validateEnvelope, type AcsRequestEnvelope, + type ToolCallRequestEnvelope, + type ToolCallResultEnvelope, } from "./validate-envelope.ts"; import { checkResponse } from "./check-response.ts"; import { buildServerHello, type ServerHello } from "./handshake.ts"; @@ -100,6 +102,7 @@ import { type IfcLabels, type SessionContextStore, } from "./session-context-store.ts"; +import type { StepFindings } from "./session-context.ts"; import { persistIfcLabels, supplySourceLabels } from "./ifc-labels.ts"; /** @@ -574,7 +577,7 @@ async function handleAcsRequest( let response: JsonRpcSuccess | JsonRpcFailure; try { - response = await dispatch(raw, bridge, mapping, onDecisionFailure, sessionContextStore); + response = await dispatch(raw, bridge, mapping, onDecisionFailure, sessionContextStore, envelopeLog); } catch (error) { // The outer net. Deliberately a bare JSON-RPC error, not an ACS `deny`: // this route is `dispatch` rethrowing entirely past denyOnInvalidEnvelope @@ -712,12 +715,115 @@ async function readCappedBody(req: Request): Promise { : { withinLimit: true, text: text + decoder.decode() }; } +/** + * What this Guardian establishes about a step as it arrives, before the step + * is appended to the chain. + * + * `request_id_ref_unresolved`: hooks.md, toolCallResult. A Guardian SHOULD + * resolve `request_id_ref` against the session's `toolCallRequest` records and + * MUST record one it cannot resolve. The decision is left to policy, so this + * only records. Resolution reads the session's own requests, because + * `request_id` is unique only within a session. A restart or an eviction + * makes an older reference unresolvable, and that is recorded too. + * + * `request_id_duplicate`: Specification §10.3 says a Guardian MUST reject a + * `request_id` the session has already carried, on any method. This Guardian + * does not enforce that yet, so it records the repeat instead. + * + * The findings go onto the step's own chain entry, which is this Guardian's + * audit record, and a copy goes to the envelope log (or stderr) for reading. + */ +function examineStep( + envelope: ToolCallRequestEnvelope | ToolCallResultEnvelope, + sessionContextStore: SessionContextStore, + envelopeLog: EnvelopeLogSink, +): StepFindings | undefined { + const sessionId = envelope.params.metadata.session_id; + const requestId = envelope.params.request_id; + const findings: StepFindings["findings"] = []; + let unresolvedRef: string | undefined; + + if (sessionContextStore.hasRequestId(sessionId, requestId)) { + findings.push("request_id_duplicate"); + recordFinding(envelopeLog, { finding: "request_id_duplicate", session_id: sessionId, request_id: requestId }, envelope.method, envelope.id); + } + + const ref = isToolCallResult(envelope) ? envelope.params.payload.request_id_ref : undefined; + if (ref !== undefined && !sessionContextStore.hasToolCallRequest(sessionId, ref)) { + findings.push("request_id_ref_unresolved"); + unresolvedRef = ref; + recordFinding( + envelopeLog, + { finding: "request_id_ref_unresolved", session_id: sessionId, request_id: requestId, request_id_ref: ref }, + envelope.method, + envelope.id, + ); + } + + if (findings.length === 0) { + return undefined; + } + return unresolvedRef === undefined ? { findings } : { findings, request_id_ref: unresolvedRef }; +} + +/** + * examineStep, unable to throw. A caller-supplied store that throws on the + * lookup costs the findings, never the decision. + */ +function examineStepSafely( + envelope: ToolCallRequestEnvelope | ToolCallResultEnvelope, + sessionContextStore: SessionContextStore, + envelopeLog: EnvelopeLogSink, +): StepFindings | undefined { + try { + return examineStep(envelope, sessionContextStore, envelopeLog); + } catch (error) { + reportCheckFailure("step findings", error); + return undefined; + } +} + +/** + * Writes a finding to the envelope log, or, when no envelope log was + * configured or it disabled itself after a write error, to stderr as + * `acs-finding ` followed by one JSON object carrying the same fields the log + * line would. Never throws: a finding is a record, and losing the record must + * never cost the decision it sits beside. + */ +function recordFinding( + envelopeLog: EnvelopeLogSink, + finding: Record, + method: string, + rpcId: string | number | null, +): void { + if (envelopeLog.write("finding", finding, method, rpcId)) { + return; + } + try { + console.error( + `acs-finding ${JSON.stringify({ recorded_at: new Date().toISOString(), method, rpc_id: rpcId, ...finding })}`, + ); + } catch { + // stderr itself failed (an EPIPE, say). Nothing is left to report to. + } +} + +/** One stderr line for a check that threw, itself unable to throw. */ +function reportCheckFailure(check: string, error: unknown): void { + try { + console.error(`${check} check failed: ${error instanceof Error ? error.message : String(error)}`); + } catch { + // stderr itself failed. The decision still goes ahead. + } +} + async function dispatch( raw: unknown, bridge: PolicyBridge, mapping: Mapping, onDecisionFailure: "proceed" | "deny" | undefined, sessionContextStore: SessionContextStore, + envelopeLog: EnvelopeLogSink, ): Promise { const rpcId = extractId(raw); @@ -773,12 +879,16 @@ async function dispatch( // shape. Anything neither predicate answers falls past both, to // METHOD_NOT_DISPATCHED_CODE below. if (isToolCallRequest(envelope)) { - return await evaluateStep(raw, envelope, assemblePreToolCallSnapshot, bridge, mapping, sessionContextStore); + // Examined before evaluateStep appends this step, so a request id is in + // the session already only if an earlier step carried it. + const findings = examineStepSafely(envelope, sessionContextStore, envelopeLog); + return await evaluateStep(raw, envelope, assemblePreToolCallSnapshot, bridge, mapping, sessionContextStore, findings); } // The result gate, beside the request gate rather than merged into it. if (isToolCallResult(envelope)) { - return await evaluateStep(raw, envelope, assemblePostToolCallSnapshot, bridge, mapping, sessionContextStore); + const findings = examineStepSafely(envelope, sessionContextStore, envelopeLog); + return await evaluateStep(raw, envelope, assemblePostToolCallSnapshot, bridge, mapping, sessionContextStore, findings); } // A well-formed envelope (it passed validateEnvelope: the method matched @@ -857,6 +967,7 @@ async function evaluateStep( bridge: PolicyBridge, mapping: Mapping, sessionContextStore: SessionContextStore, + findings?: StepFindings, ): Promise { try { // The chain entry is appended on arrival, before any verdict exists: a @@ -867,6 +978,7 @@ async function evaluateStep( method: envelope.method, request_id: envelope.params.request_id, tool_name: envelope.params.payload.tool.name, + ...findings, }); // Resolved BEFORE the snapshot is assembled, where it used to be resolved diff --git a/reference-implementations/agt/packages/guardian/src/session-context-store.ts b/reference-implementations/agt/packages/guardian/src/session-context-store.ts index 11e0ca9a..e4bb2937 100644 --- a/reference-implementations/agt/packages/guardian/src/session-context-store.ts +++ b/reference-implementations/agt/packages/guardian/src/session-context-store.ts @@ -99,6 +99,8 @@ export { /** What the Guardian's request path is told and asks on every step. */ export interface SessionContextStore { context(sessionId: string): SessionContext; + hasToolCallRequest(sessionId: string, requestId: string): boolean; + hasRequestId(sessionId: string, requestId: string): boolean; append(sessionId: string, step: SessionStep): SessionContextEntry; replaceIfcLabels(sessionId: string, labels: IfcLabels): void; sourceLabels(sessionId: string): IfcLabels; @@ -226,6 +228,16 @@ export function createMemorySessionContextStore( return { session_id: sessionId, entries: [...read(sessionId).entries] }; }, + // Straight off the map rather than through `read`, which builds an empty + // state for a session it does not hold. Neither question needs one. + hasToolCallRequest(sessionId, requestId) { + return sessions.get(sessionId)?.toolCallRequestIds.has(requestId) ?? false; + }, + + hasRequestId(sessionId, requestId) { + return sessions.get(sessionId)?.requestIds.has(requestId) ?? false; + }, + append(sessionId, step) { const state = ensure(sessionId); const previous = state.entries.at(-1); @@ -237,6 +249,9 @@ export function createMemorySessionContextStore( method: step.method, request_id: step.request_id, tool_name: step.tool_name, + ...(step.findings === undefined + ? {} + : { findings: step.findings, ...(step.request_id_ref === undefined ? {} : { request_id_ref: step.request_id_ref }) }), }; const entry: SessionContextEntry = { ...withoutHash, hash: hashEntry(withoutHash) }; // Pushed onto the array this session already owns, rather than rebuilt @@ -244,6 +259,10 @@ export function createMemorySessionContextStore( // N per step, N^2 per session -- and the module doc above says what a // Guardian that slow does to the decision that never arrives. state.entries.push(entry); + state.requestIds.add(step.request_id); + if (step.method === "steps/toolCallRequest") { + state.toolCallRequestIds.add(step.request_id); + } appendLine?.(JSON.stringify(entry)); return entry; }, diff --git a/reference-implementations/agt/packages/guardian/src/session-context.ts b/reference-implementations/agt/packages/guardian/src/session-context.ts index fe114b59..d3177858 100644 --- a/reference-implementations/agt/packages/guardian/src/session-context.ts +++ b/reference-implementations/agt/packages/guardian/src/session-context.ts @@ -89,7 +89,19 @@ export type SessionProvenance = { export type Intent = { readonly text: string; readonly recorded_at: string }; /** What a step contributes to the chain. Deliberately not the whole envelope. */ -export type SessionStep = { method: string; request_id: string; tool_name: string }; +/** + * What the Guardian established about a step when it arrived, carried on the + * step's own chain entry so the published chain commits to it. `hooks.md` + * requires a Guardian to record a `request_id_ref` it cannot resolve, and the + * chain is this Guardian's audit record. `request_id_ref` is the unresolved + * reference, present only with `request_id_ref_unresolved`. + */ +export type StepFindings = { + findings: Array<"request_id_ref_unresolved" | "request_id_duplicate">; + request_id_ref?: string; +}; + +export type SessionStep = { method: string; request_id: string; tool_name: string } & Partial; export type SessionContextEntry = { session_id: string; @@ -100,7 +112,7 @@ export type SessionContextEntry = { method: string; request_id: string; tool_name: string; -}; +} & Partial; /** One session's hash chain, and nothing else the session happens to own. */ export type SessionContext = { @@ -122,6 +134,15 @@ export type SessionContext = { */ export type SessionState = { entries: SessionContextEntry[]; + /** + * The `request_id` of every `toolCallRequest` in `entries`, kept beside the + * chain so resolving a result's `request_id_ref` is a lookup, not a scan of + * the whole history on every result. + */ + toolCallRequestIds: Set; + /** Every `request_id` the session has carried, on any method, for the + * Specification §10.3 duplicate check. */ + requestIds: Set; intent: Intent | undefined; provenance: SessionProvenance; }; @@ -136,6 +157,10 @@ export const GENESIS_HASH = "0".repeat(64); * a field was declared somewhere else. */ export function hashEntry(input: Omit): string { + // Findings are appended only when present, so an entry without any hashes + // exactly as it did before findings existed and every older chain still + // verifies. An entry with findings hashes a longer array, which no entry + // without them can produce. const canonical = JSON.stringify([ input.prev_hash, input.session_id, @@ -144,6 +169,7 @@ export function hashEntry(input: Omit): string { input.method, input.request_id, input.tool_name, + ...(input.findings === undefined ? [] : [input.findings, input.request_id_ref ?? null]), ]); return createHash("sha256").update(canonical).digest("hex"); } @@ -152,6 +178,8 @@ export function hashEntry(input: Omit): string { export function emptySessionState(sessionId: string): SessionState { return { entries: [], + toolCallRequestIds: new Set(), + requestIds: new Set(), intent: undefined, // `origin` names where data entered the system, and this record is one the // Guardian synthesizes to hold a session's labels -- so "system", with the diff --git a/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts b/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts index ba642a00..1ab47860 100644 --- a/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts +++ b/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "bun:test"; -import { mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { mkdirSync, mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { @@ -184,3 +184,33 @@ describe("extractRpcId", () => { expect(extractRpcId("not an object")).toBeNull(); }); }); + +describe("createEnvelopeLogSink -- write says whether the line landed", () => { + it("answers true for a written line, and false once a write fails mid-run", () => { + const dir = mkdtempSync(join(tmpdir(), "acs-envelope-log-")); + const path = join(dir, "envelopes.jsonl"); + const sink = createEnvelopeLogSink({ path, onError: () => {} }); + try { + expect(sink.write("request", { id: 1 }, "steps/toolCallRequest")).toBe(true); + // A directory where the log file was: the next append throws EISDIR, + // which is a failure after earlier writes succeeded. + unlinkSync(path); + mkdirSync(path); + expect(sink.write("request", { id: 2 }, "steps/toolCallRequest")).toBe(false); + expect(sink.write("request", { id: 3 }, "steps/toolCallRequest")).toBe(false); + } finally { + // Non-recursive, like the rest of this file: the log path is a + // directory if the test got that far and a file if it did not. + try { + rmdirSync(path); + } catch { + unlinkSync(path); + } + rmdirSync(dir); + } + }); + + it("answers false from the null sink, which writes nowhere", () => { + expect(NULL_ENVELOPE_LOG_SINK.write("finding", {}, "steps/toolCallResult")).toBe(false); + }); +}); diff --git a/reference-implementations/agt/packages/guardian/test/request-id-ref.test.ts b/reference-implementations/agt/packages/guardian/test/request-id-ref.test.ts new file mode 100644 index 00000000..cd1698ee --- /dev/null +++ b/reference-implementations/agt/packages/guardian/test/request-id-ref.test.ts @@ -0,0 +1,233 @@ +import { describe, expect, it, spyOn } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startGuardian } from "../src/index.ts"; +import type { EnvelopeLogEntry } from "../src/envelope-log-sink.ts"; + +// hooks.md, toolCallResult: a Guardian SHOULD resolve request_id_ref against +// the session's toolCallRequest records and MUST record a reference it cannot +// resolve. These tests drive a running Guardian and read its envelope log, +// which is the Guardian's audit record. + +let rpcId = 0; + +function envelope(method: string, sessionId: string, payload: Record, requestId: string = crypto.randomUUID()) { + rpcId += 1; + return { + jsonrpc: "2.0", + method, + id: rpcId, + params: { + acs_version: "0.1.0", + request_id: requestId, + timestamp: new Date().toISOString(), + metadata: { agent_id: "agent-1", session_id: sessionId }, + payload, + }, + }; +} + +function toolRequest(sessionId: string, requestId: string) { + return envelope( + "steps/toolCallRequest", + sessionId, + { tool: { name: "Bash" }, arguments: { command: { value: "ls" } } }, + requestId, + ); +} + +function toolResult(sessionId: string, requestIdRef?: string) { + const payload: Record = { tool: { name: "Bash" }, exit_status: "success", outputs: [{ value: "a.txt" }] }; + if (requestIdRef !== undefined) { + payload.request_id_ref = requestIdRef; + } + return envelope("steps/toolCallResult", sessionId, payload); +} + +async function post(url: string, body: unknown): Promise> { + const res = await fetch(url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) }); + return (await res.json()) as Record; +} + +function findings(path: string): EnvelopeLogEntry[] { + if (!existsSync(path)) { + return []; + } + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as EnvelopeLogEntry) + .filter((entry) => entry.direction === "finding"); +} + +async function withGuardian(run: (url: string, logPath: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath }); + try { + await run(guardian.url, logPath); + } finally { + await guardian.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +describe("request_id_ref resolution", () => { + it("records a reference the session never saw, and leaves the decision to policy", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const unknownRef = crypto.randomUUID(); + const result = toolResult(sessionId, unknownRef); + + const response = await post(url, result); + + expect(response.result).toBeDefined(); + const recorded = findings(logPath); + expect(recorded).toHaveLength(1); + const [finding] = recorded; + expect(finding?.method).toBe("steps/toolCallResult"); + expect(finding?.rpc_id).toBe(result.id); + expect(finding?.envelope).toEqual({ + finding: "request_id_ref_unresolved", + session_id: sessionId, + request_id: result.params.request_id, + request_id_ref: unknownRef, + }); + }); + }); + + it("records nothing when the reference names this session's toolCallRequest", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + await post(url, toolRequest(sessionId, requestId)); + + await post(url, toolResult(sessionId, requestId)); + + expect(findings(logPath)).toHaveLength(0); + }); + }); + + it("records nothing when the result carries no reference", async () => { + await withGuardian(async (url, logPath) => { + await post(url, toolResult(crypto.randomUUID())); + + expect(findings(logPath)).toHaveLength(0); + }); + }); + + it("treats a request from another session as unresolved, because request_id is unique only within a session", async () => { + await withGuardian(async (url, logPath) => { + const requestId = crypto.randomUUID(); + await post(url, toolRequest(crypto.randomUUID(), requestId)); + + await post(url, toolResult(crypto.randomUUID(), requestId)); + + expect(findings(logPath)).toHaveLength(1); + }); + }); + + // The record is a MUST, and the envelope log is allowed to be absent or to + // disable itself, so a finding the log did not take goes to stderr. + async function stderrFindings(envelopeLogPath: string | undefined): Promise { + const lines: string[] = []; + const spy = spyOn(console, "error").mockImplementation((...args: unknown[]) => { + lines.push(args.map(String).join(" ")); + }); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath }); + try { + await post(guardian.url, toolResult(crypto.randomUUID(), crypto.randomUUID())); + } finally { + await guardian.close(); + spy.mockRestore(); + } + return lines.filter((line) => line.startsWith("acs-finding ")).map((line) => JSON.parse(line.slice("acs-finding ".length))); + } + + it("writes the finding to stderr when no envelope log is configured", async () => { + const recorded = await stderrFindings(undefined); + expect(recorded).toHaveLength(1); + expect(recorded[0]).toMatchObject({ finding: "request_id_ref_unresolved" }); + }); + + it("writes the finding to stderr when the envelope log has disabled itself", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + // A file where the log's parent directory should be, so the sink cannot + // create the directory and disables itself before the first write. + const blocker = join(dir, "not-a-dir"); + writeFileSync(blocker, ""); + try { + const recorded = await stderrFindings(join(blocker, "envelopes.jsonl")); + expect(recorded).toHaveLength(1); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("does not also write to stderr when the envelope log took the finding", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + try { + expect(await stderrFindings(join(dir, "envelopes.jsonl"))).toHaveLength(0); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("records a toolCallRequest whose request_id the session already holds, and still decides it", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + await post(url, toolRequest(sessionId, requestId)); + + const repeat = await post(url, toolRequest(sessionId, requestId)); + + expect(repeat.result).toBeDefined(); + const recorded = findings(logPath); + expect(recorded).toHaveLength(1); + expect(recorded[0]?.envelope).toEqual({ finding: "request_id_duplicate", session_id: sessionId, request_id: requestId }); + }); + }); + + it("puts the log line's fields on a stderr finding too", async () => { + const [recorded] = await stderrFindings(undefined); + expect(recorded).toMatchObject({ method: "steps/toolCallResult", finding: "request_id_ref_unresolved" }); + expect(recorded).toHaveProperty("recorded_at"); + expect(recorded).toHaveProperty("rpc_id"); + }); + + it("records a toolCallResult that reuses an earlier request's request_id, since §10.3 covers every method", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + await post(url, toolRequest(sessionId, requestId)); + + const reused = toolResult(sessionId, requestId); + reused.params.request_id = requestId; + await post(url, reused); + + const codes = findings(logPath).map((entry) => (entry.envelope as { finding: string }).finding); + expect(codes).toEqual(["request_id_duplicate"]); + }); + }); + + it("puts the finding on the result's own chain entry, which is the Guardian's audit record", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + const chainPath = join(dir, "session-context.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", sessionContextLog: chainPath }); + try { + const unknownRef = crypto.randomUUID(); + await post(guardian.url, toolResult(crypto.randomUUID(), unknownRef)); + } finally { + await guardian.close(); + } + try { + const [entry] = readFileSync(chainPath, "utf8").trim().split("\n").map((line) => JSON.parse(line)); + expect(entry.findings).toEqual(["request_id_ref_unresolved"]); + expect(entry.request_id_ref).toMatch(/^[0-9a-f-]{36}$/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + diff --git a/reference-implementations/agt/packages/guardian/test/session-context.test.ts b/reference-implementations/agt/packages/guardian/test/session-context.test.ts index 71ecb2fb..ac421a95 100644 --- a/reference-implementations/agt/packages/guardian/test/session-context.test.ts +++ b/reference-implementations/agt/packages/guardian/test/session-context.test.ts @@ -5,6 +5,7 @@ import { createMemorySessionContextStore, loadSessionContext, } from "../src/session-context-store.ts"; +import { hashEntry } from "../src/session-context.ts"; const at = (iso: string) => () => new Date(iso); const step = (n: number) => ({ method: "steps/toolCallRequest", request_id: `req-${n}`, tool_name: "Bash" }); @@ -285,3 +286,70 @@ describe("append pushes onto the chain rather than rebuilding it", () => { expect(large / small).toBeLessThan(3); }); }); + +describe("SessionContextStore.hasToolCallRequest", () => { + it("knows a session's toolCallRequest ids and no other step's", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", step(1)); + appendContextEntry(store, "sess-a", { method: "steps/toolCallResult", request_id: "res-1", tool_name: "Bash" }); + expect(store.hasToolCallRequest("sess-a", "req-1")).toBe(true); + expect(store.hasToolCallRequest("sess-a", "res-1")).toBe(false); + }); + + it("answers per session, because request_id is unique only within one", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", step(1)); + expect(store.hasToolCallRequest("sess-b", "req-1")).toBe(false); + }); + + it("forgets an evicted session's requests along with its chain", () => { + const store = createMemorySessionContextStore({ maxSessions: 1, onEvict: () => {} }); + appendContextEntry(store, "sess-a", step(1)); + appendContextEntry(store, "sess-b", step(2)); + expect(store.hasToolCallRequest("sess-a", "req-1")).toBe(false); + }); + + it("keeps the ids across the store's other writes to the session", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", step(1)); + store.replaceIfcLabels("sess-a", ["confidential"]); + store.setIntent("sess-a", "list files"); + expect(store.hasToolCallRequest("sess-a", "req-1")).toBe(true); + }); +}); + +describe("SessionContext: findings on a chain entry", () => { + it("hashes an entry without findings exactly as before, so older chains still verify", () => { + const store = createMemorySessionContextStore({ now: at("2026-08-14T00:00:00.000Z") }); + const plain = appendContextEntry(store, "sess-a", step(1)); + const { hash, ...withoutHash } = plain; + expect(hashEntry(withoutHash)).toBe(hash); + expect("findings" in plain).toBe(false); + }); + + it("carries findings and the unresolved reference, and the hash covers both", () => { + const store = createMemorySessionContextStore({ now: at("2026-08-14T00:00:00.000Z") }); + const entry = appendContextEntry(store, "sess-a", { + method: "steps/toolCallResult", + request_id: "res-1", + tool_name: "Bash", + findings: ["request_id_ref_unresolved"], + request_id_ref: "req-9", + }); + expect(entry.findings).toEqual(["request_id_ref_unresolved"]); + expect(entry.request_id_ref).toBe("req-9"); + const { hash, ...withoutHash } = entry; + expect(hashEntry(withoutHash)).toBe(hash); + // Stripping the finding changes the digest, so a rewritten entry shows. + const { findings: _f, request_id_ref: _r, ...stripped } = withoutHash; + expect(hashEntry(stripped)).not.toBe(hash); + }); + + it("remembers every request id, on any method, for the duplicate check", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", { method: "steps/toolCallResult", request_id: "res-1", tool_name: "Bash" }); + expect(store.hasRequestId("sess-a", "res-1")).toBe(true); + expect(store.hasToolCallRequest("sess-a", "res-1")).toBe(false); + }); +}); + diff --git a/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts b/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts index f12cf67d..4522f8f8 100644 --- a/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts +++ b/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts @@ -68,6 +68,13 @@ type AuditEntryIdentity = { */ method: string | null; rpc_id: string | number | null; + /** + * The envelope's `params.request_id`, or null when no request was built. + * It differs from `rpc_id` when the shim derived the request id from the + * host's call id, and it is what the Guardian's own span for the request + * carries. Optional so entries written before it existed still read. + */ + request_id?: string | null; }; /** diff --git a/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts b/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts index bb680234..faa49e00 100644 --- a/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts +++ b/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts @@ -11,7 +11,7 @@ */ import { createHash, randomUUID } from "node:crypto"; import { readFileSync } from "node:fs"; -import { resolvePath } from "./hookmap-path.ts"; +import { pathSegments, resolvePath } from "./hookmap-path.ts"; /** * A hookmap value the host sent no field for: the constant this hook always @@ -69,6 +69,17 @@ type HookmapHookEntryCommon = { acs_method: string; /** JSONPath-lite (`$.foo.bar`) into the raw hook payload for the tool/step name. */ tool_name: string; + /** + * JSONPath-lite into the raw hook payload for the host's own id of this one + * tool call, the same on the hook before the call and the hook after it. + * Each host's hookmap names its own field. Optional. When it + * resolves, the request's `request_id` is derived from it and the result + * carries the same value as `request_id_ref`, so the Guardian can pair the + * two without the shim keeping any state. When it does not resolve, the + * request gets a random `request_id` and the result no `request_id_ref`, + * which is what every hookmap without this key gets. + */ + call_id?: string; /** * This hook's own decision -> host-output mapping, consumed by * `renderDecision` and never by this module. @@ -204,6 +215,8 @@ export type AcsToolCallRequestPayload = { /** ACS's optional `raw_command`. Present only when the hookmap declared a * path for it AND that path resolved to a string. */ raw_command?: string; + /** Only a result names the request it answers. */ + request_id_ref?: never; exit_status?: never; outputs?: never; }; @@ -221,6 +234,9 @@ export type AcsToolCallResultPayload = { * exclusive union stays exclusive and `raw_command` reads as `string | * undefined` on either payload shape without a cast. */ raw_command?: never; + /** The originating request's `request_id`, present when the hookmap's + * `call_id` resolved (see HookmapHookEntryCommon). */ + request_id_ref?: string; exit_status: string; outputs: AcsOutput[]; }; @@ -433,6 +449,80 @@ function assertToolsWellFormed(hookmap: Hookmap, path: string): void { } } +/** + * Rejects a malformed `call_id` at load time, for the reason the checks + * around it give: a throw reached only from `buildEnvelope` is answered with + * the delivery posture, so under `proceed` a typo here would run every + * governed step ungoverned. `call_id` is also refused on any hook that is not + * one of the two tool gates, because the request id it derives is shared by + * exactly one request and the result that answers it. + */ +function assertPairingPathWellFormed(hookmap: Hookmap, path: string): void { + // Per tool gate: which hooks declare `call_id` and which do not. + const declared = { "steps/toolCallRequest": [] as string[], "steps/toolCallResult": [] as string[] }; + const undeclared = { "steps/toolCallRequest": [] as string[], "steps/toolCallResult": [] as string[] }; + for (const [hookEventName, entry] of Object.entries(hookmap.hooks ?? {})) { + if (!isPlainObject(entry)) { + continue; + } + const method = (entry as { acs_method?: unknown }).acs_method; + const pairingPath = (entry as { call_id?: unknown }).call_id ?? undefined; + if (pairingPath === undefined) { + if (method === "steps/toolCallRequest" || method === "steps/toolCallResult") { + undeclared[method].push(hookEventName); + } + continue; + } + if (typeof pairingPath !== "string" || !pairingPath.startsWith("$.") || pairingPath.length <= 2) { + throw new Error( + `loadHookmap: ${path}'s "hooks.${hookEventName}.call_id" is ${JSON.stringify(pairingPath)} -- when present, ` + + `"call_id" must be one JSONPath-lite string such as "$.tool_use_id"`, + ); + } + // The same segment rules resolvePath applies on every call, applied here + // once, so a reserved segment such as `$.constructor` fails at startup and + // not on every governed step. + try { + pathSegments(pairingPath); + } catch (failure) { + throw new Error( + `loadHookmap: ${path}'s "hooks.${hookEventName}.call_id" is not a usable path: ` + + `${failure instanceof Error ? failure.message : String(failure)}`, + ); + } + if (method !== "steps/toolCallRequest" && method !== "steps/toolCallResult") { + throw new Error( + `loadHookmap: ${path}'s "hooks.${hookEventName}" declares "call_id" on ${JSON.stringify(method)} -- ` + + `"call_id" pairs a tool call's request with its result, so only those two gates may declare it`, + ); + } + declared[method].push(hookEventName); + } + // Declared on one gate and not the other is a pairing that cannot work. A + // result-side gap is the dangerous half: results would stop carrying the + // reference and the Guardian would see nothing to check, with no signal. + const requestSide = declared["steps/toolCallRequest"].length > 0; + const resultSide = declared["steps/toolCallResult"].length > 0; + const missing = requestSide || resultSide + ? [...undeclared["steps/toolCallRequest"], ...undeclared["steps/toolCallResult"]] + : []; + if (missing.length > 0) { + throw new Error( + `loadHookmap: ${path} declares "call_id" on some tool gates but not on ${JSON.stringify(missing)} -- ` + + `a request and its result are paired only when both gates name the host's call id`, + ); + } + // One side declaring it and the other side absent entirely: results would + // name requests that were never sent, or requests would be paired with + // results that never come, and every result would be a finding. + if (requestSide !== resultSide) { + throw new Error( + `loadHookmap: ${path} declares "call_id" only on ${requestSide ? "request" : "result"} gates -- ` + + `a request and its result are paired only when both a request gate and a result gate name the host's call id`, + ); + } +} + /** * Rejects a hookmap entry whose `exit_status` declares both `literal` and * `from`, at load time -- the same seam `assertMirrorsWellFormed` and @@ -577,6 +667,7 @@ export function loadHookmap(path: string): Hookmap { assertToolsWellFormed(hookmap, path); assertExitStatusNotBothForms(hookmap, path); assertRequestGateDeclaresNoOutputs(hookmap, path); + assertPairingPathWellFormed(hookmap, path); return normalizeTools(hookmap); } @@ -911,13 +1002,72 @@ export function toSessionUuid(rawSessionId: string): string { .update(Buffer.concat([namespaceBytes, Buffer.from(rawSessionId, "utf8")])) .digest(); - hash[6] = (hash[6]! & 0x0f) | 0x50; // version 5 - hash[8] = (hash[8]! & 0x3f) | 0x80; // RFC 4122 variant + return formatNameBasedUuid(hash, 5); +} - const hex = hash.subarray(0, 16).toString("hex"); +/** + * The first 16 bytes of a name-based hash as a UUID string, with the version + * nibble and the RFC 9562 variant bits set. Shared by both derivations so the + * bit layout lives in one place. The digest is the caller's: SHA-1 for the + * session id's UUIDv5, as RFC 9562 defines v5, and SHA-256 for the request + * id's UUIDv8. + */ +function formatNameBasedUuid(hash: Buffer, version: 5 | 8): string { + const bytes = Buffer.from(hash.subarray(0, 16)); + bytes[6] = (bytes[6]! & 0x0f) | (version << 4); + bytes[8] = (bytes[8]! & 0x3f) | 0x80; + const hex = bytes.toString("hex"); return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20, 32)}`; } +// A second fixed namespace, separate from SESSION_ID_NAMESPACE, so the two +// derivations draw from unrelated spaces. Never regenerate it: doing +// so changes every derived request_id, and a result built by a newer shim +// would no longer name the request an older shim sent. +const INVOCATION_ID_NAMESPACE = "5d0f6a3e-2b71-4c8e-9a54-0e1d7c3b9f62"; + +/** + * The `request_id` for one tool call, derived from the host's call id so the + * hook before the call and the hook after it compute the same value. The + * session is part of the name because `request_id` is unique only within a + * session (Specification §10.3) and a host's call ids need not be unique + * across sessions. Not a secret: anyone who knows both ids can compute it, + * which is fine for an identifier the envelope carries in the clear. + * + * A name-based UUIDv8 over SHA-256 (RFC 9562 §5.8 and Appendix B.2) rather + * than a v5: the same construction with a current hash, so no SHA-1 enters + * new code. + */ +export function toInvocationRequestId(sessionUuid: string, invocationId: string): string { + const namespaceBytes = Buffer.from(INVOCATION_ID_NAMESPACE.replace(/-/g, ""), "hex"); + const hash = createHash("sha256") + .update(Buffer.concat([namespaceBytes, Buffer.from(`${sessionUuid}\n${invocationId}`, "utf8")])) + .digest(); + + return formatNameBasedUuid(hash, 8); +} + +/** + * The host's call id for this invocation, or undefined. Never a throw: a call + * id is optional, and a step whose host did not send one is still fully + * governable, the same reasoning `raw_command` follows in buildPayload. A + * declared path that is not a string is a hookmap fault and does throw. + */ +function invocationIdOf(event: string, entry: HookmapHookEntry, payload: Record): string | undefined { + const path = entry.call_id ?? undefined; + if (path === undefined) { + return undefined; + } + if (typeof path !== "string") { + throw new Error( + `buildEnvelope: hookmap entry for hook "${event}" declares "call_id" as ${JSON.stringify(path)} -- ` + + `"call_id" names the host's per-call id with a single JSONPath-lite string`, + ); + } + const invocationId = resolvePath(payload, path); + return typeof invocationId === "string" && invocationId.length > 0 ? invocationId : undefined; +} + /** * Builds an ACS v0.1.0 request envelope from a raw hook invocation. * `hookmap` drives every host-shape decision: the ACS method, which of the two @@ -926,7 +1076,12 @@ export function toSessionUuid(rawSessionId: string): string { * `hookmap.hooks` throws, and so does an entry naming no single payload shape -- * there is no default method, no default shape, and no partial envelope. */ -export function buildEnvelope(event: string, payload: Record, hookmap: Hookmap): AcsRequestEnvelope { +export function buildEnvelope( + event: string, + payload: Record, + hookmap: Hookmap, + options: { pairWithHostCall?: boolean } = {}, +): AcsRequestEnvelope { const entry = hookmap.hooks[event]; if (!entry) { throw new Error(`buildEnvelope: hookmap has no entry for hook "${event}"`); @@ -946,19 +1101,40 @@ export function buildEnvelope(event: string, payload: Record, h throw new Error(`buildEnvelope: hook payload for "${event}" is missing a "session_id" string field`); } - const requestId = randomUUID(); + const sessionUuid = toSessionUuid(rawSessionId); + // `pairWithHostCall: false` is governStep's retry after a REPLAY_DETECTED + // answer. Only this request gets a fresh random id. The host's result hook + // runs on its own and still names the derived id, so the result resolves + // against the earlier request that carried it: the first time the host + // asked about this call. + const invocationId = options.pairWithHostCall === false ? undefined : invocationIdOf(event, entry, payload); + const invocationRequestId = invocationId === undefined ? undefined : toInvocationRequestId(sessionUuid, invocationId); + // The request gate takes the derived id as its request_id. The result gate + // names it as the request it answers and keeps a fresh id of its own. + let requestId: string = randomUUID(); + if (invocationRequestId !== undefined) { + if (acsPayload.outputs === undefined) { + requestId = invocationRequestId; + } else { + acsPayload.request_id_ref = invocationRequestId; + } + } + // The JSON-RPC id stays random even when request_id is derived. The + // envelope log and the Inspector pair a request with its response by this + // id, and a host asking twice about one call would otherwise repeat it. + const rpcId = requestId === invocationRequestId ? randomUUID() : requestId; return { jsonrpc: "2.0", method: entry.acs_method, - id: requestId, + id: rpcId, params: { acs_version: ACS_VERSION, request_id: requestId, timestamp: new Date().toISOString(), metadata: { agent_id: hookmap.host, - session_id: toSessionUuid(rawSessionId), + session_id: sessionUuid, }, payload: acsPayload, }, diff --git a/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts b/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts index 9de89c63..5ef2389c 100644 --- a/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts +++ b/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts @@ -264,6 +264,8 @@ export type ApplyFailurePostureInput = { * knowable. Never a host's own event name -- see AuditEntry.method. */ method: string | null; rpcId: string | number | null; + /** The envelope's `params.request_id`, or null when no request was built. */ + requestId?: string | null; /** Required, not optional: §6.4 makes auditing non-skippable. */ audit: AuditSink; /** @@ -305,6 +307,7 @@ export function applyFailurePosture({ sessionId, method, rpcId, + requestId = null, audit, stage = "delivery", }: ApplyFailurePostureInput): FailureResolvedAcsDecision { @@ -338,6 +341,7 @@ export function applyFailurePosture({ session_id: sessionId, method, rpc_id: rpcId, + request_id: requestId, posture, posture_source: postureSource, outcome, diff --git a/reference-implementations/agt/packages/host-adapter/src/govern-step.ts b/reference-implementations/agt/packages/host-adapter/src/govern-step.ts index 49862c86..09477730 100644 --- a/reference-implementations/agt/packages/host-adapter/src/govern-step.ts +++ b/reference-implementations/agt/packages/host-adapter/src/govern-step.ts @@ -359,6 +359,12 @@ export function governsTool(hookmap: Hookmap, hookEventName: string, tool: strin return tools === undefined || tools.includes(tool); } +/** JSON-RPC REPLAY_DETECTED (Specification §17.1), as the raw error object + * GuardianClient hands back when no decision arrived. */ +function isReplayDetected(failure: unknown): boolean { + return typeof failure === "object" && failure !== null && (failure as { code?: unknown }).code === -32005; +} + /** * Records a step this gate declined to govern -- the `tools` skip, in the * one log that outlives the process. @@ -637,6 +643,7 @@ export async function governStep({ sessionId, method: envelope?.method ?? null, rpcId: envelope?.id ?? null, + requestId: envelope?.params.request_id ?? null, audit, stage, }); @@ -724,7 +731,27 @@ export async function governStep({ // from the envelope it just built. const modificationDocument = modificationDocumentOf(envelope); - const answer = await guardian.requestDecision(envelope, { timeoutMs }); + let answer = await guardian.requestDecision(envelope, { timeoutMs }); + // A request id derived from the host's call id (see buildEnvelope's + // `call_id`) repeats if the host ever asks twice about one call, and a + // Guardian enforcing Specification §10.3 answers the repeat with + // REPLAY_DETECTED. That code carries no decision, so under `proceed` it + // would run the step ungoverned. §17.1's recovery is to regenerate the id + // and retry, so this does that once, with a random id, inside whatever is + // left of the timeout. The result hook still names the derived id, which + // resolves against the first request the host sent for this call. Only + // the request gate derives its own id, so only it can hit this. + if ( + answer.decisionArrived === false && + isReplayDetected(answer.failure) && + envelope.params.payload.outputs === undefined + ) { + const remainingMs = timeoutMs - (performance.now() - startedAt); + if (remainingMs > 0) { + envelope = buildEnvelope(hookEventName, payload, hookmap, { pairWithHostCall: false }); + answer = await guardian.requestDecision(envelope, { timeoutMs: remainingMs }); + } + } const elapsedMs = performance.now() - startedAt; if (answer.decisionArrived === false) { diff --git a/reference-implementations/agt/packages/host-adapter/src/index.ts b/reference-implementations/agt/packages/host-adapter/src/index.ts index 43c2ce40..b223ba58 100644 --- a/reference-implementations/agt/packages/host-adapter/src/index.ts +++ b/reference-implementations/agt/packages/host-adapter/src/index.ts @@ -11,6 +11,7 @@ export { loadHookmap, modificationDocumentOf, toSessionUuid, + toInvocationRequestId, type Hookmap, type HookmapHookEntry, type HookmapOutputs, diff --git a/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts b/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts index de58ac5c..998753f4 100644 --- a/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts +++ b/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "bun:test"; -import { mkdtempSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { validateEnvelope } from "guardian"; @@ -7,6 +7,7 @@ import { buildEnvelope, loadHookmap, toSessionUuid, + toInvocationRequestId, modificationDocumentOf, type Hookmap, type HookmapHookEntry, @@ -1157,3 +1158,128 @@ describe("buildEnvelope", () => { }); }); }); + +describe("call_id: pairing a result with its request", () => { + const withCallId: Hookmap = { + ...hookmap, + hooks: { + PreToolUse: { ...hookmap.hooks.PreToolUse!, call_id: "$.tool_use_id" } as HookmapHookEntry, + PostToolUse: { ...hookmap.hooks.PostToolUse!, call_id: "$.tool_use_id" } as HookmapHookEntry, + }, + }; + const before = { ...preToolUsePayload, tool_use_id: "toolu_01" }; + const after = { ...before, hook_event_name: "PostToolUse", tool_response: { stdout: "done" } }; + + it("gives the result the request's request_id as request_id_ref, and both validate", () => { + const request = buildEnvelope("PreToolUse", before, withCallId); + const result = buildEnvelope("PostToolUse", after, withCallId); + + expect(() => validateEnvelope(request)).not.toThrow(); + expect(() => validateEnvelope(result)).not.toThrow(); + expect(result.params.payload.request_id_ref).toBe(request.params.request_id); + // The JSON-RPC id stays random, so two asks about one call never share it. + expect(request.id).not.toBe(request.params.request_id); + expect(buildEnvelope("PreToolUse", before, withCallId).id).not.toBe(request.id); + expect(result.params.request_id).not.toBe(request.params.request_id); + }); + + it("derives the id from the session and the call id, so another session's call does not collide", () => { + const here = buildEnvelope("PreToolUse", before, withCallId); + const elsewhere = buildEnvelope("PreToolUse", { ...before, session_id: "another-session" }, withCallId); + + expect(here.params.request_id).toBe(toInvocationRequestId(toSessionUuid("abc123"), "toolu_01")); + expect(here.params.request_id).toMatch(UUID_RE); + expect(elsewhere.params.request_id).not.toBe(here.params.request_id); + }); + + it("falls back to a random request_id and no reference when the host sends no call id", () => { + const request = buildEnvelope("PreToolUse", preToolUsePayload, withCallId); + const result = buildEnvelope("PostToolUse", { ...preToolUsePayload, tool_response: { stdout: "done" } }, withCallId); + + expect(request.params.request_id).toMatch(UUID_RE); + expect(result.params.payload.request_id_ref).toBeUndefined(); + expect(() => validateEnvelope(result)).not.toThrow(); + }); + + it("sends no reference when the hookmap declares no call_id", () => { + const result = buildEnvelope("PostToolUse", after, hookmap); + + expect(result.params.payload.request_id_ref).toBeUndefined(); + }); + + it("refuses a call_id that is not a path string, as a hookmap fault", () => { + const broken: Hookmap = { + ...hookmap, + hooks: { ...hookmap.hooks, PreToolUse: { ...hookmap.hooks.PreToolUse!, call_id: 7 } as unknown as HookmapHookEntry }, + }; + + expect(() => buildEnvelope("PreToolUse", before, broken)).toThrow(/"call_id"/); + }); +}); + + +describe("call_id: the real hookmaps, and the derivation pinned", () => { + const CLAUDE_CODE = new URL("../../../hosts/claude-code/claude-code.hookmap.yaml", import.meta.url); + const LINE = " call_id: $.tool_use_id\n"; + + function loadEdited(edit: (yaml: string) => string): Hookmap { + const dir = mkdtempSync(join(tmpdir(), "acs-hookmap-")); + const path = join(dir, "hookmap.yaml"); + writeFileSync(path, edit(readFileSync(CLAUDE_CODE, "utf8"))); + try { + return loadHookmap(path); + } finally { + unlinkSync(path); + rmdirSync(dir); + } + } + + it("pairs a Claude Code request and result through the shipped hookmap, for a session id that is not a uuid", () => { + const shipped = loadHookmap(CLAUDE_CODE.pathname); + const before = { ...preToolUsePayload, session_id: "sess-raw-not-a-uuid", tool_use_id: "toolu_01" }; + const after = { ...before, hook_event_name: "PostToolUse", tool_response: { stdout: "done" } }; + + const request = buildEnvelope("PreToolUse", before, shipped); + const result = buildEnvelope("PostToolUse", after, shipped); + + expect(result.params.payload.request_id_ref).toBe(request.params.request_id); + }); + + it("keeps the derivation stable, because a changed one unpairs every call that straddles an upgrade", () => { + expect(toSessionUuid("abc123")).toBe("4c0c070f-241c-57eb-b406-ca28fe9f1229"); + expect(toInvocationRequestId("4c0c070f-241c-57eb-b406-ca28fe9f1229", "toolu_01")).toBe( + "044e7d73-4d3f-8dc7-abdc-6200ff5a7734", + ); + }); + + it("refuses at load a call_id that is not a path string", () => { + expect(() => loadEdited((yaml) => yaml.replace(LINE, " call_id: 7\n"))).toThrow(/"call_id" must be one JSONPath-lite string/); + }); + + it("refuses at load a call_id declared on the request gate but not the result gate", () => { + const lastAt = (yaml: string) => yaml.lastIndexOf(LINE); + expect(() => + loadEdited((yaml) => yaml.slice(0, lastAt(yaml)) + yaml.slice(lastAt(yaml) + LINE.length)), + ).toThrow(/declares "call_id" on some tool gates but not on \["PostToolUse"\]/); + }); + + it("refuses at load a call_id naming a reserved segment, which resolvePath would refuse on every step", () => { + expect(() => loadEdited((yaml) => yaml.split(LINE).join(" call_id: $.constructor\n"))).toThrow( + /call_id" is not a usable path: .*reserved segment "constructor"/, + ); + }); + + it("refuses at load a call_id on result gates when the hookmap maps no request gate", () => { + // Drop the PreToolUse entry whole: everything from its key to the next hook. + const withoutRequestGate = (yaml: string) => { + const start = yaml.indexOf(" PreToolUse:\n"); + const end = yaml.indexOf(" PostToolUse:\n"); + return yaml.slice(0, start) + yaml.slice(end); + }; + expect(() => loadEdited(withoutRequestGate)).toThrow(/declares "call_id" only on result gates/); + }); + + it("loads a hookmap that declares call_id on neither gate", () => { + expect(() => loadEdited((yaml) => yaml.split(LINE).join(""))).not.toThrow(); + }); +}); diff --git a/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts b/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts index 5134115a..563c9cf2 100644 --- a/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts +++ b/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts @@ -1188,3 +1188,79 @@ describe("governsTool — the rule both hosts and governStep share", () => { expect(governsTool(scopedHookmap, "toString", "bash")).toBe(true); }); }); + +describe("governStep: a request id derived from the host's call id", () => { + // buildEnvelope derives a request's id from the host's call id, so a host + // that asks twice about one call sends the same id twice. A Guardian + // enforcing Specification §10.3 answers the repeat with REPLAY_DETECTED, + // which carries no decision and would otherwise go to the posture. + const paired: Hookmap = { host: "test-host", hooks: { OnStep: { ...ON_STEP, call_id: "$.call" } } }; + const pairedPayload = { ...payload, call: "call-1" }; + const REPLAY = { decisionArrived: false, failure: { code: -32005, message: "duplicate request_id" } } as const; + const ALLOW = { decisionArrived: true, decision: { type: "decision", decision: "allow" } } as unknown as DecisionOrFailure; + + function scripted(answers: DecisionOrFailure[]): { guardian: GuardianClient; sent: string[] } { + const sent: string[] = []; + return { + sent, + guardian: { + requestDecision: (envelope) => { + sent.push(envelope.params.request_id); + return Promise.resolve(answers.shift() ?? REPLAY); + }, + post: () => Promise.reject(new Error("governStep must not use the wire primitive")), + }, + }; + } + + it("retries once with a fresh random id, and honours the decision that arrives", async () => { + const { sink, events } = recordingSink(); + const { guardian, sent } = scripted([REPLAY, ALLOW]); + + const governed = await govern(guardian, sink, undefined, { hookmap: paired, payload: pairedPayload }); + + expect(governed.stage).toBe("honoured"); + expect(governed.decision?.decision).toBe("allow"); + expect(sent).toHaveLength(2); + expect(sent[1]).not.toBe(sent[0]); + expect(events).toHaveLength(0); + }); + + it("retries only once, and leaves a second REPLAY_DETECTED to the delivery posture (issue #32)", async () => { + const { sink, events } = recordingSink(); + const { guardian, sent } = scripted([REPLAY, REPLAY]); + + const governed = await govern(guardian, sink, undefined, { hookmap: paired, payload: pairedPayload }); + + expect(sent).toHaveLength(2); + expect(governed.stage).toBe("delivery"); + // The audit entry names the request id the retry actually sent. + expect(events).toHaveLength(1); + expect(events[0]?.request_id).toBe(sent[1]); + }); + + it("audits the derived request_id apart from the random JSON-RPC id when no decision arrives", async () => { + const { sink, events } = recordingSink(); + const { guardian, sent } = scripted([{ decisionArrived: false, failure: new GuardianTimeoutError(120) }]); + + await govern(guardian, sink, undefined, { hookmap: paired, payload: pairedPayload }); + + expect(events).toHaveLength(1); + expect(events[0]?.request_id).toBe(sent[0]); + expect(events[0]?.rpc_id).not.toBe(sent[0]); + }); + + it("does not retry at the result gate, whose own id is always random", async () => { + const { sink } = recordingSink(); + const { guardian, sent } = scripted([REPLAY, ALLOW]); + const resultHookmap: Hookmap = { host: "test-host", hooks: { OnResult: { ...ON_RESULT, call_id: "$.call" } } }; + + await governRaw(guardian, sink, undefined, { + hookmap: resultHookmap, + hookEventName: "OnResult", + payload: { ...resultPayload, call: "call-1" }, + }); + + expect(sent).toHaveLength(1); + }); +}); diff --git a/reference-implementations/agt/packages/inspector/README.md b/reference-implementations/agt/packages/inspector/README.md index b9035961..8769628b 100644 --- a/reference-implementations/agt/packages/inspector/README.md +++ b/reference-implementations/agt/packages/inspector/README.md @@ -35,7 +35,7 @@ A flag wins over its variable. The Inspector honours `NO_COLOR` and prints no co ## What each line means -An envelope prints as a header line and the envelope body. The header carries the sequence number from the log, the time it was recorded, the direction, the method and the JSON-RPC id. A response carries a decision badge above the body, with the reason codes and policy references on the same line: +An envelope prints as a header line and the envelope body. The header carries the sequence number from the log, the time it was recorded, the direction, the method and the JSON-RPC id. A `finding` line, which the Guardian writes for a `request_id_ref` it cannot resolve, prints with `! FINDING` in place of the direction. A response carries a decision badge above the body, with the reason codes and policy references on the same line: ``` ● DENY reason_codes=[destructive_shell_command_blocked] policy_references=[agt_stock#destructive_shell_command_blocked] diff --git a/reference-implementations/agt/packages/inspector/src/render.ts b/reference-implementations/agt/packages/inspector/src/render.ts index 1d01ab2b..fd8a71dc 100644 --- a/reference-implementations/agt/packages/inspector/src/render.ts +++ b/reference-implementations/agt/packages/inspector/src/render.ts @@ -233,7 +233,7 @@ export function renderOutcome(message: OutcomeMessage, options: RenderOptions = */ export function renderEnvelopeLogEntry(entry: EnvelopeLogEntry, options: RenderOptions = {}): string { const color = options.color ?? false; - const arrow = entry.direction === "request" ? "→ REQUEST " : "← RESPONSE"; + const arrow = { request: "→ REQUEST ", response: "← RESPONSE", finding: "! FINDING " }[entry.direction]; const method = entry.method ?? "(no method)"; const id = entry.rpc_id === null ? "(unpaired)" : `id=${entry.rpc_id}`; @@ -467,9 +467,13 @@ export function renderSessionChainRow( options: RenderOptions = {}, ): string { const color = options.color ?? false; + const findings = + entry.findings === undefined || entry.findings.length === 0 + ? "" + : ` ! ${entry.findings.join(", ")}${entry.request_id_ref === undefined ? "" : ` ref=${entry.request_id_ref}`}`; const row = `#${entry.seq} ${entry.tool_name} hash=${entry.hash.slice(0, SHORT_HASH_LENGTH)} ` + - `session_id=${entry.session_id}`; + `session_id=${entry.session_id}${findings}`; return link.broken ? paint(`✖ CHAIN BREAK ${row}`, RED, color) : row; } diff --git a/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts b/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts index ffd6a31a..2bc13868 100644 --- a/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts +++ b/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts @@ -70,6 +70,8 @@ type AuditEntryIdentity = { * host's own event name: this reader knows ACS and nothing else. */ method: string | null; rpc_id: string | number | null; + /** The envelope's request_id. Absent on entries written before it existed. */ + request_id?: string | null; }; /** A step the writer's negotiated posture answered, because no decision @@ -136,7 +138,8 @@ function isAuditEntryShape(value: unknown): value is AuditEntry { typeof candidate.recorded_at === "string" && typeof candidate.session_id === "string" && (typeof candidate.method === "string" || candidate.method === null) && - (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) + (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) && + (candidate.request_id === undefined || typeof candidate.request_id === "string" || candidate.request_id === null) ) ) { return false; diff --git a/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts b/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts index ca1db33f..648cbe65 100644 --- a/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts +++ b/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts @@ -59,7 +59,10 @@ */ import { closeSync, existsSync, openSync, readSync, statSync } from "node:fs"; -export type EnvelopeLogDirection = "request" | "response"; +// "finding" is a fact the Guardian established about a request that the spec +// requires it to record, such as an unresolved request_id_ref. Its envelope is +// the Guardian's own record, not a JSON-RPC object. +export type EnvelopeLogDirection = "request" | "response" | "finding"; /** One line of the envelope log the Guardian writes. */ export type EnvelopeLogEntry = { @@ -106,7 +109,7 @@ function isEnvelopeLogEntryShape(value: unknown): value is EnvelopeLogEntry { return ( typeof candidate.seq === "number" && typeof candidate.recorded_at === "string" && - (candidate.direction === "request" || candidate.direction === "response") && + (candidate.direction === "request" || candidate.direction === "response" || candidate.direction === "finding") && (typeof candidate.method === "string" || candidate.method === null) && (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) ); diff --git a/reference-implementations/agt/packages/inspector/src/tail-session-context.ts b/reference-implementations/agt/packages/inspector/src/tail-session-context.ts index 00cc6066..11360601 100644 --- a/reference-implementations/agt/packages/inspector/src/tail-session-context.ts +++ b/reference-implementations/agt/packages/inspector/src/tail-session-context.ts @@ -55,6 +55,11 @@ export type SessionContextLogEntry = { method: string; request_id: string; tool_name: string; + /** What the Guardian established about the step as it arrived, present + * only when it established something. */ + findings?: string[]; + /** The unresolved reference, present only with `request_id_ref_unresolved`. */ + request_id_ref?: string; }; export type TailSessionContextLogOptions = { @@ -97,7 +102,10 @@ function isSessionContextLogEntryShape(value: unknown): value is SessionContextL typeof candidate.recorded_at === "string" && typeof candidate.method === "string" && typeof candidate.request_id === "string" && - typeof candidate.tool_name === "string" + typeof candidate.tool_name === "string" && + (candidate.findings === undefined || + (Array.isArray(candidate.findings) && candidate.findings.every((finding) => typeof finding === "string"))) && + (candidate.request_id_ref === undefined || typeof candidate.request_id_ref === "string") ); } diff --git a/reference-implementations/agt/packages/inspector/test/render.test.ts b/reference-implementations/agt/packages/inspector/test/render.test.ts index 758fc95d..936ada14 100644 --- a/reference-implementations/agt/packages/inspector/test/render.test.ts +++ b/reference-implementations/agt/packages/inspector/test/render.test.ts @@ -202,6 +202,21 @@ describe("renderDecisionBadge", () => { }); describe("renderEnvelopeLogEntry", () => { + it("labels a finding so it reads apart from the request and response it concerns", () => { + const rendered = renderEnvelopeLogEntry( + entry({ + direction: "finding", + seq: 2, + method: "steps/toolCallResult", + envelope: { finding: "request_id_ref_unresolved", request_id_ref: "6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36" }, + }), + ); + + expect(rendered.split("\n")[0]).toBe("── #2 12:04:31.221 ! FINDING steps/toolCallResult id=1"); + expect(rendered).toContain('"finding": "request_id_ref_unresolved"'); + expect(rendered).not.toContain("●"); + }); + it("renders a request as a header line plus pretty JSON, with no badge", () => { const rendered = renderEnvelopeLogEntry( entry({ diff --git a/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts b/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts index c1d6eb72..0564e1ba 100644 --- a/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts +++ b/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts @@ -6,7 +6,7 @@ import { tailEnvelopeLog, type EnvelopeLogEntry } from "../src/tail-envelope-log const POLL_MS = 10; -function entryLine(seq: number, direction: "request" | "response"): string { +function entryLine(seq: number, direction: "request" | "response" | "finding"): string { const entry: EnvelopeLogEntry = { seq, recorded_at: "2026-08-09T12:04:31.221Z", @@ -71,6 +71,25 @@ describe("tailEnvelopeLog", () => { }); }); + it("yields a finding line, which the Guardian writes for an unresolved request_id_ref", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request") + entryLine(2, "finding")); + const controller = new AbortController(); + const malformed: string[] = []; + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 2, controller); + expect(entries.map((e) => e.direction)).toEqual(["request", "finding"]); + expect(malformed).toEqual([]); + }); + }); + it("yields pre-existing entries when fromStart is set", async () => { await withTempDir(async (_dir, path) => { writeFileSync(path, entryLine(1, "request") + entryLine(2, "response")); diff --git a/reference-implementations/agt/test/audit-to-otlp.test.ts b/reference-implementations/agt/test/audit-to-otlp.test.ts index 5d8896db..0868e4d8 100644 --- a/reference-implementations/agt/test/audit-to-otlp.test.ts +++ b/reference-implementations/agt/test/audit-to-otlp.test.ts @@ -272,6 +272,15 @@ describe("audit-to-otlp: review fixes", () => { }); }); + it("prefers the entry's request_id over rpc_id, because a derived request id differs from the JSON-RPC id", async () => { + await withScratch(async (auditPath, otelPath) => { + const requestId = crypto.randomUUID(); + appendFileSync(auditPath, entryLine({ rpc_id: crypto.randomUUID(), request_id: requestId })); + await convertAuditLog(auditPath, otelPath); + expect(attribute(spansIn(otelPath)[0], "acs.request_id")).toBe(requestId); + }); + }); + it("derives the session UUID exactly as the host adapter does", () => { for (const raw of ["sess-1", "ses_opencode_abc", "6B1F2C9E-4D7A-4E3B-8A51-9C0D2E7F1A36", "セッション-é", ""]) { expect({ raw, uuid: converterSessionUuid(raw) }).toEqual({ raw, uuid: toSessionUuid(raw) }); diff --git a/specification/v0.1.0/hooks/tool-call-result.json b/specification/v0.1.0/hooks/tool-call-result.json index 6f4bfcd3..9de8af22 100644 --- a/specification/v0.1.0/hooks/tool-call-result.json +++ b/specification/v0.1.0/hooks/tool-call-result.json @@ -19,7 +19,7 @@ "request_id_ref": { "type": "string", "format": "uuid", - "description": "Links to the originating toolCallRequest" + "description": "The request_id of the originating toolCallRequest. How a Guardian resolves it, and what it records when it cannot, is in hooks.md, toolCallResult." }, "exit_status": { "type": "string", diff --git a/specification/v0.1.0/request-envelope.json b/specification/v0.1.0/request-envelope.json index 61218a26..2b851a81 100644 --- a/specification/v0.1.0/request-envelope.json +++ b/specification/v0.1.0/request-envelope.json @@ -122,7 +122,7 @@ }, "key_id": { "type": "string", - "description": "Identifier resolving to the public key (or the hybrid key descriptor that pins both component public keys for hybrid algorithms)." + "description": "For HMAC-SHA256, an opaque label chosen by the deployment that names one configured secret (Specification 10, key_id for HMAC). For asymmetric algorithms, the identifier resolving to the public key (or the hybrid key descriptor that pins both component public keys for hybrid algorithms)." } } } diff --git a/specification/v0.1.0/response-envelope.json b/specification/v0.1.0/response-envelope.json index 56a16a22..87f645fc 100644 --- a/specification/v0.1.0/response-envelope.json +++ b/specification/v0.1.0/response-envelope.json @@ -89,7 +89,7 @@ "chain_hash": { "type": "string", "pattern": "^[0-9a-f]{64}$", - "description": "Rolling SHA-256 audit-chain head (lowercase hex) after this step's ContextEntry was appended. The Guardian MUST include it on every response for a step where it wrote a ContextEntry, the content-bearing steps (Specification 8). Publishing the head on the wire lets an observer that records traffic detect a chain rewritten after the fact. It is covered by the response signature. Under the HMAC baseline this gives integrity and tamper-evidence to a key-holder and supports cross-Guardian comparison; non-repudiation against a compromised Guardian requires the asymmetric ACS-Crypto profile." + "description": "Rolling SHA-256 audit-chain head (lowercase hex) after this step's ContextEntry was appended. The Guardian MUST include it on every response for a step where it wrote a ContextEntry, the content-bearing steps (Specification 8). Publishing the head on the wire lets an observer that records traffic detect a chain rewritten after the fact. It is covered by the response signature. Under the HMAC baseline this gives integrity and tamper-evidence to a key-holder and supports cross-Guardian comparison. Both parties hold the key, so a signed head is attributable to neither, and non-repudiation requires the asymmetric ACS-Crypto profile (Specification 8.6)." }, "signature": { "$ref": "request-envelope.json#/$defs/Signature", diff --git a/tests/test_hmac_key_derivation.py b/tests/test_hmac_key_derivation.py new file mode 100644 index 00000000..bb0d73e9 --- /dev/null +++ b/tests/test_hmac_key_derivation.py @@ -0,0 +1,54 @@ +"""The HMAC key-derivation example in Specification section 10 matches its parameters. + +The example is the one known-answer value a second implementer checks against, so +a typo in it, or a parameter change that forgets to update it, would send every +new implementation chasing a key no correct HKDF produces. This recomputes it from +the stated parameters with the standard library alone. +""" + +import hashlib +import hmac +import re +from pathlib import Path + +SPEC = Path(__file__).resolve().parent.parent / "docs" / "spec" / "instrument" / "specification.md" + + +def _hkdf_sha256(ikm: bytes, info: bytes, length: int) -> bytes: + # RFC 5869 with no salt: extract uses HashLen zero bytes as the HMAC key. + prk = hmac.new(b"\x00" * hashlib.sha256().digest_size, ikm, hashlib.sha256).digest() + okm, block, counter = b"", b"", 1 + while len(okm) < length: + block = hmac.new(prk, block + info + bytes([counter]), hashlib.sha256).digest() + okm += block + counter += 1 + return okm[:length] + + +def test_section_10_hkdf_example_matches_its_parameters(): + text = SPEC.read_text(encoding="utf-8") + example = re.search( + r"the `session_id` `(?P[0-9a-f-]{36})` give the key `(?P[0-9a-f]{64})`", text + ) + assert example, "Specification section 10 no longer carries the HKDF example in the expected form" + derived = _hkdf_sha256(bytes(range(32)), example["sid"].encode("utf-8"), 32) + assert derived.hex() == example["key"] + + +def test_hkdf_helper_matches_rfc_5869_test_case_3(): + # RFC 5869 Appendix A.3, the SHA-256 case with no salt and empty info, so + # the helper above is anchored to the RFC and not only to the spec's own example. + okm = _hkdf_sha256(b"\x0b" * 22, b"", 42) + assert okm.hex() == ( + "8da4e775a563c18f715f802a063c5a31b8a11f5c5ee1879ec3454e5f3c738d2d9d201395faa4b61a96c8" + ) + + +def test_session_id_is_used_as_sent_not_normalized(): + # Section 10 says info is the session_id string exactly as the request carries + # it. The schema's uuid format accepts other spellings of the same UUID, and + # each one derives a different key, which is why normalizing breaks interop. + sid = "6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36" + as_sent = _hkdf_sha256(bytes(range(32)), sid.encode("utf-8"), 32) + for spelling in (sid.upper(), f"urn:uuid:{sid}"): + assert _hkdf_sha256(bytes(range(32)), spelling.encode("utf-8"), 32) != as_sent