Fix the HMAC key derivation, define key_id, and record an unresolved request_id_ref

docs/spec/instrument/hooks.md

@@ -201,7 +201,9 @@ Schema: [`hooks/tool-call-result.json`](https://genai-security-project.github.io
 
 Fires after tool execution, before the result is ingested into the agent's reasoning context. Provenance: `origin: tool_output`, with `derived_from` set to the originating `toolCallRequest`'s provenance ids when the tool's output is data-derived.
 
-**Payload:** `execution_id` (correlated to the request), outputs, exit status, duration.
+**Payload:** `request_id_ref` (the originating request), outputs, exit status, duration.
+
+`request_id_ref`, when present, is the `request_id` of the originating `toolCallRequest`. A Guardian SHOULD resolve it against the `toolCallRequest` records it holds for the session, and MUST record the reference in its audit record when it cannot resolve it. Whether an unresolvable reference is grounds to deny is the deployment's policy. The record is required because an unresolvable reference is evidence, so it has to survive even when the result is allowed.
 
 **Decision:** ALLOW / DENY / MODIFY.
 

docs/spec/instrument/specification.md

@@ -310,7 +310,7 @@ For every step where the Guardian writes a ContextEntry (the content-bearing ste
 
 A `subagentStop` payload MAY omit `final_chain_hash`: a framework that maintains no session-chain omits the field rather than fabricate a value, since fabrication corrupts the artifact the field exists to produce. A Guardian MUST treat the omission as "chain not maintained by this framework," not as an integrity failure; integrity claims about that subagent's chain are simply unavailable.
 
-Tamper-evidence here is bounded by the signature in use. Under the HMAC baseline the published, signed head gives integrity against a network tamperer and lets a key-holder verify the chain, but it does not bind the Guardian itself: the Guardian holds the symmetric key and can re-sign a rewritten head. Non-repudiation, proving to a third party that a specific Guardian issued a specific head, requires the asymmetric ACS-Crypto profile. The baseline detects accidental divergence and cross-Guardian disagreement; defeating a determined, compromised Guardian is a profile-level guarantee, not a Core one.
+Tamper-evidence here is bounded by the signature in use. Under the HMAC baseline the published, signed head gives integrity against a network tamperer and lets a key-holder verify the chain, but it binds neither party. The Guardian and the Observed Agent derive the same per-session key (§10), so the Guardian can re-sign a rewritten head, and the Observed Agent can produce a validly signed response the Guardian never sent, an `allow` or a `chain_hash` included. A valid HMAC shows that a holder of the session key produced the message, not which holder, so under HMAC a Guardian decision is not attributable to the Guardian against the Observed Agent. Non-repudiation, proving to a third party that a specific Guardian issued a specific head or decision, requires the asymmetric ACS-Crypto profile. The baseline detects accidental divergence and cross-Guardian disagreement; defeating a determined, compromised Guardian is a profile-level guarantee, not a Core one.
 
 ## 9. Approver Model
 
@@ -349,6 +349,10 @@ This rule preserves the security guarantee (actions that would have been `ASK`'d
 
 A signature over the §10 canonical input is REQUIRED in ACS-Core. The signature envelope is `{ algorithm, value, key_id }`, the algorithm drawn from the registry below and the per-direction supported set declared in the handshake. `HMAC-SHA256` is the baseline that satisfies the requirement; ACS-Core does not require asymmetric or post-quantum signatures, which are the ACS-Crypto profile (§10.1). The per-session HMAC key is `HKDF`-derived from deployment-provided input keying material (a pre-shared secret, or a transport channel binding such as a TLS exporter) together with the `session_id`. ACS v0.1 defines no in-band key-exchange: a deployment with neither a shared secret nor a secure transport cannot use the HMAC baseline and satisfies the requirement with an ACS-Crypto algorithm whose key distribution it manages out of band.
 
+**HMAC key derivation.** For `HMAC-SHA256`, the per-session key is HKDF ([RFC 5869](https://www.rfc-editor.org/rfc/rfc5869)) with SHA-256, no salt (RFC 5869's default, a string of 32 zero bytes, which gives the same key as an empty salt), `info` set to the UTF-8 bytes of `metadata.session_id`, and a 32-byte output, applied to the secret that `key_id` names. Both parties MUST derive the key with exactly these parameters, so independent implementations holding the same secret derive the same key. `info` is the `session_id` string exactly as the request carries it, not parsed or normalized first: an uppercase or `urn:uuid:` spelling of the same UUID derives a different key. A response carries no `session_id`, so it is signed and verified with the key of the request it answers. The secret MUST be at least 32 bytes from a cryptographically secure random source. HKDF does no key stretching, so a passphrase used as the secret can be recovered offline from one captured signature. This derivation is defined for a pre-shared secret. v0.1 defines no exporter label or context for a channel-binding input, so that input is not yet interoperable between implementations. For example, the 32-byte secret `000102…1e1f` (the bytes 0 through 31) and the `session_id` `6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36` give the key `80f0dc8c611279bfc2af2d5f374c5cb685888192ce1af5c3e0673d8fa748e8a4`.
+
+**`key_id` for HMAC.** For `HMAC-SHA256`, `key_id` is an opaque label, chosen by the deployment, that names one configured secret. This specification reserves no values, and an implementation takes the label from its configuration. It does not hardcode one. A verifier MUST reject an unknown `key_id` (`SIGNATURE_INVALID`, §17.1) and MUST NOT fall back to another secret. `key_id` MUST NOT be derived from the secret. A Guardian that signs its response with `HMAC-SHA256` uses the secret, and the `key_id`, that the request named. The `SIGNATURE_INVALID` error for an unknown `key_id` is unsigned, because no configured secret matches it. A deployment rotating its secret configures the new secret under a new label, such as `"2026-10"`, so a verifier can hold both secrets while the rotation is in progress. That is what the §17.1 recovery for `SIGNATURE_INVALID`, re-resolving `key_id`, relies on. For the asymmetric algorithms of ACS-Crypto, `key_id` identifies the public key or hybrid key descriptor (§10.2).
+
 **Signed input.** The signed input for every algorithm is the RFC 8785 (JCS) canonicalization of the request or response envelope with the `signature` field removed, encoded as UTF-8. This binds the signature to the whole envelope, including `method`, `metadata.session_id`, `request_id`, and `timestamp`, so a captured signature cannot be lifted into a different envelope (for example, re-wrapping a signed `toolCallResult` under a different `session_id`). A verifier MUST recompute this canonical form and MUST reject a signature that does not cover it (`SIGNATURE_INVALID`, §17.1). The input is fixed by this rule, not declared per message: a discoverable `signed_fields` would let a producer advertise partial coverage that omits `session_id`, reopening the gap this closes. Alternative canonicalization is not permitted in v0.1, matching the chain-hash rule in §8.
 
 ### 10.1 Algorithm registry
@@ -357,7 +361,7 @@ The registry is crypto-agile: the `{ algorithm, value, key_id }` envelope suppor
 
 | Algorithm | Class | v0.1.0 status |
 |---|---|---|
-| `HMAC-SHA256` | Symmetric MAC | RECOMMENDED. Shared-secret integrity; simplest deployment path. Sufficient for same-host and trusted-network topologies where the threat is accidental tampering or replay. |
+| `HMAC-SHA256` | Symmetric MAC | RECOMMENDED. Shared-secret integrity; simplest deployment path. Sufficient for same-host and trusted-network topologies where the threat is accidental tampering or replay. Both parties hold the key, so it attributes a message to neither (§8.6). |
 | `ECDSA-P256` | Classical asymmetric | OPTIONAL. Strongest current ecosystem support across Java, Node, .NET, HSMs, and major cloud KMS providers. |
 | `RSA-PSS-SHA256` | Classical asymmetric | OPTIONAL. Legacy interop; deployments with existing RSA PKI. |
 | `ML-DSA-65` | PQC, lattice (FIPS 204) | OPTIONAL. ~128-bit post-quantum security; ~3.3 KB signatures. Recommended for deployments shipping PQC libraries today. |

reference-implementations/agt/README.md

@@ -312,7 +312,7 @@ Verified on macOS (arm64): natively with otelcol-contrib 0.162.0, and with the D
 
 **Run it again.** Every output here except the converter's appends, so a second run on the same `.acs` doubles every count, and the converter refuses to run at all. Stop the Guardian and the collector first, with Ctrl-C or `docker stop acs-otelcol`. A process that is still running keeps writing into the deleted file, and its spans are lost without an error. Then delete it: `rm -rf .acs && mkdir -p .acs && chmod 700 .acs`.
 
-**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability` and `acs.request_id_ref`, because the shim doesn't send them yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). The committed sample drives the Guardian directly, so it has both. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229).
+**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229).
 
 The trace files, the audit log and the collector's port can be written by any process running as you, the observed agent included, and in a container by other containers on the same Docker network, so the output isn't tamper-evident. `service.name` (`acs-reference-guardian`, `acs-reference-audit-converter`) is a label each writer sets for itself, not authentication, and loopback isn't authentication either. These labels belong to the reference implementation, not to the ACS specification.
 
@@ -420,7 +420,7 @@ The `.acs/` directory holds four kinds of local file. None is ever committed.
 
 | File | Written by | Holds |
 |---|---|---|
-| `envelopes.jsonl` | Guardian | Every envelope, both directions, before validation |
+| `envelopes.jsonl` | Guardian | Every envelope, both directions, before validation, plus a `finding` line for a fact the spec requires the Guardian to record, such as a `request_id_ref` it cannot resolve |
 | `session-context.jsonl` | Guardian | One hash-chain entry per governed step |
 | `audit.jsonl` | Host shim | Every fail-open proceed and every posture-driven block |
 | `sessions/<session_id>.json` | Host shim, Claude Code only | The negotiated ServerHello, so the next hook subprocess finds it |
@@ -492,7 +492,7 @@ The suite has 68 test files. Most of them drive a real Guardian, the real host s
 
 These are known limits of the tree as it stands. None is fixed.
 
-- **Memory and disk grow without bound.** The Guardian's session store never evicts a session. The envelope log, the audit log and the session-context log have no rotation and no size cap. A long-lived Guardian grows until something else stops it.
+- **Memory and disk grow without bound.** The Guardian's session store keeps at most 1024 sessions and evicts the least recently written one, but it bounds sessions, not the steps within one. The envelope log, the audit log and the session-context log have no rotation and no size cap. A long-lived Guardian grows until something else stops it.
 - **Log writes sit on the decision path.** The Guardian writes two synchronous file appends per request in a single-threaded server. A slow disk blocks every in-flight decision.
 - **The wire is unauthenticated.** See [What this project is, and is not](#what-this-project-is-and-is-not). Loopback by default is the only protection.
 - **The egress gate has three measured soft spots.** A shell command whose destination the extractor cannot parse falls through to `allow`, not `deny`. With the shipped allowlist, any command whose text contains an off-list URL is denied, whether or not it reaches that URL. A `WebFetch` URL reaches AGT's gate with no parsing, so four userinfo-style URL shapes pass on the fetch route and fail on the shell route. Closing the last one means editing a `.rego` file, which this project does not do.
@@ -522,6 +522,11 @@ These findings are open on this tree. They are recorded here so a reader does no
 - A `steps/skillLoad` hook that drives AGT's `content_hash` gate is planned, not built. No code for it exists in the tree, and the ServerHello does not name it.
 - Two AGT intervention points, `pre_model_call` and `post_model_call`, have no ACS v0.1.0 method. Whether this project proposes `steps/modelCall` for ACS v0.2 is open.
 - Whether to report a fail-open to AGT upstream is open: a `./`-prefixed `bundle:` path silently disables all policy and returns `allow`. This tree throws on that path as a backstop.
+- A finding is recorded on the step's own entry in the hash chain (`session-context.jsonl`), as `findings` plus the unresolved `request_id_ref`, and the entry's hash covers both. Entries with no findings hash exactly as before. A readable copy goes to the envelope log, or, when there is no envelope log or it has disabled itself, to stderr as `acs-finding ` followed by one JSON object with the same fields plus `recorded_at`, `method` and `rpc_id`. Findings don't reach the OTel export. The chain is tamper-evident only as far as §8.6 allows, since this Guardian publishes no chain head.
+- A reference fails to resolve for three ordinary reasons as well as a suspicious one: the request was recorded before a Guardian restart, its session was evicted, or the request never reached this Guardian and the host proceeded under its posture. The finding doesn't say which. Run one Guardian per session, or route by session, or every result that lands on a different Guardian is a finding.
+- A reference to a request the Guardian denied counts as resolved, so a result for a denied tool call gets no finding.
+- Specification §10.3 says a Guardian MUST reject a duplicate `request_id` in a session. This Guardian records a `request_id_duplicate` finding instead, for a `steps/toolCallRequest` or `steps/toolCallResult` that reuses an id any earlier step in the session carried, and still decides the step. It sees only the two methods it dispatches.
+- The shim derives a request's `request_id` from the host's call id, so asking twice about one call repeats the id. If a Guardian answers that with `REPLAY_DETECTED`, the shim retries that request once with a random id. The result hook runs on its own and still names the derived id, so the result resolves against the first request the host sent for that call. The JSON-RPC `id` stays random either way, so the envelope log never pairs two requests under one id, and the host audit log records the `request_id` separately as `request_id`.
 - Six required Trace-pillar attributes exist on the wire but are optional there. Requiring them is an ask for v0.2. The Guardian also leaves `AcsResult.metadata` empty, which it could fill today.
 
 ## Verify

reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml

@@ -19,6 +19,9 @@ hooks:
   PreToolUse:
     acs_method: steps/toolCallRequest
     tool_name: $.tool_name
+    # Claude Code's id for this one tool call, the same on PreToolUse and
+    # PostToolUse, so the result can name the request it answers.
+    call_id: $.tool_use_id
     arguments: $.tool_input
     # ACS's own raw_command: the command line this step IS, verbatim, which is
     # a different thing from the argument bag above and is why it gets its own
@@ -128,6 +131,9 @@ hooks:
   PostToolUse:
     acs_method: steps/toolCallResult
     tool_name: $.tool_name
+    # Claude Code's id for this one tool call, the same on PreToolUse and
+    # PostToolUse, so the result can name the request it answers.
+    call_id: $.tool_use_id
     # THIS GATE'S MATCHER STAYS SINGLE-TOOL WHILE `outputs.from` IS A SINGLE
     # PATH, and that is the fact to read before widening `^Bash$` in
     # settings.json to match the request gate's `^(Bash|WebFetch)$`. JSON

reference-implementations/agt/hosts/claude-code/test/pairing.test.ts

@@ -0,0 +1,72 @@
+import { describe, expect, it } from "bun:test";
+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { fileURLToPath } from "node:url";
+// Test-only import, the same precedent as hook.test.ts: a real Guardian, so
+// this proves the pairing on the wire rather than against a copied shape.
+import { startGuardian } from "guardian";
+import {
+  createGuardianClient,
+  createMemorySessionConfigStore,
+  DEFAULT_TIMEOUT_MS,
+  governStep,
+  loadHookmap,
+  NULL_AUDIT_SINK,
+  resolveSessionConfig,
+  toSessionUuid,
+} from "host-adapter";
+
+const HOOKMAP_PATH = fileURLToPath(new URL("../claude-code.hookmap.yaml", import.meta.url));
+
+// The shipped hookmap names `tool_use_id` as the call id. Claude Code's hooks
+// reference shows the field on both PreToolUse and PostToolUse, and a
+// transcript check found the same value on both hooks of one call.
+function findingLines(path: string): unknown[] {
+  if (!existsSync(path)) {
+    return [];
+  }
+  return readFileSync(path, "utf8")
+    .split("\n")
+    .filter((line) => line.trim() !== "")
+    .map((line) => JSON.parse(line) as { direction: string; envelope: unknown })
+    .filter((entry) => entry.direction === "finding")
+    .map((entry) => entry.envelope);
+}
+
+async function runCall(sessionId: string, beforeId: string, afterId: string): Promise<unknown[]> {
+  const dir = mkdtempSync(join(tmpdir(), "acs-pairing-"));
+  const logPath = join(dir, "envelopes.jsonl");
+  const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath });
+  try {
+    const hookmap = loadHookmap(HOOKMAP_PATH);
+    const client = createGuardianClient(guardian.url);
+    const session = await resolveSessionConfig(
+      { guardian: client, agentId: hookmap.host, sessionId: toSessionUuid(sessionId), timeoutMs: DEFAULT_TIMEOUT_MS },
+      createMemorySessionConfigStore(),
+    );
+    const common = { session_id: sessionId, tool_name: "Bash", tool_input: { command: "ls" } };
+    for (const [hookEventName, payload] of [
+      ["PreToolUse", { ...common, hook_event_name: "PreToolUse", tool_use_id: beforeId }],
+      ["PostToolUse", { ...common, hook_event_name: "PostToolUse", tool_use_id: afterId, tool_response: { stdout: "a.txt" } }],
+    ] as const) {
+      await governStep({ hookEventName, payload, hookmap, guardian: client, session, sessionId, audit: NULL_AUDIT_SINK });
+    }
+    return findingLines(logPath);
+  } finally {
+    await guardian.close();
+    rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+describe("Claude Code: a result names the request it answers", () => {
+  it("records no finding when PostToolUse carries the same tool_use_id as PreToolUse", async () => {
+    expect(await runCall("sess-raw-not-a-uuid", "toolu_01", "toolu_01")).toHaveLength(0);
+  });
+
+  it("records an unresolved reference when the result's call never reached the Guardian", async () => {
+    const findings = await runCall("sess-raw-not-a-uuid", "toolu_01", "toolu_02");
+    expect(findings).toHaveLength(1);
+    expect(findings[0]).toMatchObject({ finding: "request_id_ref_unresolved" });
+  });
+});