Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,9 @@ as synchronous file appends on the decision path, with no batching. Trace export
partial: set `ACS_OTEL_FILE` or `ACS_OTEL_ENDPOINT` and the Guardian exports OTLP spans
for the two live methods, `steps/toolCallRequest` and `steps/toolCallResult`, as the
[Guardian README](reference-implementations/agt/packages/guardian/README.md#export-traces)
describes.
describes. A [sample trace](reference-implementations/agt/samples/trace/acs-trace-sample.jsonl)
shows that output without running anything, and [Produce a trace](reference-implementations/agt/README.md#produce-a-trace)
walks you from a fresh clone to a trace file of your own.

That exporter does not make the reference ACS-Trace conformant, and this tree claims no
Trace profile. It covers two methods, emits no OCSF, and leaves out a required attribute
Expand Down
146 changes: 146 additions & 0 deletions docs/spec/trace/extend_opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,152 @@ A typical agent session produces a tree of spans:

This hierarchy is deterministic — implementations that follow the mapping produce comparable traces across vendors and Guardians.

## Example: a denied tool call

This is one span the reference Guardian exported, a `steps/toolCallRequest` the stock policy denied, copied from the committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl). Every value in it is synthetic. It's the OTLP/JSON shape the exporter writes, so attribute values are typed (`stringValue`, `arrayValue`).

The span carries the join pair (`acs.session.id` and `acs.request_id`), the tool name, and the two Provenance columns. `acs.provenance.source_id` has `""` where the second object has no `source_id`. The decision rides on the span as the `acs.decision` event, with the verdict, the reasoning, the reason codes and the three policy arrays. `acs.policy.versions` is `[""]`, because this Guardian doesn't send a policy version.

```json
{
"traceId": "85c4878c9bda9e5b4f2fc6aae242d31f",
"spanId": "dea6530ad1681c0e",
"name": "gen_ai.tool.call",
"kind": 1,
"startTimeUnixNano": "1791928279698000000",
"endTimeUnixNano": "1791928279749301083",
"attributes": [
{
"key": "acs.session.id",
"value": {
"stringValue": "5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"
}
},
{
"key": "acs.request_id",
"value": {
"stringValue": "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03"
}
},
{
"key": "gen_ai.tool.name",
"value": {
"stringValue": "Bash"
}
},
{
"key": "acs.provenance.origin",
"value": {
"arrayValue": {
"values": [
{
"stringValue": "user_input"
},
{
"stringValue": "agent_generated"
}
]
}
}
},
{
"key": "acs.provenance.source_id",
"value": {
"arrayValue": {
"values": [
{
"stringValue": "sample-user-1"
},
{
"stringValue": ""
}
]
}
}
}
],
"droppedAttributesCount": 0,
"events": [
{
"attributes": [
{
"key": "acs.decision",
"value": {
"stringValue": "deny"
}
},
{
"key": "acs.reasoning",
"value": {
"stringValue": "This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."
}
},
{
"key": "acs.reason_codes",
"value": {
"arrayValue": {
"values": [
{
"stringValue": "destructive_shell_command_blocked"
}
]
}
}
},
{
"key": "acs.policy.ids",
"value": {
"arrayValue": {
"values": [
{
"stringValue": "agt_stock"
}
]
}
}
},
{
"key": "acs.policy.versions",
"value": {
"arrayValue": {
"values": [
{
"stringValue": ""
}
]
}
}
},
{
"key": "acs.policy.rule_ids",
"value": {
"arrayValue": {
"values": [
{
"stringValue": "destructive_shell_command_blocked"
}
]
}
}
}
],
"name": "acs.decision",
"timeUnixNano": "1791928279749295500",
"droppedAttributesCount": 0
}
],
"droppedEventsCount": 0,
"status": {
"code": 0
},
"links": [],
"droppedLinksCount": 0,
"flags": 257
}
```

To produce a trace like this yourself, follow [Produce a trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/README.md#produce-a-trace) in the reference implementation's README.

## Provenance attributes

When a hook payload carries Provenance, the resulting span MUST carry `acs.provenance.origin` and SHOULD carry `acs.provenance.source_id` when populated. The set of provenance ids cited by a Guardian decision (the response envelope's `cited_provenance_ids`) MAY be expressed as OTel span links keyed by `provenance_id`, letting tools that index span links reconstruct the lineage graph without parsing payloads.
Expand Down
105 changes: 105 additions & 0 deletions reference-implementations/agt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,111 @@ echo '{"session_id":"demo","hook_event_name":"PreToolUse","tool_name":"Bash","to

The shim never runs the command. It only asks the Guardian.

## Produce a trace

This walks you from a fresh clone to an OpenTelemetry trace file of your own, in about ten minutes. You don't need Claude Code or a collector for it. Every command runs from `reference-implementations/agt` unless the step says otherwise.

If you only want to see what the output looks like, open [`samples/trace/acs-trace-sample.jsonl`](samples/trace/acs-trace-sample.jsonl). It's a committed run of the same exporter.

**1. Get the code.** You need `git` and [Bun](https://bun.sh) 1.3 (CI runs 1.3.11). If someone gave you a commit to use, check it out, so your results match theirs.

```bash
git clone https://github.com/GenAI-Security-Project/agent-control-standard.git
cd agent-control-standard
git checkout <commit> # skip this line to use the latest integration
cd reference-implementations/agt
bun install --frozen-lockfile
mkdir -p .acs && chmod 700 .acs # owner-only, even if .acs already existed
export NO_PROXY=127.0.0.1,localhost no_proxy=127.0.0.1,localhost # keep local traffic off any HTTP_PROXY
```

Run every later step in a terminal where you've run the `export` line. Without it, an `HTTP_PROXY` in your shell sends the shim's requests and the Guardian's spans to the proxy.

**2. Start the Guardian with trace export on.** Leave it running in its own terminal.

```bash
ACS_OTEL_FILE=.acs/otel-traces.jsonl bun run guardian
```

**3. Send it some steps.** In a second terminal, pipe hook payloads through the Claude Code shim. The shim asks the Guardian and prints its answer, and it never runs the command. These three give you an allowed call, a denied call and a tool result.

```bash
echo '{"session_id":"trace-demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"ls -la"}}' \
| bun run hosts/claude-code/acs-hook.ts
echo '{"session_id":"trace-demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"rm -rf /"}}' \
| bun run hosts/claude-code/acs-hook.ts
echo '{"session_id":"trace-demo","hook_event_name":"PostToolUse","tool_name":"Bash","tool_input":{"command":"ls -la"},"tool_response":{"stdout":"README.md"}}' \
| bun run hosts/claude-code/acs-hook.ts
```

Use made-up session ids and commands, because the trace records what you send.

**4. Stop the Guardian** with Ctrl-C in its terminal. Spans are written in batches, and stopping the Guardian writes the last one.

**5. Read the trace.** `.acs/otel-traces.jsonl` holds one OTLP export request per line. This prints one line per span: its name, then the decision, or the failure class when no decision arrived.

```bash
bun -e 'const clean = (s) => String(s ?? "").replace(/[\u0000-\u001f\u007f-\u009f]/g, "?"); let bad = 0; for (const l of (await Bun.file(process.argv[1]).text()).split("\n").filter(Boolean)) { try { const req = JSON.parse(l); for (const r of req.resourceSpans) for (const s of r.scopeSpans) for (const p of s.spans) { const v = (e, k) => e.attributes?.find((a) => a.key === k)?.value.stringValue; console.log(clean(p.name), (p.events ?? []).map((e) => clean(v(e, "acs.decision") ?? v(e, "acs.failure.class"))).join(" ")) } } catch { bad++ } } if (bad) console.log(`${bad} line(s) were not readable and were skipped`)' .acs/otel-traces.jsonl
```

To read the collector's file in step 7, run the same command with `.acs/otel-collector-traces.jsonl` at the end.

You should see two `gen_ai.tool.call` spans, one `allow` and one `deny`, and one `gen_ai.tool.result` span with `allow`. The spans carry the attributes [`trace/otel-mapping.json`](../../specification/v0.1.0/trace/otel-mapping.json) lists, and the [Guardian README](packages/guardian/README.md#export-traces) says which ones appear when.

**6. Capture the failures only the host sees.** With the Guardian stopped, send one more step. The shim can't reach the Guardian, so it proceeds under the posture the Guardian declared when the session started, which is `proceed` for the stock Guardian, and records that in `.acs/audit.jsonl`. The converter turns that record into a span in the same format.

```bash
echo '{"session_id":"trace-demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"ls"}}' \
| bun run hosts/claude-code/acs-hook.ts
bun run packages/guardian/src/audit-to-otlp.ts .acs/audit.jsonl .acs/otel-audit.jsonl
```

The converter prints how many spans it wrote. Its spans carry the same session and request ids as the Guardian's, so you can read both files as one trace.

Run the converter after the agent session has ended. It refuses to write to a file that already exists, so it can't append into the audit log or the Guardian's trace. Delete `.acs/otel-audit.jsonl` first if you're running it again. If its summary shows a non-zero replay count, those were steps that proceeded without a decision that this trace doesn't show yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)).

**7. Check it with a real collector (optional).** If you want an independent check that the output is valid OTLP, install [otelcol-contrib](https://github.com/open-telemetry/opentelemetry-collector-releases/releases) 0.162.0, which the repository was tested with, and send the spans to it, not to a file. Check that nothing else is listening first: `lsof -nP -iTCP:4318 -sTCP:LISTEN` should print nothing.

```bash
umask 077
otelcol-contrib --config otel-collector.yaml # wait for "Everything is ready"
NO_PROXY=127.0.0.1,localhost no_proxy=127.0.0.1,localhost \
ACS_OTEL_ENDPOINT=http://127.0.0.1:4318/v1/traces bun run guardian # in another terminal
```

To run the collector in Docker instead, from `reference-implementations/agt`:

```bash
docker run --rm --user "$(id -u):$(id -g)" -p 127.0.0.1:4318:4318 \
-v "$PWD/otel-collector.yaml":/etc/otelcol-contrib/config.yaml:ro \
-v "$PWD/.acs":/work/.acs -w /work \
otel/opentelemetry-collector-contrib:0.162.0@sha256:39923a8e431bd1f57be82411999d389fcfe40857492e4365456d97a4c1f74be6 \
--config /etc/otelcol-contrib/config.yaml \
--set=receivers::otlp::protocols::http::endpoint=0.0.0.0:4318
```

With Podman, use the same command with `podman` in place of `docker`, plus `--userns=keep-id --umask 077`, and `:z` on both mounts on an SELinux host.

Start the collector before the Guardian. The Guardian stops sending to an endpoint after its first failed send, and prints `disabled after failure` once. If you see that, or you restart the collector, stop the Guardian, restart the collector, then start the Guardian again.

Repeat step 3. The collector writes what it received to `.acs/otel-collector-traces.jsonl`. That file can still be empty right after you repeat step 3, because spans go out in batches about every five seconds.

Stop the collector with Ctrl-C or `docker stop`, never `kill -9` or `docker kill`. A hard kill can leave half a line, and the next batch the restarted collector writes is joined onto it. The step 5 reader skips that combined line and tells you, but the spans in it are lost.

The collector appends across restarts. Natively under `umask 077` the file is `0600`. In a container it's `0644`, and the `0700` `.acs` directory is what keeps other users out ([issue #71](https://github.com/GenAI-Security-Project/agent-control-standard/issues/71) tracks file modes).

The `export NO_PROXY` line from step 1 keeps the spans off any `HTTP_PROXY`. The Guardian also adds a loopback endpoint to `NO_PROXY` itself. `OTEL_EXPORTER_OTLP_*HEADERS` variables still add headers to every send, so unset them if you don't mean them.

If `.acs/otel-collector-traces.jsonl` doesn't appear within about ten seconds of step 3, the collector isn't running this config.

Verified on macOS (arm64): natively with otelcol-contrib 0.162.0, and with the Docker command above on Docker Desktop for Mac. Rootless Docker, Podman, SELinux hosts and musl-based Linux haven't been verified.

**Run it again.** Every output here except the converter's appends, so a second run on the same `.acs` doubles every count, and the converter refuses to run at all. Delete it first: `rm -rf .acs && mkdir -p .acs && chmod 700 .acs`.

**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability` and `acs.request_id_ref`, because the shim doesn't send them yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). The committed sample drives the Guardian directly, so it has both. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229).

The trace files, the audit log and the collector's port can be written by any process running as you, the observed agent included, and in a container by other containers on the same Docker network, so the output isn't tamper-evident. `service.name` (`acs-reference-guardian`, `acs-reference-audit-converter`) is a label each writer sets for itself, not authentication, and loopback isn't authentication either. These labels belong to the reference implementation, not to the ACS specification.

## View the contract

The contract has four layers in this tree. Read them in this order.
Expand Down
43 changes: 34 additions & 9 deletions reference-implementations/agt/otel-collector.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,30 @@
# An OpenTelemetry Collector that receives the Guardian's spans and writes
# them as OTLP/JSON, one export request per line -- the same shape
# ACS_OTEL_FILE writes directly. Run it with the contrib distribution, which
# is where the `file` exporter lives:
# otelcol-contrib --config otel-collector.yaml
# then start the Guardian with ACS_OTEL_ENDPOINT=http://127.0.0.1:4318/v1/traces
# Tested with otelcol-contrib 0.162.0. In a container, run it as your own
# user (--user "$(id -u):$(id -g)") so it can write into .acs/.
# them as OTLP/JSON, one export request per line, the same shape
# ACS_OTEL_FILE writes directly. Tested with otelcol-contrib 0.162.0, which
# is where the `file` exporter lives. The walkthrough in README.md
# ("Produce a trace", step 7) has the full sequence. In short:
#
# Native:
# otelcol-contrib --config otel-collector.yaml
#
# Docker (run from reference-implementations/agt, after
# mkdir -p .acs && chmod 700 .acs):
# docker run --rm --user "$(id -u):$(id -g)" -p 127.0.0.1:4318:4318 \
# -v "$PWD/otel-collector.yaml":/etc/otelcol-contrib/config.yaml:ro \
# -v "$PWD/.acs":/work/.acs -w /work \
# otel/opentelemetry-collector-contrib:0.162.0@sha256:39923a8e431bd1f57be82411999d389fcfe40857492e4365456d97a4c1f74be6 \
# --config /etc/otelcol-contrib/config.yaml \
# --set=receivers::otlp::protocols::http::endpoint=0.0.0.0:4318
#
# Podman: the same, plus --userns=keep-id --umask 077, and :z on both
# mounts on an SELinux host.
#
# The endpoint below is loopback and literal on purpose. Inside a container
# the receiver must listen on 0.0.0.0, and that override lives only on the
# container command line above, where -p 127.0.0.1:... keeps the host side
# on loopback. Never set it for a native run: it listens on every interface.
# Loopback is not authentication: any local process can send to this port,
# and in a container so can other containers on the same Docker network.
receivers:
otlp:
protocols:
Expand All @@ -15,16 +34,22 @@ receivers:
exporters:
file:
path: .acs/otel-collector-traces.jsonl
# A fresh checkout has no .acs/ yet, and without this the exporter
# refuses to start.
# Keep what earlier runs wrote: without this a restart truncates the file.
append: true
# A fresh checkout has no .acs/ yet. 0700 applies only when the collector
# creates it. The walkthrough creates it first and sets 0700 itself.
create_directory: true
directory_permissions: "0700"
# basic, not detailed: detailed prints every attribute, reasoning text
# and provenance source ids included, to a stdout that often ends up in a
# log system nobody planned for.
debug:
verbosity: basic

service:
telemetry:
metrics:
level: none
pipelines:
traces:
receivers: [otlp]
Expand Down
3 changes: 2 additions & 1 deletion reference-implementations/agt/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
"conformance": "bash scripts/run-conformance.sh",
"watch:upstream": "bash scripts/run-upstream-watch.sh",
"guardian": "bun run packages/guardian/src/main.ts",
"inspector": "bun run packages/inspector/src/main.ts"
"inspector": "bun run packages/inspector/src/main.ts",
"sample:trace": "bun run scripts/make-sample-trace.ts"
},
"dependencies": {
"agent-control-specification": "0.3.1-beta.0"
Expand Down
Loading
Loading