Repository navigation
Export Guardian traces as OTLP to a file and over HTTP - #244
rocklambros merged 22 commits into
Conversation
Six packages, each at an exact version: @opentelemetry/api 1.9.1, core 2.12.0, resources 2.12.0, sdk-trace-base 2.12.0, otlp-transformer 0.223.0 and exporter-trace-otlp-http 0.223.0. Exact rather than a range for the same reason ajv is: CI installs from the lockfile and a floating range would let the installed SDK drift from the one the exporter was written against. sdk-trace-base 2.12.0 is a shim over @opentelemetry/sdk-trace, which the lockfile therefore carries too. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
One OpenTelemetry span per step the Guardian decides, so a session can be rebuilt offline from a trace file. Span names and attribute lists come from specification/v0.1.0/trace/otel-mapping.json at startup; the one table the mapping cannot supply, which envelope field feeds which attribute, lives in otel-trace-exporter.ts. The span opens when the request is recorded and closes when the response is, at the envelope log's own two call sites, paired by JSON-RPC id. The decision rides on it as the mapping's acs.decision event, with acs.evaluator set to deterministic and the reason codes, policy references, cited provenance ids and chain hash added only when the response carries them. Two outputs, both off by default: ACS_OTEL_FILE writes OTLP/JSON through the official serializer, one export request per line, and ACS_OTEL_ENDPOINT sends to an OTLP/HTTP collector. With neither set the Guardian is handed a null exporter and behaves as before, banner included. Spans go through a BatchSpanProcessor and are flushed when the Guardian closes, which main.ts now does on SIGINT and SIGTERM when export is on. Total the way the envelope log sink is: any failure disables export for the process, reports once, and the response goes out unchanged. The tests drive both a failing exporter and an unwritable file through a real Guardian to hold that line. METHODS_EVALUATED is exported from handshake.ts so the exporter emits spans only for methods this Guardian dispatches, without a second copy of that list. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
The Guardian README gains the two variables, a section on what a span and its decision event carry, where acs.chain_hash does and does not appear, and the note that lab runs should use synthetic sessions because raw envelopes reach disk before any redaction. otel-collector.yaml at the tree root receives the endpoint output and writes the same OTLP/JSON file shape, for a deployment that prefers a Collector in the path. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
The exporter kept open spans in a map keyed by JSON-RPC id and closed the first one it found when a response arrived. A JSON-RPC id is the client's: hosts run hooks in parallel and number requests per session, so two sessions can be in flight with the same id, and a lookup keyed on it could close one session's span with the other's decision. `start` now returns the span it opened and `end` takes that handle back. handleAcsRequest holds it between the envelope log's two writes, so a span can only close on its own response and a request with no id at all still gets one. The map, its queues and the id extraction are gone. A test drives two sessions through a real Guardian concurrently with the same id, one denied and one allowed, and checks each decision landed on its own span. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
otel-mapping.json says a span whose payload has a Provenance object attached must carry acs.provenance.origin, and may carry acs.provenance.source_id and acs.provenance.lineage_depth. A payload can carry several such objects, one per argument at the request gate and one per output at the result gate, and a span has one value per attribute, so each attribute becomes a column: an array with one element per Provenance object, in wire order, the i-th element describing the i-th object. A column is emitted only when every object carries the field, the rule the policy columns already follow, so columns line up by position and no placeholder is invented. The names come from the mapping's own provenance_attributes block. acs.provenance.lineage_depth is not emitted. provenance.json defines no such field, so there is nothing on the wire to read it from; the source table says so where the entry would be. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
|
This pull request's description says it closes #91, which is on the project roadmap. When it merges, GitHub closes that issue and the roadmap counts it as delivered. Please confirm the change delivers the whole issue before merging. |
…le asynchronously Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
|
Hi @sabasiddique1, thank you for this. I read the whole diff, ran it, and had a few reviewers attack it from different sides. The core design is good. Pairing the request and the response by passing the span handle is the right call. I checked that your two-sessions-same-id test really fails if someone pairs by JSON-RPC id instead. I also ran the file through the collector the way you did (otelcol-contrib 0.162.0, There are changes I need before I can approve. I won't push anything to your branch. I'd like you to make these changes yourself, so I wrote this comment as a step-by-step guide. Each task says why it matters, what to change (with file, line, and code where it helps), and how to test it. If you use an AI coding assistant, the last section has a prompt you can paste in as it is. If anything is unclear, ask me here and I'll answer. Line numbers below refer to your current head, Task 1. Policy and provenance arrays: same length,
|
failure.kind in the audit log |
acs.failure.class |
|---|---|
timeout |
timeout |
transport |
unreachable |
error_without_decision |
guardian_error |
refused |
guardian_error |
host_configuration |
observed_agent_error |
decision_unrenderable |
observed_agent_error |
unknown, or anything else |
other |
The kinds are defined in packages/host-adapter/src/failure-kinds.ts. Please read the comments there and tell me in the PR if you think any row is wrong.
- It skips entries whose
outcomeis"ungoverned". Those are steps no gate asked about, not decision failures, so Bar's classes don't cover them. Please say this in the README. - It flushes and shuts down the provider at the end, so every span is written.
The audit entries don't carry elapsed time, timeout or payload size, so leave those attributes out. They are "when known" attributes.
Running it. Please tell people in the README to run it after the Guardian has stopped. If both write to the same file at the same time, the lines could mix. Running it twice on the same audit log writes the spans twice, so the README should say that too.
Test (with the Guardian down). Please put this test in reference-implementations/agt/test/, because it uses two packages.
- Drive one hook through the host adapter with the Guardian URL pointing at a closed port, so the host writes a
transportaudit entry.packages/host-adapter/test/govern-step.test.tsalready has this kind of setup, andtest/audit-sink-roundtrip.test.tsshows how audit entries look. - Run the converter on that audit log.
- Check that the OTLP file has one span with an
acs.decision.failureevent whereacs.failure.class = "unreachable",acs.failure.reported_by = "observed_agent", andacs.failure.posture = "fail_open".
Task 7. Shutdown: wait for the first call, and don't wait too long
Why.
shutdown()returns at once on a second call (line 560), instead of waiting for the first one. So SIGINT followed by SIGTERM can exit in the middle of a flush.- With a dead endpoint, shutdown took about 9 seconds in my run, because the OTLP exporter retries for up to 10 seconds by default.
Change.
- Store the shutdown promise and return it on every call:
let shutdownPromise: Promise<void> | undefined;
// ...
shutdown(): Promise<void> {
shutdownPromise ??= (async () => {
if (provider === undefined) return;
stopped = true;
// close open spans (Task 5f), then:
try {
await provider.shutdown();
} catch (error) {
fail(error);
}
})();
return shutdownPromise;
},- Create the HTTP exporter with a short timeout:
new OTLPTraceExporter({ url: endpoint, timeoutMillis: 2000 }).
Task 8. A test that every span name comes from the mapping
Why. In #91 I asked for "a test that every emitted span name is a key in the mapping". I can't find it. The tests compare against two specific entries, which is not the same thing. It matters soon. #229 adds protocols/MCP/tools/call to the evaluated methods, and the mapping has no entry for it, so those tool calls would get no span and nothing would report it.
Change.
- In
otel-trace-exporter.ts, export a small list of methods that are evaluated on purpose without a span, each with a reason. Start it empty:
/** Methods this Guardian evaluates that have no step_to_span entry in
* otel-mapping.json yet, and why. A method here gets no span. The test
* fails for an evaluated method that is neither mapped nor listed here. */
export const METHODS_WITHOUT_SPAN: Readonly<Record<string, string>> = {};- In
begin, nothing changes. A method with no mapping entry already returnsnull.
Tests.
- For every method in
buildServerHello().methods_evaluated, eitherMAPPING.stepToSpan[method]is defined orMETHODS_WITHOUT_SPAN[method]is defined. - In the wiring test, every emitted span name is one of the
span_namevalues instep_to_span.
I'm asking #229 to add its MCP method to METHODS_WITHOUT_SPAN with a reason when the two PRs meet.
Task 9. Make the docs match the code
Guardian README (packages/guardian/README.md). Please replace these sentences.
-
Line 59, "It carries the attributes the mapping lists for it, read from the envelope," becomes:
It carries each attribute the mapping lists for it when the envelope supplies a value. A required attribute the envelope omits, such as
acs.capability([Bug]: otel-mapping.json requires acs.capability on gen_ai.tool.call, but capability is optional in tool-call-request.json #227), is absent from the span.Please add a test for this: a request with no
capabilitygives a span withoutacs.capability, and nothing breaks. -
Line 61, the sentence about
acs.chain_hashappearing after a ContextEntry was written, becomes:This Guardian does not send
chain_hash,cited_provenance_ids, orpolicy_versionon its responses yet, so those attributes are absent from its spans today. The three policy arrays always have the same length aspolicy_references, with""where a reference has no value.I checked the collector output: none of those three ever appear in a real run.
-
Line 63, "stop it with Ctrl-C rather than killing it", becomes:
Spans are flushed when the Guardian closes on SIGINT or SIGTERM. A SIGKILL loses the last batch.
-
Add a short paragraph about
acs.decision.failure: the Guardian emits it for errors, late decisions and shutdown, and the converter from Task 6 produces it from the host audit log. -
In the security paragraph, name
acs.provenance.source_idnext toacs.reasoning. Byprovenance.jsonit can be a URL or a file path, and a signed URL or a home-directory path would go out as is. -
Add one sentence:
This is an exporter for the two live methods, not an ACS-Trace conformance claim.
The lab will read this README first.
-
STYLE.md asks for 2 to 4 sentences per paragraph. Lines 59 and 61 have 6 and 5, so please split each into two paragraphs.
Top-level README.md, around line 292. It still says "No OpenTelemetry collector or exporter exists yet." Please replace that paragraph with:
Two gaps are named and open, not vague. The Guardian writes its envelope and audit logs as synchronous file appends on the decision path, with no batching. Trace export is partial: set
ACS_OTEL_FILEorACS_OTEL_ENDPOINTand the Guardian exports OTLP spans for the two live methods,steps/toolCallRequestandsteps/toolCallResult, as the Guardian README describes.That exporter does not make the reference ACS-Trace conformant, and this tree claims no Trace profile. It covers two methods, emits no OCSF, and leaves out a required attribute when the request omits it. The conformance harness still measures which attributes a consumer could emit from the wire, not what the exporter ships. Start in
reference-implementations/agt/packages/.
otel-collector.yaml. Please change the debug exporter to verbosity: basic. With detailed, every attribute (including reasoning and source_id) goes to stdout, and stdout often ends up in a log system nobody planned for.
Task 10. Fix the PR description
The squash commit message is the PR description, so these sentences become permanent history.
- "Span names and attribute names come from
otel-mapping.json" is true for the span names and the mapped attributes. It isn't true foracs.request_id, the six decision extras (acs.reason_codes, the threeacs.policy.*,acs.cited_provenance_ids,acs.chain_hash), and theacs.decision.failurenames. Those live in code until [Bug]: acs.decision trace event cannot carry reason_codes, policy_references, cited_provenance_ids, or chain_hash, so replay from the trace alone is not possible #228 adds them to the mapping. Please say which names come from where. - The
closing-choicecheck fails because "([Bug] §6.4's mandatory fail-open audit event has no conformant shape, so the decision-failure rate is unmeasurable #37)" sits in the issue section without a keyword. When Tasks 5 and 6 are done, please delete the "Still to come" line and add#37to theRefsline, so it readsRefs #37, #228. Please don't useCloses #37. [Bug] §6.4's mandatory fail-open audit event has no conformant shape, so the decision-failure rate is unmeasurable #37 also covers the spec text and thesessionEndcount, which this PR doesn't do. - Please describe the new
acs.decision.failurework and the converter in "What changed".
Your five questions
Q1. Should the Guardian set metadata.evaluator on the response?
Yes, but not in this PR, because #229 owns acs-result.ts. I've asked for the one-line change on #229. Until it lands, keep your deterministic fallback, and add one sentence to the README saying the trace has an evaluator that the envelope log doesn't. When the wire carries it, delete DEFAULT_EVALUATOR.
Q2. Is acs.request_id fine, or should it go into the mapping?
Keep the name, and it should go into the mapping. Both envelopes require request_id, and acs.request_id_ref on the result span has nothing to join to without it. I'll add it to the mapping in my #228 PR. You don't need to change anything.
Q3. lineage_depth: drop it, or define how to compute it?
I'll drop it in my #227 PR. provenance.json has no such field, and depth is a property of the whole derived_from graph, not of one object. Defining a graph algorithm in a patch release is the wrong move. It appears in four places, and I'll remove it from all of them: otel-mapping.json (the description at line 121 and the list at line 131), ocsf-mapping.json:78, events.md:41, and extend_opentelemetry.md:20. Your exporter already does the right thing by not emitting it.
Q4. Are positional provenance arrays enough, or do we need argument names?
Positional is fine for the lab once the README says it plainly, and Task 1 keeps the positions lined up. Argument names aren't the missing piece. provenance_id is. The decision event can carry acs.cited_provenance_ids, but no span carries provenance ids, so a reader can't join a cited id to its origin and source_id. provenance_id is required in provenance.json, so an acs.provenance.ids column would always be complete. That needs a name from the Spec leads, so it's a follow-up and doesn't block this PR.
Q5. Does the trace-pillar.ts update go in #228 or here?
It goes in #228, and you don't carry it. decisionAttributeSite in trace-pillar.ts:444 throws on any decision attribute it doesn't know. So the PR that adds names to the mapping has to update the checker in the same commit, or integration breaks. There's also a CI reason. The reference implementation job only runs on reference-implementations/**, so a spec-only #228 PR would pass CI and break things later. Touching trace-pillar.ts in #228 makes CI run.
Merge notes
- Dependency age. Five of the six OpenTelemetry pins were published on 2026-10-06. Our
dependabot.ymlwaits 7 days before taking a new release, so I won't merge before 2026-10-13. That still fits the 16th, and you don't need to change the pins. Give the AGT reference implementation Dependabot coverage #126 (Dependabot for this tree) is still open, so nothing watches these packages yet. That's on me. - Batch Guardian log writes and drain them on shutdown #112. It also adds SIGINT/SIGTERM handlers and a draining
close(). Whichever PR lands second should keep one signal handler and oneclose()that drains the logs and then callstraceExporter.shutdown(). I left the same note on Batch Guardian log writes and drain them on shutdown #112. - fix(agt): implement ACS-Core wire safeguards #229. With Task 4, you no longer touch
handshake.ts.server.tsandmain.tswill still conflict at thestartGuardiancall and inhandleAcsRequest. Whoever lands second resolves it, and I'm happy to help. - After approval. I'll mark the PR ready, squash it with
Closes #91, and add the sign-offs on the squash commit. Your commits already carrySigned-off-by, so DCO is fine.
If you use an AI coding assistant
Paste this in as it is. It points the assistant at this comment, so you don't have to retype anything.
I am working on PR #244 in GenAI-Security-Project/agent-control-standard
(branch refimpl/otel-exporter). The maintainer left a review comment with
Tasks 1-10. Read it first:
gh pr view 244 --comments
Then do the tasks in order, in reference-implementations/agt/. Rules:
- Make one commit per task, signed off with `git commit -s`. Do not add
any AI co-author or "generated by" lines.
- Follow the code in the comment, but read the surrounding code first and
match its style, its comment density and its "total by construction"
rule: nothing in the exporter may throw into, or wait on, the decision
path.
- Do not edit packages/guardian/src/acs-result.ts,
packages/guardian/src/handshake.ts (Task 4 reverts it to the base), or
anything under packages/host-adapter/src/.
- After each task run, from reference-implementations/agt:
bun install --frozen-lockfile
bun run typecheck
bun test
and fix anything that fails before moving on.
- From the repository root, also run `uv run pytest -v` and
`uv run mkdocs build --strict`.
- At the end, show me `git diff origin/integration...HEAD --stat` and a
one-line summary per task, and list anything you could not do.
Thanks again. This is close, and the lab will get a file they can use.
|
One more thing I found after posting, @sabasiddique1. It's small, but please add it to Task 8. On #227, Bar and Maurits decided that each The fix is two lines. Add collect(attributes, [...entry.required_attributes, ...(entry.optional_attributes ?? []), ...(entry.conditional_attributes ?? [])], STEP_ATTRIBUTE_SOURCES, parts);It does nothing today, and it keeps working after #227. Please add a unit test that uses a small mapping file (your |
The HMAC signing block in handleAcsRequest ran outside any try, on unauthenticated input. A session_id over 1024 bytes makes HKDF throw, and a deeply nested body overflows the stack in JCS canonicalization. Either escaped handleAcsRequest as an HTML 500, which a host reads as a hook that never fired. The block is now wrapped. A request that throws there cannot carry a valid signature, so its response goes out unsigned, like any other unauthenticated request's, and the failure is reported on stderr. The envelope log records the same response it did before, minus the signature it could never have had. Every AcsResult now carries metadata.evaluator "deterministic", so the trace exporter in GenAI-Security-Project#244 reads the evaluator from the wire instead of assuming it. Both raised by @rocklambros on GenAI-Security-Project#229. Signed-off-by: Jason Crittenden <jrcrittenden@gmail.com>
|
@sabasiddique1, a short update. The spec PR for #227 is now open as #261, and the one for #228 is ready on a branch. Checking them against your exporter turned up four things. None of them is new work beyond the review, but two of them change what I asked earlier, so I'm writing them down.
The merge order note on #261 explains how to check, before merging, that the exporter reads the new list. |
|
@sabasiddique1, one decision to settle item 3 of my earlier comment today, so you aren't guessing about the evaluator. Please remove the The Guardian side is already done. #229 ( What this means for your PR:
On order: #261 will most likely merge before your PR, so please do the |
…-exporter Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
A column was dropped whenever one reference or one Provenance object lacked the field, so a single reference without a version hid the versions every other reference carried, and one argument without a source_id hid the source_ids of all the others. The rule agreed on GenAI-Security-Project#228 is the opposite: the three policy arrays are always the length of policy_references, in its order, with "" where a value is missing, and acs.provenance.source_id is padded to the length of acs.provenance.origin. A reader lines the columns up by position and loses nothing that was sent. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
Each output already fails on its own: a collector that is down disables the endpoint and nothing else, and the file keeps every span. What was unbounded was the wait. A refused connection is retryable to the OTLP exporter, so it kept retrying for its 10 s default before reporting, and a flush or a shutdown behind a dead collector held the process that long. The endpoint now gives up after 2 s. The test drives the real OTLPTraceExporter against port 1 through a Guardian: decisions match a Guardian with export off before and after the endpoint dies, the file holds the spans sent after it died, the endpoint is reported once, and both the flush that reports it and the close that follows finish within a bound. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
The file holds reasoning text and provenance source ids, which by provenance.json can be a URL or a file path, so other local users should not read it: GenAI-Security-Project#71 is the same problem in the envelope log. The exporter now creates its directory 0700 and the file 0600. The append itself was already asynchronous and chained, so a blocked file cannot hold the event loop that answers decisions; a test now pins that two exports handed over back to back land as two lines in export order. Signed-off-by: Saba Siddique <sabasiddiqdev@gmail.com>
|
@sabasiddique1, good catch on both rows, and the second one exposed a mistake in my own Task 6 table, so we've got it from here.
I'm running the full review now and will push the fixes to your branch, with a note here on what changed. |
… into traces - Span limits and an always-on sampler are set explicitly, so a large request can't grow the export queue without bound and no OTEL_* environment variable can drop spans or decision events. - Provenance columns stop at 64 objects, both cut at the same index. - A JSON-RPC error carries only its code. The message can quote the request, and GenAI-Security-Project#228 forbids copying error bytes into the trace. - REPLAY_DETECTED gets no failure event, because the pair belongs to the earlier request (GenAI-Security-Project#228). - acs.provenance.source_id is emitted only when some object has one. - Shutdown is bounded, and a stalled output is reported, not waited on. - The decision extras are emitted whether or not the mapping lists them, so this and GenAI-Security-Project#228 can land in either order. Signed-off-by: rocklambros <rock@rockcyber.com>
…ctly - Spans carry the derived session UUID and the request id, the same pair the Guardian's spans carry, so the two files can be joined. - Posture comes from the entry's own posture field. Deriving it from the outcome reported a refusal under proceed as fail_closed. - The detail is a fixed sentence per failure kind. The host's message quotes the Guardian's error text, which GenAI-Security-Project#228 forbids copying. The error code is parsed out as an integer, and nothing else is. - decision_unrenderable is other, because a decision did arrive. - One span per export. The batch queue overflowed in the synchronous loop, so 5000 entries wrote 2560 spans while the summary claimed 5000. The summary now reports what reached the file, and the command exits non-zero when that differs. - Entries with no method are counted by kind, apart from unmapped methods. Replays, and entries with no posture, are skipped and counted. - A host-adapter test pins the request id being the JSON-RPC id, which the join depends on. Signed-off-by: rocklambros <rock@rockcyber.com>
…eate its directory The Guardian README now says when source_id is left out, that values stop at 4096 characters and provenance at 64 objects, that the environment can't change sampling, that errors carry a code and no text, and how shutdown is bounded. The converter section covers the join pair, what posture means on a refused entry, the fixed detail, and every count. otel-collector.yaml creates .acs/, without which the file exporter refused to start on a fresh checkout. Signed-off-by: rocklambros <rock@rockcyber.com>
The Guardian and the audit converter worded the same failure differently, so a query on status text found only one side. The README no longer promises acs.request_id on a span whose envelope has none. Signed-off-by: rocklambros <rock@rockcyber.com>
|
@sabasiddique1, I pushed four commits on top of yours, Here's what each one does and why:
All of this is validated end to end: the collector parsed every span live, decisions with a dead endpoint match an export-off run, and |
|
@afogel @bar-capsule, this is ready for your review, and it's the last piece the trace work needs this week. Saba built the exporter and the audit-log converter. I pushed four fixes on top after a review, and the comment above lists each one with its reason. The parts worth your eyes are the converter's class table ( |
8ab7e1e
into
GenAI-Security-Project:integration
The HMAC signing block in handleAcsRequest ran outside any try, on unauthenticated input. A session_id over 1024 bytes makes HKDF throw, and a deeply nested body overflows the stack in JCS canonicalization. Either escaped handleAcsRequest as an HTML 500, which a host reads as a hook that never fired. The block is now wrapped. A request that throws there cannot carry a valid signature, so its response goes out unsigned, like any other unauthenticated request's, and the failure is reported on stderr. The envelope log records the same response it did before, minus the signature it could never have had. Every AcsResult now carries metadata.evaluator "deterministic", so the trace exporter in GenAI-Security-Project#244 reads the evaluator from the wire instead of assuming it. Both raised by @rocklambros on GenAI-Security-Project#229. Signed-off-by: Jason Crittenden <jrcrittenden@gmail.com>
…he MCP method GenAI-Security-Project#244's wiring tests and the sample-trace generator posted steps with no handshake/hello, which GenAI-Security-Project#229's Guardian refuses with -32003. postRaw and the generator now send one first; the hello gets no span. protocols/MCP/tools/call is evaluated but otel-mapping.json maps only steps/* and agbom/* methods, so it is named in METHODS_WITHOUT_SPAN with that reason until the spec gives it a step_to_span entry. The decision event now carries acs.evaluator, and the response carries a chain_hash that covers each entry's recorded time. The two tests that compare a Guardian with export on and off give both the same clock, and the sample comparison sets the hash's value aside with the timestamps. The committed sample is regenerated. Signed-off-by: Jason Crittenden <jrcrittenden@gmail.com>
The HMAC signing block in handleAcsRequest ran outside any try, on unauthenticated input. A session_id over 1024 bytes makes HKDF throw, and a deeply nested body overflows the stack in JCS canonicalization. Either escaped handleAcsRequest as an HTML 500, which a host reads as a hook that never fired. The block is now wrapped. A request that throws there cannot carry a valid signature, so its response goes out unsigned, like any other unauthenticated request's, and the failure is reported on stderr. The envelope log records the same response it did before, minus the signature it could never have had. Every AcsResult now carries metadata.evaluator "deterministic", so the trace exporter in GenAI-Security-Project#244 reads the evaluator from the wire instead of assuming it. Both raised by @rocklambros on GenAI-Security-Project#229. Signed-off-by: Jason Crittenden <jrcrittenden@gmail.com>
What changed
The reference Guardian exports a span for every step it decides, as OTLP/JSON to a file (
ACS_OTEL_FILE) and over OTLP/HTTP (ACS_OTEL_ENDPOINT). Both are off by default, and with neither set the Guardian behaves exactly as before. The exporter sits next to the envelope log sink inserver.tsand follows the same rule: the decision path never waits on it or sees an error from it. Each output fails on its own, so a collector that is down costs the endpoint and nothing else. The file keeps every span, which is what the lab reads.Which names come from where. Span names, the attributes listed per span (
required_attributes,conditional_attributes,optional_attributes), theacs.decisionevent name and its two attribute lists, and the provenance attribute names are read fromotel-mapping.jsonat startup. A test checks, against the real file, that every name the mapping lists for an evaluated method has a source in the exporter, so a mapping change cannot silently drop an attribute. The names that live in code until #228 adds them to the mapping areacs.request_id, the six decision extras (acs.reason_codes,acs.policy.ids,acs.policy.versions,acs.policy.rule_ids,acs.cited_provenance_ids,acs.chain_hash), and theacs.decision.failureevent with itsacs.failure.*attributes. Each set is declared in one place inotel-trace-exporter.ts.What a span carries.
gen_ai.tool.callandgen_ai.tool.resultfor the two methods this Guardian evaluates, read from the ServerHello it sends.METHODS_WITHOUT_SPANnames any evaluated method that has no mapping entry yet, and a test fails for a method that is neither mapped nor listed. Every span carriesacs.session.idandacs.request_id. Theacs.decisionevent carriesacs.decision,acs.reasoning, andacs.evaluatoronly when the response carriesmetadata.evaluator(this Guardian sends it once #229 lands; the exporter no longer fills it in). The three policy arrays are always the length ofpolicy_references, in its order, with""where a value is missing, andacs.provenance.source_idis padded to the length ofacs.provenance.originthe same way, and left out when no Provenance object has one. Both provenance columns stop at 64 objects, and every string attribute stops at 4096 characters, so one large request can't grow the export queue without bound. The sampler and span limits are set in code, so noOTEL_*environment variable can sample spans away or drop events.acs.decision.failure(#37). A step that got no usable decision gets this event on its own span, withacs.failure.class,acs.failure.posture(from the ServerHello this Guardian sends) andacs.failure.reported_by, plus the error code, the negotiated timeout, the elapsed time and the request size when known. The Guardian emits it for the three cases it can see itself:guardian_errorfor a JSON-RPC error instead of a decision,timeoutfor a decision finished after the negotiated timeout (the host had already applied its posture), andotherfor a step still open at shutdown, which it now closes rather than loses. It carries the JSON-RPC error code but never the error text, which can quote the request (#228 forbids copying it), and a replay rejection (-32005) gets no event, because its pair belongs to the earlier request. The six decision extras are emitted whether or not the mapping lists them, so this PR and #228 can land in either order.The converter.
packages/guardian/src/audit-to-otlp.tsturns the host adapter's audit log (.acs/audit.jsonl) into the same OTLP/JSON file: one span per posture entry, with anacs.decision.failureevent reported byobserved_agentand the class read off the host'sfailure.kindthrough the table below. Each span carries the sameacs.session.id(the host's derived UUID) andacs.request_idas the Guardian's span for that request, so the two files join.acs.failure.postureis the deployment's posture from the entry, the detail is a fixed sentence per kind and never the host's message, and the error code is parsed out as an integer. Ungoverned entries are skipped, since they are steps no gate asked about. Entries with no method, unmapped methods, replays and unreadable lines are skipped and counted. Spans are exported one at a time, and the command exits non-zero if fewer reach the file than it built. It reads the log with plainJSON.parseand imports nothing from the adapter. The test drives a real hook through the adapter against a closed port and converts thetransportentry it leaves into oneunreachablespan.failure.kindacs.failure.classtimeouttimeouttransportunreachableerror_without_decisionguardian_errorrefusedguardian_errorhost_configurationobserved_agent_errordecision_unrenderableotherunknown, anything elseotherWriting and shutdown. The file is appended asynchronously, created
0600in a0700directory. Spans go through aBatchSpanProcessor, the HTTP exporter gives up on a dead collector after 2 s, a secondshutdown()waits for the first, andclose()drains in-flight requests for the negotiated timeout before it shuts the exporter down, so a step whose bridge never answers still gets its span. The exporter's own shutdown is bounded at 3 s, so a stalled output can't hold the process open.How I checked the output: besides the tests, I ran the Guardian with
ACS_OTEL_FILEand read the file back with the OTel Collector'sotlpjsonfilereceiver (otelcol-contrib 0.162.0). Every span, event and attribute parsed. I also sent live traffic over OTLP/HTTP into the committedotel-collector.yamland got the same spans out.Which issue does this implement
Closes #91
Refs #37
Refs #228
Open, not blocking
provenance_idis required inprovenance.json, so anacs.provenance.idscolumn would always be complete and would let a reader joinacs.cited_provenance_idsto an origin and a source. It needs a name from the Spec leads.refusedstaysguardian_error, because §6.4 counts an error in place of a decision as a decision failure, anddecision_unrenderableis nowother, because a decision did arrive.Base branch
integration, because this touches the specification, schemas, a referenceimplementation, an adapter, tests, or CI
main, because every changed path is on the documentation lane allowlist inCONTRIBUTING.md
Type of change
I tested this
uv run pytest -vpasses on my machineuv run mkdocs build --strictpasses on my machineAlso:
bun install --frozen-lockfileclean,bun run typecheckclean,bun test1172 pass / 1 skip / 0 fail across 70 files, on Bun 1.3.14 and on 1.3.11, the version CI pins. The timing tests (dead endpoint, timeout, open-at-shutdown, second shutdown) each passed five runs in a row.Checklist
git commit -s(required by the DCO)Security
When enabled, the file and the HTTP stream carry decision reasoning text, tool names, capabilities, and provenance
source_idvalues, which byprovenance.jsoncan be URLs or file paths and go out as the host sent them. Raw tool arguments are not exported. The file is created readable by its owner only. Export is off by default, and the collector config's debug exporter prints atbasicverbosity so attributes do not reach stdout. For the lab run I would use synthetic sessions only, since #71 is still open.No new dependencies beyond the six OpenTelemetry pins already in the lockfile.