Add Braintrust instrumentation for Cloudflare Agents (`agents`, `@cloudflare/ai-chat`, `@cloudflare/think`)

[AbhiPrasad]

Summary

Add first-class Braintrust instrumentation for the Cloudflare Agents SDK and its AI agent surfaces:

• agents — base Agent, durable execution, schedules, RPC, sub-agents, agent-as-tool, and observability
• @cloudflare/ai-chat — AIChatAgent
• @cloudflare/think — Cloudflare's opinionated agent harness

As of 2026-07-17, the current stable versions are agents@0.17.4, @cloudflare/ai-chat@0.9.3, and @cloudflare/think@0.13.0; agents had about 898K npm downloads in the preceding week. This repository has no Cloudflare Agents wrapper, plugin, vendored types, auto-instrumentation config, or workerd integration test.

Instrumenting only the underlying model client is not enough. Cloudflare Agents adds the useful framework-level contract: durable agent turns, conversation/agent identity, tool and approval lifecycle, sub-agent delegation, stream completion, retries, and recovery after Durable Object eviction.

Scope: AI-relevant operations

We should trace AI execution, not every Durable Object state/storage operation.

Surface	Suggested Braintrust representation
AIChatAgent.onChatMessage() / a Think turn	Parent task span containing the triggering input, final assistant output, and aggregate token metrics; no ad hoc framework IDs/metadata
AI SDK generation inside the turn (when used)	Existing AI SDK operation/llm spans, nested under the turn; omit this layer for provider-direct agents
Server tool execution	One child tool span per model-requested execution; approvals are control flow, not separate spans unless the instrumentation specification adds them
Agent-as-tool / sub-agent invocation	A tool span for the model-requested tool execution containing a nested child-agent task span
Terminal errors/cancellation	Error/cancel status on the owning span without changing stream/error behavior
Fiber/chat recovery, retry, compaction, transcript repair	Trace only AI work that actually re-runs; omit housekeeping events and do not keep a live span across an eviction
Generic state updates, entity CRUD, connections, and schedule creation	Out of scope unless directly needed to explain an AI operation

The high-level turn should stay open until the returned response stream completes, errors, or is cancelled—not merely until onChatMessage() returns its Response.

Data contract

This integration must follow the Braintrust instrumentation guide, especially its restrictive capture policy: do not log a field merely because Cloudflare exposes it. A new field must first be allowed by the specification.

Cloudflare agent-turn task span

For AIChatAgent.onChatMessage() and a Think turn, collect only:

Span field	Value
span_attributes.type	task
span_attributes.name	A descriptive, low-cardinality operation name, preferably the concrete class plus public operation, e.g. Chat.onChatMessage
input	Only the prompt/message(s) that triggered this run, normalized to ordered OpenAI-style messages where practical. Do not duplicate the full persisted Agent/DO state or historical transcript here.
output	The final assistant response after stream aggregation, not the Response, stream, provider client, or intermediate chunks
metrics	Aggregated tokens, prompt_tokens, and completion_tokens from child LLM calls when available; normal start/end timing
error	The original Error object for a thrown/rejected/in-stream failure; do not stringify it locally
context.span_origin	name: "braintrust.sdk.javascript", the Braintrust SDK version, instrumentation.name: "cloudflare-agents", and a reliably detected server/cloudflare_workers environment when available

Do not put the full model context on this parent span. Exact system instructions, history, prior reasoning, and tool results belong in the input of the individual child llm call that actually received them.

Child AI SDK / LLM spans

The packages have different relationships to Vercel AI SDK:

• Base Agent is model/provider agnostic and does not inherently invoke AI SDK.
• AIChatAgent provides persistence, protocol, and streaming lifecycle, but the user's onChatMessage() override owns generation. Cloudflare's documented/default pattern calls AI SDK streamText(), though users can call a provider directly or use another framework.
• Think owns its agentic loop and directly calls AI SDK v6 streamText() internally.

Therefore AI SDK spans are expected only on AI SDK-backed paths. Reuse the existing AI SDK instrumentation rather than inventing another model schema; for provider-direct agents, nest the existing provider llm span directly under the Cloudflare operation. For every actual model call collect:

• type: "llm" and the existing provider/AI SDK span name.
• The exact ordered messages sent for that call, including system/developer messages, prior assistant tool calls, tool results, and prior reasoning context.
• One fully accumulated output in canonical OpenAI Chat Completions form, unless the resolved provider is Anthropic or Google and the existing provider-native UI normalizer is used. Tool calls remain in the LLM output.
• metadata.model and metadata.provider, resolved from the response when possible. provider must be the provider/gateway/reseller whose pricing applies, not simply cloudflare-agents.
• Only the allowlisted request metadata: temperature, top_p, max_tokens, frequency_penalty, presence_penalty, stop, and response_format.
• Serializable tool definitions in metadata.tools plus supplied tool_choice, parallel_tool_calls, and max_tool_calls. Never capture executable handlers, closures, or code bodies.
• Braintrust prompt provenance in metadata.prompt when a Braintrust-managed prompt was used.
• Only specification-approved metrics: tokens, prompt_tokens, completion_tokens, streaming time_to_first_token, and reported reasoning/cache/audio/image token metrics. Do not add Cloudflare-specific numeric metrics without a spec change.
• Inline media converted in-place to braintrust_attachment references. Do not retain raw base64/buffers after successful conversion and do not fetch remote URLs solely for attachment upload.

streamText/generateText may add an intermediate AI SDK task span. That extra framework span is acceptable, but leaf model calls must still be llm spans and the Cloudflare turn must remain the outer framework boundary.

Child tool spans

For each model-requested tool execution collect:

Span field	Value
span_attributes.type	tool
span_attributes.name	Exact tool/function name
input	Arguments supplied by the model after the framework's normal parsing/validation; preserve malformed input when it is the attempted execution input
output	The tool's JSON-serializable return value
error	Original Error object if execution fails

Tool schemas belong in each relevant LLM span's metadata.tools, not on the tool span. Client-side tools should be traced in the runtime where they actually execute; the Worker should not fabricate a server tool span for work performed in the browser.

For an agent exposed as a model tool, the outer execution is a tool span and the delegated agent run is a nested task span. Propagate Braintrust context over the Durable Object/RPC boundary so the child does not become an unrelated root trace.

Explicitly do not collect

Unless the instrumentation specification is amended, omit:

• Durable Object state, SQLite rows, schedules/alarms, connection objects, MCP connection state, RPC request/response dumps, and framework CRUD operations.
• Arbitrary Cloudflare observability event payloads.
• Request, connection, Durable Object instance, session, conversation, submission, fiber, incident, or run IDs as span metadata.
• Tool-call IDs as new tool-span metadata (they remain in the standardized LLM input/output payloads).
• Headers, environment bindings, secrets, provider client objects, executable handlers, stack-local closures, and raw Response/stream objects.
• Chunk counts, retry counts, queue depths, storage timings, or other metrics not listed by the instrumentation guide.

Agent instance/conversation/request IDs and recovery correlation would be useful, but the current guide does not authorize them on framework spans. If product requirements need these fields, update the spec first with their exact location, type, cardinality, and privacy semantics; do not ship ad hoc metadata names.

Example trace diagrams

These are the expected logical Braintrust trees for the common AI SDK-backed AIChatAgent/Think paths. streamText is not intrinsic to base Agent, and AIChatAgent users can choose another model layer.

0. Provider-direct base Agent (no AI SDK layer)

MyAgent.run [task]
  input:  triggering prompt/messages
  output: final operation response
  │
  └── Chat Completion [llm]
      input:  exact provider request messages
      output: canonical/provider-native model response
      metadata: { model, provider, ...allowlisted request configuration }
      metrics: { tokens, prompt_tokens, completion_tokens }

No synthetic streamText span should appear when the application did not call AI SDK.

1. Streaming AIChatAgent/Think turn without tools

Chat.onChatMessage [task]
  input:  [{ role: "user", content: "Summarize this document" }]
  output: { role: "assistant", content: "...final accumulated text..." }
  metrics: { tokens, prompt_tokens, completion_tokens }
  │
  └── streamText [task]
      │
      └── doStream [llm]
          input:  [system message, persisted conversation, triggering user message]
          output: [{ index: 0, finish_reason: "stop", message: {...} }]
          metadata: { model, provider, temperature?, tools? }
          metrics: { tokens, prompt_tokens, completion_tokens,
                     time_to_first_token, prompt_cached_tokens? }

The doStream span ends when the model stream ends; streamText and Chat.onChatMessage end only after the framework consumes/finalizes the response stream.

2. Parallel tool-use loop

Chat.onChatMessage [task]
  input:  [{ role: "user", content: "Compare Paris and Tokyo weather" }]
  output: { role: "assistant", content: "Paris is ...; Tokyo is ..." }
  metrics: aggregate of both LLM calls
  │
  └── streamText [task]
      │
      ├── doStream [llm]
      │   input:  [system, user]
      │   output: assistant tool_calls(get_weather Paris, get_weather Tokyo)
      │   metadata.tools: [get_weather JSON schema]
      │
      ├── get_weather [tool]          ┐
      │   input:  { location: "Paris" }  │ execute concurrently
      │   output: { temperature: 18 } │ when the framework does
      │                               │
      ├── get_weather [tool]          │
      │   input:  { location: "Tokyo" }  │
      │   output: { temperature: 25 } ┘
      │
      └── doStream [llm]
          input:  [system, user, assistant tool_calls,
                   Paris tool result, Tokyo tool result]
          output: assistant final response

Tool spans are siblings in the tree even when their wall-clock intervals overlap. Their order/timestamps must reflect actual execution.

3. Agent used as a tool / Durable Object sub-agent

Coordinator.onChatMessage [task]
  │
  └── streamText [task]
      ├── doStream [llm]
      │   output: assistant tool_call(research_agent)
      │
      ├── research_agent [tool]
      │   input: { topic: "Cloudflare Workers limits" }
      │   │
      │   └── ResearchAgent.onChatMessage [task]
      │       input:  [{ role: "user", content: "Research ..." }]
      │       output: { role: "assistant", content: "Research result" }
      │       │
      │       └── streamText [task]
      │           ├── doStream [llm]
      │           │   output: assistant tool_call(web_search)
      │           ├── web_search [tool]
      │           │   input:  { query: "..." }
      │           │   output: { results: [...] }
      │           └── doStream [llm]
      │               output: assistant research result
      │
      └── doStream [llm]
          input:  [..., research_agent tool result]
          output: assistant final response

The trace context must cross the Agent/Durable Object RPC boundary. The child agent's internal model/tool spans nest under its child task, while the model-facing delegation remains represented by the outer tool span.

4. Durable Object eviction and recovery

A JavaScript span object cannot survive isolate eviction. Do not hold an in-memory span open and pretend it resumed:

Invocation/isolate A
Chat.onChatMessage [task]                 (may be absent/incomplete after hard eviction)
  └── streamText [task]
      └── doStream [llm]                 (partial generation; isolate evicted)

Invocation/isolate B
Chat recovery [task]                     (new bounded AI operation)
  input:  recovered triggering message/context used for this attempt
  output: final recovered assistant response
  └── streamText [task]
      └── doStream [llm]                 (retry or continuation call)

Persisting and restoring a serialized Braintrust parent is preferable if it can be done safely and the parent row is guaranteed to exist; otherwise the recovery operation should be a new trace rather than a dangling child. Non-AI recovery bookkeeping (incident scans, transcript repair counts, alarm scheduling) should not create spans. Never mark an interrupted span successful or emit a fake final output.

Recommended public API

Cloudflare users subclass a Durable Object class, so an object/namespace proxy like wrapOpenAI() is not a good fit. The safest manual API appears to be a typed base-class mixin:

import { AIChatAgent } from "@cloudflare/ai-chat";
import { wrapCloudflareAgent } from "braintrust";

const BraintrustAIChatAgent = wrapCloudflareAgent(AIChatAgent);

// The concrete exported class keeps the exact name used by Wrangler bindings
// and Durable Object migrations.
export class Chat extends BraintrustAIChatAgent<Env> {
  async onChatMessage(_onFinish, options) {
    // Existing application code; no per-handler traced() boilerplate.
  }
}

The same helper should accept Agent and Think base classes. It should wrap instance lifecycle callbacks after super() while preserving generics, instanceof, method this, decorators, response/stream semantics, and the concrete exported class name.

A supplemental composable observability adapter would be useful for durable point events:

override observability = createBraintrustAgentsObservability({
  delegate: genericObservability,
});

However, the adapter cannot be the only integration: Cloudflare's current observability API emits synchronous point events and does not itself execute the user callback inside an active Braintrust context. It may use those events as control signals to start/finalize the allowed spans, but must not copy arbitrary event payloads into Braintrust. By itself it cannot reliably create a nested turn → model → tool span tree.

We should also provide bundle-time auto-instrumentation through the Braintrust Vite/Orchestrion plugin. Node loader hooks do not run in Workers. Manual and automatic paths should publish the same Braintrust tracing-channel contract and use the same plugin/span mapping.

Important API constraints:

• Do not require users to wrap the already-exported concrete class in a way that changes its class/export name; Wrangler class_name and Durable Object migrations depend on that identity.
• Do not reconstruct or eagerly buffer a streaming Response if doing so changes backpressure, cancellation, headers, or runtime branding.
• Compose with, rather than replace, genericObservability so Cloudflare Tail Worker events continue working.
• Setup must be idempotent per isolate.

Upstream observability and integration points

Cloudflare already has a useful public observability seam:

• agents/observability exports Observability, genericObservability, and a typed subscribe() helper.
• Events are published on native node:diagnostics_channel channels such as agents:message, agents:chat, agents:fiber, and agents:agent_tool.
• The diagnostics-channel implementation landed in e9ae070 and first shipped in agents@0.7.0; agent class/instance identity landed in 2cde136 and first shipped in agents@0.7.1.
• Current 0.17.x adds chat:turn:start / chat:turn:finish, fiber lifecycle, agent-tool recovery, request IDs, durations, and richer failure/recovery events.

There is also an active, unreleased upstream feat/agent-tracing branch (26 commits ahead of main when inspected) adding Workers-native custom spans, an AI SDK v6 wrapAISDK(), AI SDK v7 telemetry support, and out-of-box Think tracing. We should coordinate with that work and avoid creating two incompatible semantic schemas. Its native cloudflare:workers spans flow to Workers Observability/OTLP, not directly into a Braintrust project, so it does not remove the need for a Braintrust integration.

Version support recommendation

Cloudflare's packages are pre-1.0 and changing quickly, so avoid claiming one broad untested range.

1. Initial fully tested matrix
   • agents@0.17.x
   • @cloudflare/ai-chat@0.9.x
   • @cloudflare/think@0.13.x
   • ai@6.x (all current packages peer-depend on AI SDK v6)
2. Compatibility fixture / event-only floor, if inexpensive
   • agents@0.7.1 and @cloudflare/ai-chat@0.1.7, the first releases with native diagnostics channels plus agent identity
3. Forward support
   • Add the next agents minor that contains the active upstream native-tracing work.
   • Add AI SDK v7 only after Cloudflare publishes package versions that support it; do not infer v7 support from an unreleased branch.

agents/ai-chat-agent is no longer a usable implementation—it throws and directs users to @cloudflare/ai-chat—so tests and docs should use the separate package.

Cloudflare platform gotchas

• Use the workerd build. braintrust has a workerd conditional export / braintrust/workerd; the integration must not pull Node-only loader, filesystem, or process behavior into a Worker bundle.
• Native diagnostics-channel mismatch. Cloudflare Agents publishes through node:diagnostics_channel, while Braintrust's current workerd configuration uses dc-browser for its own tracing channels. Those are different registries. A Cloudflare plugin must subscribe to the native channels explicitly, while Braintrust-injected provider channels must consistently use the workerd-compatible registry (including the Vite plugin's useDiagnosticChannelCompatShim behavior). This needs an integration test, not an assumption.
• Vite plugin composition. Cloudflare projects commonly use both agents/vite and @cloudflare/vite-plugin. Verify Braintrust's Vite transform order against both, including decorated @callable() methods and transformed dependencies.
• nodejs_compat is required. Agents requires it; Cloudflare's diagnostics-channel docs also require it (with nodejs_compat_v2 enabled for compatibility dates on/after 2024-09-23).
• Isolate and Durable Object lifetime. Module state is per isolate, and an agent can be evicted during generation. Never depend on an in-memory open-span map surviving hibernation/deploy/OOM. Recovery should create a new bounded span. Restore a serialized Braintrust parent only when it is durably stored and known-valid; otherwise use a new trace instead of adding request/fiber/conversation metadata that the instrumentation guide does not allow.
• Async context. Preserve workerd AsyncLocalStorage context across await, stream consumption, tools, alarms, RPC, and sub-agent calls. Cloudflare has previously had duplicate-ALS problems under Vite SSR, so this needs real-runtime coverage.
• Flush lifetime. Braintrust uploads are outgoing I/O. Tie flush() to DurableObjectState.waitUntil() / the current invocation where needed so logs are not dropped after a response, WebSocket message, or alarm returns. Do not perform request-bound I/O at module initialization.
• Resource limits. Workers have 128 MB per isolate and six simultaneous outgoing connections per request. Avoid buffering full response streams or unbounded message/tool payloads, and account for Braintrust uploads sharing the outgoing-connection budget. waitUntil() only extends ordinary request work for up to 30 seconds after completion/disconnect.
• Tail Worker semantics. Native diagnostics messages are synchronously delivered in-isolate and also structured-cloned to Tail Workers. Keep emitted payloads cloneable and narrow. Tail Worker delivery is useful for platform diagnostics but cannot establish the original invocation's Braintrust async context after the fact.
• Privacy and cost. The Braintrust instrumentation guide requires model/agent inputs and outputs, so capture those through the normal Braintrust masking and attachment pipeline. Do not copy Cloudflare's entire Agent/DO state or arbitrary observability payloads, and do not retain raw inline media after attachment conversion.
• Two tracing systems may coexist. Workers native tracing is head-sampled and billable as observability events; Braintrust traces are separate. Document duplicate overhead and how to avoid exporting the same Cloudflare trace to Braintrust through OTLP while also using this SDK integration.

Acceptance criteria

• Manual wrapCloudflareAgent()-style API for Agent, AIChatAgent, and Think, with vendored minimal interfaces
• Bundle-time Vite/Orchestrion path sharing the same channel/plugin logic
• Parent agent-turn span with the exact data contract above and nested model/tool spans matching the rendered examples; include the AI SDK layer only when the application/framework actually uses it
• Correct streaming completion, cancellation, thrown errors, and in-stream errors
• Agent-as-tool/sub-agent coverage with Braintrust context propagation across Durable Object/RPC boundaries; tool-call IDs remain in standardized LLM payloads
• Recovery/eviction behavior does not leak spans or depend on in-memory state surviving
• genericObservability / Tail Worker events remain intact
• Tests in an actual workerd environment (@cloudflare/vitest-pool-workers or Wrangler), not Node-only mocks
• Fixtures for the supported package matrix and Vite plugin composition
• Documentation covering Wrangler flags, secrets, flushing, masking, sampling, and coexistence with Workers native tracing

References

• Braintrust instrumentation guide: https://github.com/braintrustdata/braintrust-spec/blob/main/docs/instrumentation-guide.md
• Cloudflare Agents docs: https://developers.cloudflare.com/agents/
• Upstream repository inspected: https://github.com/cloudflare/agents
• Agents observability docs/source: https://github.com/cloudflare/agents/blob/main/docs/agents/observability.md
• Cloudflare diagnostics channels: https://developers.cloudflare.com/workers/runtime-apis/nodejs/diagnostics-channel/
• Workers tracing/custom spans: https://developers.cloudflare.com/workers/observability/traces/
• Workers limits: https://developers.cloudflare.com/workers/platform/limits/
• Durable Object limits: https://developers.cloudflare.com/durable-objects/platform/limits/