Skip to content

telemetry: generate span, event and metric names from the protocol registry - #7973

Open
pblazej wants to merge 10 commits into
agents-2.0from
blaze/otel-semconv-agents
Open

pblazej wants to merge 10 commits into
agents-2.0from
blaze/otel-semconv-agents

Conversation

@pblazej

@pblazej pblazej commented Oct 20, 2026 •

Copy link
Copy Markdown

Summary

Agents 2.0 now takes all of its telemetry names from the semantic conventions registry in livekit/protocol#1866. Semantic conventions are the agreed names, types and units of telemetry data. Every span name, event name, attribute key, enum value, metric unit and log key comes from generated constants. Only the lk.sip. prefix stays a literal, because it is a prefix and not a key.

The emitted telemetry is byte-identical to agents-2.0. Pin tests in Rust and Python freeze each value. Two scans fail when a new literal name appears at an emitter site.

Protocol change

This PR depends on three stacked protocol PRs: livekit/protocol#1866 (the registry), livekit/protocol#1872 (TypeScript values grouped by attribute) and livekit/protocol#1881 (the same grouping for Python). The protocol submodule points at 8b2aa35d, the head of #1881. All three must merge first. After they merge, move the submodule to the merged commit and regenerate the copies as the next section describes.

How Agents regenerates the conventions

Agents does not run Weaver. lk-agents-codegen copies the generated files from the submodule to rust/crates/lk-agents-abi/src/telemetry/semconv.rs and livekit-agents/livekit/agents/_semconv.py, next to the host views. To change a name, change it in livekit/protocol first, as otel/README.md there describes. Then do these steps:

  1. Move the protocol submodule to the new protocol commit.
  2. From rust/, run cargo run -p lk-agents-codegen -- --write.
  3. Commit the submodule pointer and the two copies together.

Do not edit the copies by hand. The next --write overwrites them. cargo run -p lk-agents-codegen -- --check exits 1 when a copy differs from the submodule.

Local testing

git submodule update fetches 8b2aa35d from the #1881 branch, so a normal checkout works.

git fetch origin blaze/otel-semconv-agents && git switch blaze/otel-semconv-agents
git submodule update --init protocol
(cd rust && cargo test -p lk-agents-abi semconv_pins)
PYTHONPATH="$PWD" uv run pytest tests/test_semconv_pins.py
(cd rust && cargo run -p lk-agents-codegen -- --check)

If your default Rust toolchain is older than 1.95, uv run fails while it rebuilds the extension. Use uv run --no-sync, or set RUSTUP_TOOLCHAIN=1.96.0. The last command exits 1 and names only _generated/storage.py, which is already stale on agents-2.0.

Known gaps

  • The ruff and type-check jobs fail on files that this PR does not change, and they fail the same way on agents-2.0. This PR fixes only the missing protocol checkout in the type-check and realtime jobs.
  • No workflow runs lk-agents-codegen --check.
  • Live-check cannot validate an Agents capture yet. The registry matches spans by their lk.* names, and Agents span names have no prefix.
  • lk.speech_id is a string in Python and a number in the Node binding. The registry types it as a string.

Move the `protocol` submodule to 59034411 (livekit/protocol#1866), which
generates the telemetry semantic conventions under otel/gen. `cargo run -p
lk-agents-codegen -- --write` copies the Rust and Python files into
lk-agents-abi and the Python package. `--check` reports a stale copy like a
stale host view. The agents-rust layout gets no copy.
Span names, event names, attribute keys, enum values and metric units now
come from the vendored semconv constants in Rust and Python. Every emitted
string stays byte-identical. Pin tests freeze each value, and one scan per
language rejects a new literal name at an emitter site.
…agent name keys

Move the `protocol` submodule to 948b07e7 (livekit/protocol#1866), which
registers lk.track_source, lk.disconnect_reason and lk.agent_name under the
Agents spelling, and regenerate the vendored copies. trace_types now takes
the three keys from `_semconv`, and the scan allowlist no longer exempts
them. A new pin checks that every protobuf enum name room_io emits for the
track source and disconnect reason is a registered value. The emitted
strings do not change.
types.py kept lk.simulation.enabled, lk.simulation.run_id,
lk.simulation.job_id and lk.redaction.enabled as literals because importing
the telemetry package from it is circular. The vendored `_semconv` module
imports nothing, so it moves from livekit.agents.telemetry to livekit.agents,
where types.py can import it without loading the telemetry package.
lk-agents-codegen writes the copy to the new path, and the scan allowlist for
types.py now lists its non-telemetry keys instead of exempting the file. The
emitted strings do not change.
`uv sync` builds the package, and lk-agents-storage reads
protocol/livekit/agent/thread_schema_v1.sql at build time. Without the
submodule the build fails before mypy runs.
`uv sync` builds the package, and lk-agents-storage reads
protocol/livekit/agent/thread_schema_v1.sql at build time. Without the
submodule the build fails before the tests run.
Move the `protocol` submodule to 41166317 (livekit/protocol#1866, schema
0.7.0) and regenerate the vendored copies. The three GenAI duration metric
names and their units, the six finish reason values, the shipped
`mistral_ai` provider value and the ten session report log-record keys now
come from the generated constants in Python and Rust. The tool-argument
repair warning takes its `lk.pii.*` keys as parenthesized constants, and its
test looks them up by constant, so a bare identifier key fails it. Pins
cover the newly bound strings, and every emitted string stays
byte-identical.
Move the `protocol` submodule to 25657ad3 (livekit/protocol#1866), which
drops the `SCHEMA_URL` constant until a producer emits it, and regenerate
the two vendored copies. Agents never referenced the constant, so nothing
else changes.
@pblazej
pblazej marked this pull request as ready for review October 20, 2026 11:54
@pblazej
pblazej requested a review from a team as a code owner October 20, 2026 11:54

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Move the `protocol` submodule to 8b2aa35d (livekit/protocol#1872), which
groups each attribute's Python values under a class with the TypeScript
const-object name and removes the 141 flat value constants. Regenerate the
Python copy; the Rust copy does not change. The trace_types aliases keep
their names and values and now read the class members, and the pin test
looks up the track source and disconnect reason values the same way. Every
exposed string stays byte-identical.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant