Repository navigation
Conversation
Add a weaver registry under otel/ pinned to OTel semantic conventions v1.44.0. It registers the shared identity, PII and SDK keys and what the client core emits today under its current names, with Rego policies for ownership, sensitivity, PII naming, flat identity, units and metric cardinality. The lk.client.* namespace rule is advice that never fails. Every violation has a negative fixture that must fail with its finding.
Templates render SCHEMA_URL, attribute, enum value, event and span constants for Rust and TypeScript, and a markdown reference with owner, sensitivity, units and bindings. The outputs are committed; generate.sh reproduces them.
live-check.sh checks an OTLP/JSON capture against the registry, with matchers for spans, plain logs and the resource. On the reference client session every finding is one of the expected identity/PII renames. The workflow pins weaver 0.28.0 and runs the check, the negatives, the generated-output diff and compile, the live-check smoke test and the schema-version bump rule.
The per-file policy saw only v1 metrics and their direct refs, so a definition/2 metric in ms, or a metric inheriting lk.room_sid through extends, passed. It also skipped refs, so a ref could re-label lk.room_sid as owner bogus / sensitivity pii and keep that after resolution. Metric units and dimensions are now checked in an after_resolution policy on the effective groups, and a ref that overrides the annotations of an lk.* key is rejected from its lineage, in either file format. New negatives cover a definition/2 metric unit, definition/2 and extends cardinality, and v1 and definition/2 ref overrides.
livekit_unit matched duration words only at the end of a key, so lk.client.rtt.avg in ms with no unit_exception passed. A duration word followed by an optional _ms and one aggregation segment (.avg, .min, .max, _p99, ...) now counts. The RTC keys' unit_exception still applies; the new rtt-aggregate negative fails with livekit_unit.
RtcStatsSample's counters and gauges are Option and the core omits None, so a window can carry only the track, kind, direction, window and sample count. lk.rtc.bytes and lk.rtc.packets were required, which made such a window a live-check violation, and the RTT gauges were recommended. They are now conditionally required, and a sparse-window fixture must pass live-check with no findings.
schema-bump.sh compared raw manifest lines, so a comment or quotes on an unchanged URL, or a downgrade from 0.1.0 to 0.0.1, counted as a bump. It now reads schema_url as YAML does, parses its MAJOR.MINOR.PATCH and requires a strict increase whenever otel/model changed. schema-bump-test.sh exercises the rule in a throwaway repo (comment-only, downgrade, quoted, non-semver, patch/minor/major bumps) and runs in CI next to the PR check, which calls the same script.
The Rust and TypeScript templates wrote values verbatim, so an enum value with a quote, backslash or newline, or a brief with a newline (Rust ///) or */ (TS JSDoc), produced code that did not compile. Rust literals now escape backslash, quote, CR, LF and tab; TypeScript literals use tojson; doc comments are kept to one line and */ is broken up. codegen.sh generates a fixture with such a value and briefs, compiles both outputs and checks the value comes back byte for byte from rustc and tsc + node; it runs in CI. Committed TS output only switches to double quotes.
A span no matcher places gets no finding from weaver when it carries no attributes, so an unknown lk.* span passed silently. livekit_unknown_span now reports any lk.* span without a registry match; the live-check test asserts it. The README documents what remains unchecked (spans outside lk.*, span event names) and when CI runs.
Bring the registry to everything SPEC.md (a9ec8d52, with the stage-1 rules) and the JS port say about names and meaning. Every key, event and span now has a brief and a note, with W3C stats sources for RTC keys, platform sources for device keys and protocol enum names on members. Events carry their severity and body; spans their checkpoints (with the dynamic attempt pattern), outcome rules and caps; the custom namespace its reserved keys and limits. The resource, every-record and log sets say which keys go where. JS-only keys are registered as client-owned: browser.name/version and four lk.telemetry counters. Schema 0.2.0.
gen/docs/semconv.md now reads as the spec: the naming rules (flat shared identity, the lk.pii.* redaction contract, seconds with the documented RTC ms exception, keys the SDK owns, custom.* and its limits), the Resource, the attributes on every record, plain log records, each event with its severity, body and requirement table, each span with its kind, requirement table, checkpoints and limits, an attribute reference with where each key is used, enum tables, the bindings and the open naming decisions. Rust and TypeScript also get constants for browser.* and the checkpoint names. OTel refs used for logs get short notes of their own.
…heck The event and span annotations are now enforced on matched samples: livekit_severity flags an event sent at a severity it is not declared with, livekit_checkpoint a span event its span does not list (or that misses the attempt pattern), and livekit_span_outcome a span whose status disagrees with lk.outcome or that has error.type without an error outcome. The live-check test plants one of each. The reference session still reports only the six identity/PII renames.
The registry and the generated spec now read as the conventions as they are: the TBD, open-decision and pending-rename wording is gone from the model notes, the advice and live-check rules, the README and the docs, and the spec drops its Open decisions section. The names and choices stay as they were; the questions behind them are kept out of the spec. Schema 0.2.1 (documentation-only model change).
Weaver skips before_resolution policies with --v2 and hands after_resolution a different representation, so with --v2 (the mode generate.sh uses) no LiveKit rule ran: the metric, override and every per-file probe passed. The rules now live in policies/lib.rego. livekit.rego applies them per file in default mode; resolved.rego applies them to the resolved registry in both representations (default groups, --v2 registry and refinements), including metric refinements and ref overrides compared with each key's definition. check.sh checks both modes, negatives.sh runs every fixture in both, a refinement fixture is added, and generate.sh only replaces gen/ when every target passes.
The JS port sends lk.device.network.changed on every browser, and without the Network Information API it sends only network.connection.type (unknown or unavailable), so requiring lk.device.network.expensive and .constrained made that event a live-check violation, and the model called the event Chromium-only. The flags are now conditionally required, the event is described for all platforms, and a fixture of the JS shape must pass live-check. Schema 0.3.0 (requirement change; the README rule now names it).
An empty checkpoint name matched the fallback pattern ^$ on spans that declare no pattern; lk.outcome=ok with status Ok passed although the model requires Unset; an INFO-only event sent at severity number 17 with text INFO passed because only the text was compared. A checkpoint must now be listed or match an explicitly declared pattern, status must be exactly the one lk.outcome maps to, and the severity number must fall in a declared severity's range (the text, when present, is still checked). The live-check test plants one of each.
The spec stated the reserved keys, app limits and span caps as what producers do; they are requirements, and not every producer meets them yet, so they are now worded as rules (must) with no claim about a producer's behaviour. Gauge averages divide by the readings that reported the gauge, not by lk.rtc.samples. The RTC _ms keys carry milliseconds converted from the W3C stats' seconds; the spec no longer calls ms the W3C unit or names a rounding. The attribute reference gains a description column, so a key no signal references (lk.job_id) keeps its meaning.
severity_text is optional in OTLP. Omitted, it reaches the live-check rule as null, which passed the != "" test and was then not among the declared texts, so a valid INFO record (severity number 9, no text) was flagged. The text is now compared only when it is a non-empty string; the number is still checked. Two positive fixtures, as weaver samples and as OTLP/JSON lines, must pass live-check.
The spec and README said every RTC _ms key is converted from the W3C stats' seconds, but lk.rtc.window_ms is the window's length as the producer measures it. They now say so; key names and units are unchanged.
🦋 Changeset detectedLatest commit: d3f75f4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Adds otel/model/agents: the lk.* attributes LiveKit Agents 2.0 emits (Python trace_types.py, types.py, otel_metrics.py, observability.py and the Rust core), 50 spans as shipped (unprefixed), the 20 lk.agents.* metrics with their bare dimensions, and the gen_ai.*/LangFuse names no upstream registry defines. The OTel GenAI registry becomes a second dependency (pinned by commit, it has no tags) so the 46 current gen_ai.* keys are referenced, never redefined. Three keys that collide with a client spelling (lk.track_source, lk.disconnect_reason, lk.agent_name) are left out on purpose and stay literals in the producers until one spelling is picked. lk.rpc.caller_identity and lk.rpc.destination_identity carry a pii_heuristic_exception: shipped unmarked, documented rather than renamed. Codegen: a Python template (flat module of constants), an upstream section in the Rust and Python output for referenced OTel keys, a markdown template that renders spans without annotations, and the codegen test round-trips Python too. Schema 0.3.0 -> 0.4.0.
…lowlist Metric units are now exactly what Agents ships (nine counters unitless, unit: ""), and the Rust/Python output carries a _UNIT constant per metric so the producers can pin them. Every span-event name Agents passes to add_event is registered as a checkpoint on its span (21 names on 8 spans) and lk.pii.interim_transcript is declared. pii_heuristic_exception only takes effect for keys on the policy's own allowlist (lib.rego), with a negative fixture proving an annotated third key still fails. Schema 0.4.0 -> 0.5.0.
…r the Agents spelling Clients and Agents named three concepts differently. The Agents spelling and value set is now the shared key in otel/model/shared, and the client is the side that changes: - lk.track_source, values the protobuf TrackSource names (SOURCE_CAMERA, SOURCE_MICROPHONE, ...), replaces the client's lk.track.source (camera, microphone, ...) - lk.disconnect_reason, values the protobuf DisconnectReason names (CLIENT_INITIATED, UNKNOWN_REASON, ...), replaces the client's lk.disconnect.reason; RECONNECT_FAILED stays as the one client-only value - lk.agent_name is the shared agent key; the participant attribute lk.agent.name and the session report's bare agent_name are bindings The client spellings follow the pattern of the six identity/PII renames: not registered, rejected by livekit_shared_rename in the policies (new negative, both modes) and flagged by the same finding in live-check, which names the key to send. Client spans and the disconnect event now reference the shared keys; the Agents job, session and track-wait spans reference them too, with the participant_disconnected checkpoint added. On the reference client session live-check now reports 136 violations and 62 improvements (was 118 and 56), every one from the eight renames; with them applied it reports nothing. A new Agents fixture with the shared keys as Agents sends them goes from 5 findings to 0. Schema 0.5.0 -> 0.6.0.
…tever the deprecation Codex round 5. A deprecated old spelling slipped past livekit_shared_rename: the rule ran only on non-deprecated attributes, so defining lk.track.source as deprecated and requiring it from a span passed both registry-check modes and policy-gated generation. Each shared key now records the spelling it replaces in annotations.livekit.renamed_from: the old key for all eight renames (the six identity/PII keys, lk.track.source, lk.disconnect.reason) and, for the two enums, the map of old values to members (5 track source, 18 disconnect reason). The old spellings stay unregistered. - policies: livekit_shared_rename moves to resolved.rego, derives the map from renamed_from, and fires on any definition of an old spelling, deprecated or not, in both modes; livekit_rename_value rejects a mapped value that is not a member of the target enum. New negatives shared-rename-deprecated (Codex's probe) and rename-value, both modes. - live-check: live-check/renames.jq builds the map from the model; one livekit_shared_rename finding per old spelling names the key and value to send. The hard-coded identity, PII and shared maps are gone. The real session stays at 136 violations / 62 improvements. - docs: the spec lists every old spelling from renamed_from; the track source values are the Agents RTC FFI names, not the signal protocol's, so clients map explicitly; README drops a duplicated sentence. Schema 0.6.0 -> 0.6.1 (annotations and notes only).
check.sh always checks the default mode and --v2, so `otel/check.sh --v2` gave weaver --v2 twice and exited 2. A --v2 (or --v2=...) among the arguments is now dropped; other arguments still reach both checks, and plain `otel/check.sh` is unchanged.
pblazej
added a commit
to livekit/agents
that referenced
this pull request
Oct 20, 2026
…agent name keys Move the `protocol` submodule to 948b07e7 (livekit/protocol#1866), which registers lk.track_source, lk.disconnect_reason and lk.agent_name under the Agents spelling, and regenerate the vendored copies. trace_types now takes the three keys from `_semconv`, and the scan allowlist no longer exempts them. A new pin checks that every protobuf enum name room_io emits for the track source and disconnect reason is a registered value. The emitted strings do not change.
Agents still wrote five groups of telemetry names as literals. Four of them now have generated constants, with no wire change: - GenAI metrics: model/agents/groups.yaml imports gen_ai.invoke_agent.duration and gen_ai.execute_tool.duration from the GenAI registry, and the Rust and Python upstream sections list them with their units. Weaver 0.28.0 refuses to import gen_ai.client.operation.duration (semconv v1.44.0 keeps a deprecated copy excluded from dependency resolution), so signals.yaml declares it as emitted, next to the other GenAI metrics declared that way. - Values beyond upstream: a ref to an upstream key can list annotations.livekit.shipped_values. gen_ai.provider.name gains mistral_ai and gen_ai.response.finish_reasons its six values, each generated next to the upstream values. The new policy livekit_shipped_value checks refs and definitions in both modes. It rejects a list on an lk.* key or on a key that is not a string, string[] or all-string enum; an entry whose id is not lowercase snake_case (so the uppercase id is exactly the constant name); a value that upstream already lists; an id that repeats another entry or an upstream member id read as lowercase snake_case; and a second distinct list for the same key anywhere in the registry, since codegen merges them. Eight negative fixtures cover it, and the CI Rust compile is the backstop for any collision it misses. - Session report: the ten attributes Agents sends on its chat_history and evaluations log records are registered as emitted, owner agents; the three session.* keys carry upstream_namespace: session. The Rust log kv keys need no registry change: a parenthesized constant is a valid log 0.4 key, and the generated Rust header says so. README and the generated spec describe each mechanism. Schema 0.6.1 -> 0.7.0.
pblazej
added a commit
to livekit/agents
that referenced
this pull request
Oct 20, 2026
Move the `protocol` submodule to 41166317 (livekit/protocol#1866, schema 0.7.0) and regenerate the vendored copies. The three GenAI duration metric names and their units, the six finish reason values, the shipped `mistral_ai` provider value and the ten session report log-record keys now come from the generated constants in Python and Rust. The tool-argument repair warning takes its `lk.pii.*` keys as parenthesized constants, and its test looks them up by constant, so a bare identifier key fails it. Pins cover the newly bound strings, and every emitted string stays byte-identical.
No producer sets the OTLP scope schema_url: the Rust core, the JS core and Agents all leave it empty. A version that nothing emits only costs a bump in every model change, so the registry is unversioned for now. - model/manifest.yaml keeps the fixed placeholder https://livekit.io/schemas/telemetry/0.0.0, because Weaver requires a schema_url with a version segment. It does not change with the model. - The Rust, TypeScript and Python templates no longer generate SCHEMA_URL, the docs no longer print the schema URL, and no template filter passes schema_url. - tests/schema-bump.sh and tests/schema-bump-test.sh are gone, with their two CI steps and the full-history checkout only they needed. - The README replaces the bump rule with a Versioning section: the registry stays unversioned until a producer needs schema_url, and then the release step stamps the protocol release version, as OTel semantic-conventions does.
pblazej
added a commit
to livekit/agents
that referenced
this pull request
Oct 20, 2026
Move the `protocol` submodule to 25657ad3 (livekit/protocol#1866), which drops the `SCHEMA_URL` constant until a producer emits it, and regenerate the two vendored copies. Agents never referenced the constant, so nothing else changes.
pblazej
added this pull request to stack #1877
October 20, 2026 11:08
Weaver requires a schema_url and reads its last path segment as the version. It now points at the registry itself, https://github.com/livekit/protocol/tree/main/otel, instead of a livekit.io placeholder that never resolves. Producers do not emit it, and the link resolves once otel/ is on main. The generated code does not change: it no longer carries the schema URL.
`mage otel` runs otel/generate.sh next to `mage proto`. It is a separate, opt-in target: `mage proto` (the default) and the JS generation are unchanged and never need weaver, so only a change under otel/ does. The Generate workflow installs weaver 0.28.0 with the same pinned setup action as otel.yaml, runs `mage otel` after the protobuf and JS generation, and commits otel/gen with the rest, the same way it treats the generated protobuf code. Generate already runs on every push to a non-main branch with no path filter. The weaver setup and generation steps are continue-on-error: a weaver install failure, a registry failure or a network error while cloning the upstream registries leaves otel/gen untouched and never blocks the protobuf and JS commit. otel.yaml keeps its freshness check as the independent gate: a commit whose otel/gen is stale fails it. Generate's commit is pushed with the workflow token, so it does not run the checks by itself (GitHub can ask a maintainer to approve that run), the same as for the regenerated protobuf code. After a regeneration commit, push again to run the checks.
Contributor
|
It is a huge PR and it is impossible to review to some extend. can you break it down which parts of this PR the reviewers should pay attention to ? Or we should just let AI drive the development here and iterate from here ? |
| - {id: agent_name, type: string, stability: development, brief: "The agent's name.", examples: ["support-agent"], annotations: {livekit: {owner: agents, sensitivity: none}}} | ||
| - {id: sdk_version, type: string, stability: development, brief: "The Agents SDK version.", examples: ["2.0.0"], annotations: {livekit: {owner: agents, sensitivity: none}}} | ||
| - {id: usage, type: any, stability: development, brief: "Model usage, as a list of maps, one per model. Absent when there was none.", examples: ["[{\"model\": \"gpt-4o\", \"input_tokens\": 120}]"], annotations: {livekit: {owner: agents, sensitivity: none}}} | ||
| - {id: chat.item, type: any, stability: development, brief: "One chat history item, as a map. Carries conversation content.", note: "With redaction on, the collector drops the whole `chat.item` record.", examples: ["{\"type\": \"message\", \"role\": \"user\"}"], annotations: {livekit: {owner: agents, sensitivity: pii}}} |
Contributor
There was a problem hiding this comment.
is this chat.item used for conversation content ?
if yes, I'd think it should be lk.pii.chat_item, given that the conv content is high sensitivity.
Author
There was a problem hiding this comment.
It is conversation content, and the registry already marks it as PII, so the collector drops the whole record by this exact name when redaction is on. A rename would change what Agents sends, so f2831d4 keeps the name as the one exception to the PII marker rule.
| for target in rust:rust typescript:ts python:python markdown:docs; do | ||
| "$W" registry generate --v2 -r model -p policies -t templates "${target%%:*}" "$tmp/${target#*:}" | ||
| done | ||
| cp -R "$tmp"/. gen/ |
Contributor
There was a problem hiding this comment.
this seems to be a merge rather than replace ?
I think that will cause obsolete files if something gets deleted. and replace should be a better option ?
Author
There was a problem hiding this comment.
Agreed, fixed in 044450a. generate.sh now replaces otel/gen after every target passes, and CI now catches a deleted output.
| --input-source "$samples" --input-format json \ | ||
| --format json -o "$out" "${@:3}" > "$out/weaver.log" 2>&1 || status=$? | ||
| python3 -I "$here/live-check/summarize.py" "$out/live_check.json" | ||
| echo "report: $out/live_check.json" |
Contributor
There was a problem hiding this comment.
should we check the existence of live_check.son before doing the report ?
Author
There was a problem hiding this comment.
Fixed in 582b26c. Without a report, the script now prints Weaver's log and exits with Weaver's status.
| here=$(cd "$(dirname "$0")" && pwd) | ||
| W=${WEAVER:-weaver} | ||
| capture=$1 out=${2:-$(mktemp -d)} | ||
| mkdir -p "$out" |
Contributor
There was a problem hiding this comment.
should you empty the out folder before this line ?
Author
There was a problem hiding this comment.
Partly, fixed in 582b26c. The caller owns that folder, so the script deletes only the old live_check.json.
| - name: Generated outputs compile | ||
| run: | | ||
| rustc --edition 2021 --crate-type lib -D warnings -D missing_docs otel/gen/rust/semconv.rs --out-dir "$RUNNER_TEMP" | ||
| npx --yes -p typescript@5.9.3 tsc --strict --noEmit --target es2020 otel/gen/ts/semconv.ts |
Contributor
There was a problem hiding this comment.
curiously, only rust and ts ? do we need to check the python as well ? or python is using rust under the hood ?
Author
There was a problem hiding this comment.
Agents imports the generated Python directly, not through Rust. In e1d0b94 CI also runs the committed Python file.
| lk_attrs contains attr if { | ||
| some group in input.groups | ||
| some attr in group.attributes | ||
| startswith(attr.id, "lk.") |
Contributor
There was a problem hiding this comment.
I think it is fair to handle lk., but how about some other registry-owned keys ? for instance, like
agent_name, sdk_version, usage, chat.item, evaluation, tag, outcome, connection_reused, browser.name, browser.version
Author
There was a problem hiding this comment.
Agreed, fixed in f2831d4. The rules now cover every key in otel/model, and only chat.item needed an exception.
generate.sh copied the staged outputs over otel/gen, so a file that generation no longer writes stayed behind. It now removes otel/gen and moves the staged tree in, after every target succeeds. When a target fails, otel/gen stays unchanged. The CI freshness check also missed a deleted output: `git add --intent-to-add` stages the deletion, so `git diff` against the index stayed clean. The check now stages with `git add --all` and compares the index with the commit, so an obsolete output fails it. otel/tests/generate.sh runs generate.sh against a stub weaver: a failing target leaves otel/gen byte-identical, and a successful run removes an obsolete file and directory. CI runs it in the freshness step.
When weaver failed before it wrote a report, live-check.sh ran the summary anyway: the user got a Python traceback and exit 1, not weaver's error or exit status. When the out-dir held a report from an earlier run, the script printed that old report as this run's result. live-check.sh now deletes only the previous live_check.json before the run, because the out-dir can be the caller's and hold other files. If weaver writes no report, the script prints the weaver log and exits with weaver's status, or 1 if weaver exited 0. A valid report is summarized whatever weaver's status is. tests/live-check.sh adds four cases against a stub weaver: no report with status 7 and with status 0, a valid report with status 1, and a reused out-dir whose earlier report is not summarized while an unrelated file stays.
Agents imports the generated Python directly, but the compile step ran only rustc and tsc on the committed files. It now also runs `python3 -I -W error otel/gen/python/semconv.py`, the counterpart of `rustc -D warnings`: an invalid escape sequence fails it. The README uses the same command. The codegen step name now says that the step runs rustc, tsc and python3.
Every attribute rule, the ref-override rule and the client-namespace advice selected keys by the `lk.` prefix, so 23 keys defined in otel/model without that prefix skipped them. The rules now cover every attribute definition in otel/model, `lk.*` or not: a definition, not a ref, in the default mode, and every entry of the resolved registry with --v2. lk_key_shipped and the live-check rules keep the prefix. The ref-override rule covers every key with a definition here, in both modes, and takes that membership from the definition, not from the ref's copy. In the default mode, a definition's copy has no lineage entry and a ref's copy has one, so a ref that drops or nulls the owner, or empties the annotations, still fails. With --v2 the rule compares each ref's copy with the definition. Of the 23 keys, only chat.item broke a rule: it is PII without a `pii` key segment. The collector drops every record that carries chat.item by that exact name (pii-redaction/redact/otellogs.go:19), and Agents sends that name, so it keeps it: lib.rego names it in pii_by_name, used only by the pii_marker rule. The advice now also lists browser.name and browser.version. New negatives, each run in both modes: unprefixed-owner.yaml (an unprefixed key with no owner), unprefixed-pii.yaml (a bare key with sensitivity pii), and four refs that re-label a LiveKit key: ref-override-unprefixed.yaml (chat.item as non-PII), ref-override-drop-owner.yaml (chat.item without its owner), ref-override-empty-annotations.yaml (chat.item with no annotations) and ref-override-null-owner.yaml (lk.pii.room_name with a null owner). otel/model and otel/gen do not change.
otel/gen holds the generated Rust, TypeScript, Python and docs, about 6,200 of the PR's added lines. Marking it linguist-generated lets GitHub collapse it in the diff, as it already does for the generated JS and Go.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds machine-readable semantic conventions for LiveKit telemetry under
otel/. Semantic conventions are the agreed names, types and units of telemetry data. The registry uses the OTel Weaver format and builds on OTel semantic conventions v1.44.0 and the OTel GenAI registry.The registry covers the shared identity, PII and SDK keys, and everything that the client cores and LiveKit Agents emit today. When a client and Agents use different names for one concept, the shared key takes the Agents spelling, and the client changes. The generated file
otel/gen/docs/semconv.mdis the readable specification. The stability level isdevelopment.What this PR adds
otel/model/renamed_from. Two GenAI metrics come in from the GenAI registry throughimports:.otel/policies/lk.pii.*marker, seconds for durations, ownership and metric cardinality. The rules apply to every key thatotel/modeldefines, with or without thelk.prefix. They also reject a reference that changes the annotations of such a key, unknownlk.*spans, and any definition of an old spelling.chat.itemis the one named exception to thelk.pii.*marker rule, because the collector redacts it by that exact name. A reference to an upstream key can list values that LiveKit sends beyond upstream inannotations.livekit.shipped_values. A policy limits those values to snake_case ids on string keys, with one list per upstream key.otel/tests/--v2mode.otel/gen/.gitattributesmarks it as generated, so GitHub collapses it in the diff.otel/live-check/.github/workflows/otel.yamlotel/**or to the workflow.magefile.go,.github/workflows/generate.yamlmage otelregeneratesotel/gen/. The Generate workflow runs it next tomage proto..changeset/telemetry-conventions-registry.mdHow to regenerate the conventions
You need Weaver only when you change
otel/.mage protoand the JS generation do not run it.WEAVERto the path of the binary.otel/model/.mage otel. It runsotel/generate.sh, which replacesotel/gen/only when every target passes the policies.otel/check.shandotel/check.sh --v2.otel/model/andotel/gen/together.The Generate workflow installs Weaver and runs
mage otelon every push to a branch. Ifotel/gen/changes, the workflow commits it together with the regenerated protobuf code. That commit does not run the checks by itself, the same as for the protobuf code, so push again after a regeneration commit.otel.yamlruns the policies and the negative tests, and it fails whenotel/gen/differs from a fresh generation. It also compiles the generated Rust and TypeScript, and it runs the generated Python.Versioning
The registry has no version for now. Weaver requires a
schema_urlinotel/model/manifest.yaml, so the manifest points it at this registry,https://github.com/livekit/protocol/tree/main/otel. Weaver reads the last path segment,otel, as the version. The generated code has noSCHEMA_URLconstant, and no producer emits a schema URL. A model change does not need a version bump.When a producer needs to emit
schema_url, the release step can stamp the protocol release version into the URL. OTel semantic-conventions and OBI version their registries the same way. The OTel schema specification needs only aMAJOR.MINOR.PATCHnumber that increases, so a protocol version is valid. A commit hash is not valid, because it does not have that form and it has no order.How producers use the output
Producers copy the generated files into their repository and commit them. They do not run Weaver. livekit/agents#7973 is the first consumer, as a draft. It copies the Rust and Python constants into Agents 2.0 with
cargo run -p lk-agents-codegen -- --write, and nothing that Agents emits changes.The client SDKs do not use the constants yet. Each SDK end-to-end test already writes a capture, so each SDK can run live-check on its own output in CI. That check only reports until the client renames below land in that SDK.
Shared keys
Every producer must send these join keys under one name. In the table, Yes means that the producer sends the key today. No means that it sends a different key, which the table shows, or nothing.
Shared keys: who sends what today (11)
lk.room_sidlk.room.sidroom_id.room_idcolumn reads onlyroom_idandroomID, so client rows get''.lk.participant_idlk.participant.sidlk.track_sidlk.track.sidlk.job_idjob_id.otel_*columnlk.pii.room_namelk.room.name, never redactedpiisegment)lk.pii.participant_identitylk.participant.identity, never redactedlk.pii.remote_participant_identitylk.participant.remote_identitylk.sdk.name/lk.sdk.versionservice.name/service.versionsdk_versionon the session reportlk.track_sourceTrackSourcename (SOURCE_MICROPHONE)lk.track.source(microphone)track_subscribedcheckpointlk.disconnect_reasonDisconnectReasonname (CLIENT_INITIATED)lk.disconnect.reason(client_initiated)participant_disconnectedcheckpointlk.agent_namelk.agent.name.Two other keys join records today, but they are not shared. Only clients send
session.id, which equals the session trace id (decision 9). The OTLP trace id is the other one.Naming decisions
Each open decision ships an assumption for review. Decisions 12, 18 and 19 are resolved in favor of the Agents spelling. In these tables, Yes means that the producer sends the shipped key today, and No means that it does not. Different means that it sends another form, which the table shows.
Shared keys: decisions that bind Agents and clients (7)
lk.pii.remote_participant_identitylk.pii.publisher_identityroom_idcolumnlk.room_sid. The column is a binding.room_id, or the ingest readslk.room_sidroom_idon the resource and metricslk.track_source=SOURCE_MICROPHONEand the other Agents valueslk.track.sourcein snake_caselk.track.source=microphonelk.sdk.name/.version, open strings, recommendedservice.nameschema_urlpoints at this registry on GitHub, and no producer emits it (see Versioning).lk.disconnect_reason=CLIENT_INITIATEDand the other Agents values, plus the client-onlyRECONNECT_FAILEDlk.disconnect.reasonin snake_caselk.disconnect.reason=client_initiatedlk.agent_name, with the participant attributelk.agent.nameas a bindinglk.agent.nameClient keys: decisions on client-owned keys only (12)
lk.client.*namespacelk.device.*,lk.rtc.*). The rule is advice only.lk.rtc.*_mskeys in milliseconds, throughunit_exceptionlk.device.battery.level(%) and.charginghw.battery.chargeand.statelk.outcome= ok, error or cancelled on every spanlk.client.connection.resulton connect onlylk.connect.attempt, no connect idlk.client.connect_attemptandlk.client.connect_idattempt <n> <mode>namesession.idsession.id= the trace id, on every recordlk.session_idotel.event.nameevent_namefield.event_namenetwork.connection.typevaluesvpn,bluetoothandotheraddedunknowncustom.<name>, registered as one prefix eventbrowser.nameand.version, defined by LiveKituser_agent.*Client renames
Clients must make eight renames to conform to the registry. The SDK PRs track them:
lk.room.sid→lk.room_sidlk.participant.sid→lk.participant_idlk.track.sid→lk.track_sidlk.room.name→lk.pii.room_namelk.participant.identity→lk.pii.participant_identitylk.participant.remote_identity→lk.pii.remote_participant_identitylk.track.source→lk.track_source(microphone→SOURCE_MICROPHONE, and the other values)lk.disconnect.reason→lk.disconnect_reason(client_initiated→CLIENT_INITIATED, and the other values)Not in this PR
This PR adds no npm export for the generated TypeScript, because no SDK needs it yet. The JS SDK writes its telemetry names as literals, and Agents for Node takes its names through the Rust binding. The export can come as a separate entry, such as
@livekit/protocol/otel, when the JS SDK uses the generated constants. livekit/agents#7973 is the first consumer of the generated Rust and Python, as a draft.