feat(otel-telemetry): scaffold libdd-otel-telemetry crate - #2292
Draft
mabdinur wants to merge 7 commits into
Draft
feat(otel-telemetry): scaffold libdd-otel-telemetry crate#2292mabdinur wants to merge 7 commits into
mabdinur wants to merge 7 commits into
Conversation
Adds a new crate providing shared OpenTelemetry metrics aggregation and OTLP export, built on the upstream opentelemetry_sdk/opentelemetry-otlp crates. The public API exposes only primitives (opaque instrument ids, numeric values, string attributes) so any dd-trace-xx language binding can consume it without depending on OTel SDK types itself. Logs support and the counting-exporter wrapper for export_counters() are follow-ups.
Contributor
📚 Documentation Check Results📦
|
Contributor
🔒 Cargo Deny Results📦
|
Contributor
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 9b08d7d | Docs | Datadog PR Page | Give us feedback! |
5 tasks
The new crate's files had no owner, failing the "Not Owned File Checker". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
BenchmarksComparisonBenchmark execution time: 2026-07-31 17:24:06 Comparing candidate commit 9b08d7d in PR branch Found 20 performance improvements and 9 performance regressions! Performance is the same for 113 metrics, 0 unstable metrics.
|
mabdinur
added a commit
to DataDog/dd-trace-py
that referenced
this pull request
Jul 29, 2026
Adds a native module (TelemetryAggregatorBuilder / TelemetryAggregator) wrapping libdatadog's new libdd-otel-telemetry crate, exposing OTel metrics aggregation + OTLP export to Python over a primitives-only boundary (opaque instrument ids, numeric values, string attr pairs). All libdatadog deps point at the munir/otel-telemetry-libdatadog-v37 branch, which is the v37.0.0 tag plus only the additive libdd-otel-telemetry crate (DataDog/libdatadog#2292). Every other crate on that branch is byte-identical to v37.0.0, so no unrelated changes are pulled in, and a single git ref avoids duplicate copies of shared crates that would break trait bounds across the graph. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ggregator The type is OTel-metrics-specific (SdkMeterProvider + PeriodicReader + OTLP metric exporter), but "TelemetryAggregator" read as generic and collided with the tracer-telemetry concept. Rename to make the scope explicit and leave room for per-signal siblings (e.g. OtelLogs*) later: TelemetryAggregator -> OtelMetricsAggregator TelemetryAggregatorBuilder -> OtelMetricsAggregatorBuilder TelemetryAggregatorError -> OtelMetricsError Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mabdinur
added a commit
to DataDog/dd-trace-py
that referenced
this pull request
Jul 29, 2026
…tricsAggregator Follows the libdatadog rename (DataDog/libdatadog#2292): the native classes are OTel-metrics-specific, so make the names say so. Also drop the custom TelemetryAggregatorInternalError exception (nothing caught it by reference) and raise a plain RuntimeError from force_flush/shutdown instead. TelemetryAggregator{,Builder} -> OtelMetricsAggregator{,Builder} TelemetryAggregatorInternalError -> removed (RuntimeError) Bumps the libdatadog dep to the renamed v37-based branch commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Bump opentelemetry/opentelemetry_sdk/opentelemetry-otlp from 0.31 to 0.32 so the exporter matches dd-trace-rs's SDK version. Add DatadogMetricExporter, a PushMetricExporter that wraps the OTLP MetricExporter and tracks export attempts/successes/failures (centralized equivalent of dd-trace-rs's old TelemetryTrackingExporter), plus build_datadog_metric_exporter to construct it. Refactor the existing build_metric_exporter and Counters to be shared. The in-process OtelMetricsAggregator is unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…uild Under --all-features the `http` feature is on, so http/protobuf is a *supported* protocol and the aggregator builds a real reqwest-backed exporter — which, under opentelemetry-otlp 0.32 + workspace feature unification, eagerly builds a rustls client and panics with "No provider set". The test's scenario (unsupported protocol -> warning, not panic) only exists without the http feature, so gate it there. A positive http-exporter test needs a crypto provider installed (TODO). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The aggregator registered every instrument on one hardcoded meter
("libdd-otel-telemetry"), so exported OTLP metrics lost the host's
instrumentation scope — all metrics reported scope name "libdd-otel-telemetry"
and distinct meters collapsed into one scope. Carry the meter name/version/
schema_url on InstrumentDescriptor and create/cache one SDK Meter per scope so
exports keep the host's get_meter identity.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… + fix http TLS Add canonical, pure config-parsing helpers to `config.rs` so every consumer (dd-trace-rs, dd-trace-py via PyO3) shares one implementation instead of hand-rolling case-sensitive matches: - `OtlpProtocol::from_config_str` accepts `grpc`/`http/protobuf`/`http/json` case-insensitively (adds an `HttpJson` variant for parity with dd-trace-rs; it reports as an unsupported export protocol like before). - `Temporality::from_config_str` accepts `delta`/`cumulative` case-insensitively, defaulting to `Delta` for empty/unknown. - `parse_otlp_headers` parses `k1=v1,k2=v2` header strings. Also fix the `http` exporter: reqwest+rustls 0.23 panics with no process-default crypto provider under feature unification. Best-effort install the ring provider before building the http exporter (ring only; aws-lc-rs stays FIPS-only). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First step towards centralizing OpenTelemetry metrics (and, in a follow-up, logs) aggregation + OTLP export into libdatadog, so
dd-trace-xxlanguage libraries can stop depending on the OpenTelemetry SDK package and keep only the lightweight API package.This PR scaffolds a new
libdd-otel-telemetrycrate:opentelemetry_sdk/opentelemetry-otlpcrates internally (reuses real aggregation, temporality, and OTLP encoding — nothing hand-rolled).register_instrument-> opaqueInstrumentId), then push numeric values + string attribute pairs (record_counter,record_histogram,observe_gauge, ...). No SDK types, trait objects, or closures cross the boundary in either direction, so any consumer (Python via PyO3, eventually Node/Ruby/PHP/Rust via FFI) only ever touches primitives.grpc(tonic, default) andhttp(reqwest/protobuf) OTLP transport features, matching the split already used elsewhere in the org (e.g. dd-trace-rs's OTel integration).BuildWarningand the aggregator falls back to a no-op exporter, since a broken OTel pipeline must never block a tracer from starting.libdd-shared-runtime'sBlockingRuntimefor exporter construction, consistent withlibdd-data-pipeline.Scope: metrics only for now; logs support and a counting-exporter wrapper for
export_counters()(currently a stub returning zeros) are explicit follow-ups. No-ffiC-ABI crate yet — deferred until there's a concrete non-Rust/non-PyO3 consumer.Next: a companion draft PR against
dd-trace-pyadding a PyO3 wrapper around this crate.Test plan
cargo check -p libdd-otel-telemetry(defaultgrpcfeature)cargo check -p libdd-otel-telemetry --no-default-features --features httpcargo +stable clippy -p libdd-otel-telemetry --all-targets -- -D warnings(both feature sets)cargo +nightly-2026-02-08 fmt --all -- --checkcargo nextest run -p libdd-otel-telemetry(both feature sets)LICENSE-3rdparty.csvregenerated via./scripts/update_license_3rdparty.sh