feat(data-pipeline): add meta struct blob setter - #2303
Open
lloeki wants to merge 5 commits into
Open
Conversation
Expose a C setter that copies an opaque binary value into a span's `meta_struct` map. Native tracers can now supply pre-encoded MessagePack without transferring ownership or requiring Rust to interpret it. Reject null span handles and non-UTF-8 keys. Reusing a key replaces its value, and the copied bytes remain owned by the span after the call.
Contributor
📚 Documentation Check Results📦
|
Contributor
🔒 Cargo Deny Results📦
|
BenchmarksComparisonCandidateCandidate benchmark detailsBaselineBaseline benchmark details |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 6926f58 | Docs | Datadog PR Page | Give us feedback! |
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
Member
Author
|
Draft dd-trace-rb example consumer: DataDog/dd-trace-rb#6130. |
lloeki
force-pushed
the
lloeki/native-meta-struct-m1
branch
from
July 30, 2026 12:02
69926bd to
27ad1af
Compare
This was referenced Jul 30, 2026
ekump
reviewed
Aug 2, 2026
| }; | ||
| span.0 | ||
| .meta_struct | ||
| .insert(key, Bytes::copy_from_slice(value.as_bytes())); |
Collaborator
There was a problem hiding this comment.
value.as_bytes() can panic when there is invalid input (example: null pointer, non-zero length). You can use try_as_bytes instead to get a more meaningful error.
ekump
reviewed
Aug 2, 2026
| /// The `key` and opaque binary `value` are copied into the span. The value is | ||
| /// not interpreted or validated as MessagePack. | ||
| /// | ||
| /// Repeating a `key` appends a new entry rather than replacing the previous one. The last value |
Collaborator
There was a problem hiding this comment.
This exposes an internal implementation detail, so maybe shouldn't be here? I figure it's worth it since it could have performance implications for meta_struct
ekump
approved these changes
Aug 2, 2026
ekump
left a comment
ekump
left a comment
Collaborator
There was a problem hiding this comment.
LGTM but someone else should review since I also pushed commits
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated.
What does this PR do?
Adds an opaque per-key
meta_structblob setter toTracerSpan. Keys are validated as UTF-8, while values are copied as uninterpreted binary data. This transitional API lets consumers pass their existing per-key MessagePack blobs without changing the Rust trace model.Motivation
Native exporter consumers cannot currently populate
meta_struct, which blocks payload parity tracked by APMSP-3219. The opaque setter provides the safe first stage before the language-neutral structured-value APIs.Additional Notes
Stack position: L1, current PR: DataDog/libdatadog PR 2303.
Complete expected review and merge order for one libdatadog release:
L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. L4 and L5 remain drafts while their prerequisite APIs are reviewed and rebased; their matching consumer PRs are also drafts pending the single libdatadog/libdatadog-rb release.
L0 tracks APMSP-3830; L1-L5 implement the payload work tracked by APMSP-3219.
How to test the change?
Prior local validation completed successfully:
nix develop -c cargo test -p libdd-data-pipeline-ffipassed, including arbitrary binary values, overwrite semantics, null handles, and invalid UTF-8 keys.nix develop -c cargo clippy -p libdd-data-pipeline-ffi --all-targets -- -D warningspassed.nix develop .#nightly -c cargo fmt --all -- --checkpassed.nix develop -c cargo run --bin release -- --out /tmp/opencode/native-meta-struct-libdatadog/x86_64-linuxand consumed successfully by the matching dd-trace-rb branch.nix develop .#ruby34 --command bundle exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/conformance_spec.rb spec/datadog/tracing/transport/native/transport_spec.rb spec/datadog/appsec/actions_handler/serializable_backtrace_spec.rbpassed 116 examples, including AppSec and AI Guard-shaped wire conformance.